### Sarah Dean PRO

asst prof in CS at Cornell

Lydia T. Liu, **Sarah Dean**, Esther Rolf, Max Simchowitz, and Moritz Hardt

*University of California, Berkeley*

“21 definitions of fairness” [Narayanan 2018]

- Demographic Parity
- Equality of Opportunity
- Predictive value parity
- Group Calibration

. . .

**Many fairness criteria can be achieved individually using efficient algorithms***post-processing*[e.g. Hardt et al. 2016];*reduction to black-box machine learning*[e.g. Dwork et al. 2018; Agarwal et al. 2018];*agnostic learni*ng [e.g. Kearns et al. 2018; Herbert-Johnson et al. 2018];*unconstrained machine learning*[Liu et al. 2018b]**But typically impossible to satisfy simultaneously**[e.g. Kleinberg et al. 2016; Chouldechova, 2017]

**?**

Two groups with different score distributions (e.g. credit scores)

Approve loans according to **Demographic Parity**.

Credit scores change with repayment (+) or default (-).

Harm!

Fairness criteria didn’t seem to *help* the protected group,

once we considered the *impact* of loans on scores.

- Introduce the “
**outcome curve**”, a tool for comparing the delayed impact of fairness criteria - Provide a
**complete characterization**of the delayed impact of different fairness criteria - Show that fairness constraints
**may cause harm**to groups they intended to protect

- A score \(R\) is a scalar random variable corresponding to
*success probability*if*accepted*

- Scores of accepted individuals change depending on their success.
- The average change in score of each group is the
**delayed impact**\(\Delta \mu\)

- Institution
*accepts*individuals based on threshold \(T\), chosen to maximize expected utility

**Lemma**: \(\Delta\mu\) is a concave function of acceptance rate \(\beta\) under mild assumptions.

average score change

Alternative to unconstrained utility maximization

**Demographic Parity**: Equal Acceptance Rate

**Equal Opportunity**: Equal True Positive Rates

**All outcome regimes are possible**

**Result 1**

Equal opportunity and demographic parity may cause relative improvement, relative harm, or active harm.

Unconstrained utility maximization never causes active harm.

**Result 2**

Demographic parity (DP) may cause active or relative harm by **over-acceptance**; equal opportunity (EO) doesn't.

**Result 3**

Equal opportunity may cause relative harm by **under-acceptance**; demographic parity never under-accepts.

**Choice of Fairness Criteria Matters**

**Experiments on FICO Credit Scores**

- 300,000+ TransUnion TransRisk scores from 2003

- Estimate score distributions and repayment probabilities
- Model bank’s profit/loss ratio as +1:-4
- Model impact of repayment and default on credit score as +75 and -150

Why the large difference in delayed impact?

Maxima of outcome and utility curves under fairness criteria are more **misaligned** in the minority black group

- Outcome curves provide a way to deviate from maximum utility while improving outcomes
- Need for
*domain-specific*models of delayed impact -
*Context-sensitive*nature of fairness in machine learning

- Measuring and modeling outcomes directly
- Practical tradeoffs of using machine learning for human-centered policies
- Long term dynamics of the distributional outcomes of algorithmic decisions [Hu and Chen 2018; Hashimoto et al. 2018; Mouzannar et al. 2019]

Details in full paper:

**https://arxiv.org/abs/1803.04383**

Algorithmic decisions are everywhere

- A score \(R\) is a scalar random variable, e.g. credit scores 300-850
- A group has a distribution over scores:

- Scores correspond to
*success probabilities*(e.g. likelihood of repaying a loan) if*accepted* - Higher score implies higher probability of success

- Institution
*accepts*individuals based on threshold \(T\), chosen to maximize expected utility:

- When there are multiple groups, thresholds can be group-dependent.

\(\mathbb{E}[\mathrm{utility}|T] = \mathbb{E}[\mathrm{reward~from~repayments}|T] - \mathbb{E}[\mathrm{loss~from~defaults}|T]\)

- Scores of accepted individuals change depending on their success.

- The average change in score of each group is the
**delayed impact**:

\(R_\mathrm{new} = \begin{cases} R_\mathrm{old}+c_+ &\text{if repaid} \\ R_\mathrm{old}-c_- &\text{if defaulted} \end{cases}\)

\(\Delta \mu = \mathbb{E}[R_\mathrm{new}-R_\mathrm{old}]\)

**Measurement Error Increases Potential for Improvement**

**Result 4**

If scores are systematically *underestimated* in the protected group, then regime of relative improvement is widened

- 300,000+ TransUnion TransRisk scores from 2003
- Scores range from 300 to 850 and are meant to predict default risk

What we did

- Use empirical data labeled by race (“white” and “black”) to estimate group score distributions, repayment probabilities, and relative sizes
- Model the bank’s profit/loss ratio, e.g. +1:-4
- Model the delayed impact of repayment/default on credit score, e.g. +75/-150
- Compute “outcome curves” and delayed impact under different fairness criteria

**Experiments on FICO Credit Scores**

By Sarah Dean

- 688