Ishanu Chattopadhyay
Assistant Professor of Medicine
University of Chicago
DATA
AI
ETHICS
Friday the 13th, October 2023
Legal framework for ensuring that machine learning technologies
protect fairness, transparency, privacy, human safety, confidentiality of personal records.
AI governance
Fair housing laws prohibit financial officials from making loan decisions based on race, gender, and marital status in their assessments.
Yet AI designers inadvertently can find proxies that approximate these characteristics and therefore allow the incorporation of information about protected categories without the explicit use of demographic background.
Autism prevalence is less among Black and Hispanic children.
It is not clear if this under-diagnosis due to biases in heathcare access, or there is an underlying biological mechanism
Tools to make diagnosis more efficient might reflect this same "inequity".
Ian Cero, Peter A. Wyman, I. Chattopadhyay, Robert D. Gibbons, Predictive equity in suicide risk screening, Journal of the Academy of Consultation-Liaison Psychiatry, 2023. https://doi.org/10.1016/j.jaclp.2023.03.005
AI
Equity
&
Fairness
Helping patients with suicidal ideation
Suicide is a major public health concern
1 death by suicide every 40 seconds
As per the data from the CDC, in 2019, there were over 47,500 suicide deaths in the U.S., with an age-adjusted rate of 13.9 per 100,000 individuals.
10th leading cause of death in the United States
Screening Tests are Increasingly common
Columbia-Suicide Severity Rating Scale (C-SSRS)
Patient Health Questionnaire-9 (PHQ-9)
Ask Suicide-Screening Questions (ASQ)
These screening tools are not meant to be diagnostic but rather to help identify individuals who may need further evaluation or intervention to prevent suicide.
Primary Care
Emergency Dept
School & Community
The increasing standardization of suicide risk screening suggests predictive models balance not only accuracy, but also fairness for the different groups of people whose futures are being predicted
Accuracy
Fairness
Group A
Group B
Ask Suicide-Screening Questions (ASQ) has high and equivalent sensitivity and specificity for suicide ideation across black and white youth in the emergency department.
Black
Sensitivity
Specificity
Non-Hispanic White
Equal across groups
ASQ
Different Base rates (prevalence)
6.11 per 100,000*
15.68 per 100,000*
Non-Hispanic White
Black
*CDC 2019 Data
Uneven base rates
Mathematically unavoidable trade-off between model accuracy and fairness
Predictive disparity is likely caused by uneven base rates on the outcome being predicted*
UCM Data
Blacks
Non-Hispanic Whites
AUC~90%
AUC~88%
Universal SCreening for Suicidal Ideation / Attempts
UCM Data
Universal SCreening for Suicidal Ideation / Attempts
UCM Data
Universal SCreening for Suicidal Ideation / Attempts
15
Assume you have $1,000,000 to allocate to the post-screening followup service
67%
33%
25
Number of actual individuals helped
Demographic breakdown at UCM
=40
9
Assume you have $1,000,000 to allocate to the post-screening followup service
44%
66%
49
Number of actual individuals helped
Demographic breakdown at UCM
+
Differential
base
rate
=58
Race-blind followup
17
Assume you have $1,000,000 to allocate to the post-screening followup service
77.5%
22.5%
17
Number of actual individuals helped
Equal outcome
allocation
=34
The Ethics Question
Distribute resources race-blind
Distribute resources to make equal outcomes
Lives saved
58
34
The new frontier of predictive fairness in suicide prediction
Crime Prediction
Predictive Technologies in Policing
Performance verified in 8 other US cities
Chicago, LA, Philadelphia, San Francisco, Detroit, Austin, Portland, Atlanta
Rotaru, V., Huang, Y., Li, T. et al.
Event-level prediction of urban crime reveals a signature of enforcement bias in US cities. Nature Human Behavior 6, 1056–1068 (2022).
For every 10 crimes,
11 flags, 3 false, 2 missed
(1 week advance, with 2 city blocks)
Infer cross-dependencies at different spatial and temporal scales
Signature of enforcement
inequity
Results corroborated in signature observed in raw data
The Problem of Free Will
3 day ahead prediction
Jan 1 2019
to
April 1 2019
Play Movie
Triangles: actual events
heatmap: predicted risk 3 days ahead
Triple homicide incident
Jan 7 2019
https://www.inquirer.com/crime/kensington-triple-shooting-homicide-philadelphia-police-20190107.html
Triangles: actual events
heatmap: predicted risk 3 days ahead
Question:
How do we effectively leverage AI for social good?