Ishanu Chattopadhyay

Assistant Professor of

Data Science / AI / Informatics

Data, ethics, and biases of predictive policing with emerging AI

Rotaru, V., Huang, Y., Li, T. et al. 

Event-level prediction of urban crime reveals a signature of enforcement bias in US cities. Nature Human Behavior 6, 1056–1068 (2022).

https://doi.org/10.1038/s41562-022-01372-0

Data, ethics, and biases of predictive policing with emerging AI

Rotaru, V., Huang, Y., Li, T. et al. 

Event-level prediction of urban crime reveals a signature of enforcement bias in US cities. Nature Human Behavior 6, 1056–1068 (2022).

https://doi.org/10.1038/s41562-022-01372-0

"You're a real sack of orwellian $xxt. I hope the jackbooted thugs to which you're providing this crime forecast AI never turn it on anyone you hold dear. You're, quite simply, a vile, oppression enabling worm."

The Problem of Free Will

Predictive Policing

Broad patterns are easily predictable

  • Predictions cannot be self-fulfilling prophecies
  • Transparency
  • Account for interactions between crime, social factors and enforcement

Ethically Acceptable Predictive Algorithm For  Enforcement?

The Prediction Problem:

  • Predicting crime sufficiently ahead of time to be actionable
  • Prediction precise enough in time and space to be actionable
  • Use ONLY data that is realistically and cheaply available

1 Week in advance

Within ~2 city blocks

ONLY Past eventlog as input

  • Simulate impact of different policy choices
  • Reveal policy biases

A good predictive model gives us the following fro free:

Predict Hotspots?

Predpol

2 weeks, Chicago, 2017

property crime

violent crime

complex urban topology

Some historical patterns are predictive

Data: Event Log

ID,Case Number,Date,Block,IUCR,Primary Type,Description,Location Description,Arrest,Domestic,Beat,District,Ward,Community Area,FBI Code,X Coordinate,Y Coordinate,Year,Updated On,Latitude,Longitude,Location\\
8316800,HT550945,08/11/2011 11:00:00 AM,086XX S MARQUETTE AVE,1120,DECEPTIVE PRACTICE,FORGERY,RESIDENCE,false,false,0423,004,7,46,10,1195654,1848294,2011,02/04/2016 06:33:39 AM,41.738615478,-87.558741896\\
8316805,HT550781,10/20/2011 05:00:00 AM,056XX S ABERDEEN ST,0890,THEFT,FROM BUILDING,RESIDENCE,false,false,0712,007,16,68,06,1169943,1867457,2011,02/04/2016 06:33:39 AM,41.791797599,-87.652385205\\
8316806,HT550706,10/20/2011 05:45:00 AM,079XX S LOOMIS BLVD,031A,ROBBERY,ARMED: HANDGUN,STREET,false,false,0612,006,21,71,03,1168370,1852331,2011,02/04/2016 06:33:39 AM,41.750323974,-87.658588247\\
8316811,HT539324,10/12/2011 12:23:52 PM,003XX E 75TH ST,2027,NARCOTICS,POSS: CRACK,SMALL RETAIL STORE,true,false,0323,003,6,69,18,1179641,1855355,2011,02/04/2016 06:33:39 AM,41.758372192,-87.61719416\\
8316822,HT551031,10/19/2011 02:00:00 AM,071XX W DICKENS AVE,0910,MOTOR VEHICLE THEFT,AUTOMOBILE,SIDEWALK,false,false,2512,025,36,25,07,1127877,1913161,2011,02/04/2016 06:33:39 AM,41.918027518,-87.805606689\\
8316824,HT551032,10/20/2011 12:00:00 AM,034XX N NATCHEZ AVE,2825,OTHER OFFENSE,HARASSMENT BY TELEPHONE,RESIDENCE,false,false,1632,016,36,17,26,1132429,1922272,2011,02/04/2016 06:33:39 AM,41.94295109,-87.788669409\\
8316825,HT549690,10/19/2011 12:51:00 PM,079XX S ADA ST,2820,OTHER OFFENSE,TELEPHONE THREAT,APARTMENT,false,false,0612,006,21,71,26,1168711,1852015,2011,02/04/2016 06:33:39 AM,41.749449482,-87.657347764\\
8316826,HT549865,10/19/2011 06:00:00 AM,011XX N LEAMINGTON AVE,0810,THEFT,OVER \$500,RESIDENTIAL YARD (FRONT/BACK),false,false,1531,015,37,25,06,1141821,1907155,2011,02/04/2016 06:33:39 AM,41.9012995,-87.754523767\\
8316827,HT550963,09/01/2011 04:00:00 PM,079XX S LOOMIS BLVD,0610,BURGLARY,FORCIBLE ENTRY,RESIDENCE-GARAGE,false,false,0612,006,21,71,05,1168380,1851969,2011,02/04/2016 06:33:39 AM,41.749330381,-87.658562005\\
8316838,HT548010,10/17/2011 03:20:00 PM,055XX N KEDZIE AVE,0820,THEFT,\$500 AND UNDER,"SCHOOL, PUBLIC, GROUNDS",true,false,1712,017,40,13,06,1154047,1936545,2011,02/04/2016 06:33:39 AM,41.981712678,-87.708829703\\
8316839,HT551049,10/20/2011 08:50:00 AM,102XX S AVENUE N,0430,BATTERY,AGGRAVATED: OTHER DANG WEAPON,STREET,false,false,0432,004,10,52,04B,1201166,1837763,2011,02/04/2016 06:33:39 AM,41.709579698,-87.538903651\\
8316871,HT549680,10/19/2011 01:03:00 PM,044XX N BROADWAY,0460,BATTERY,SIMPLE,DEPARTMENT STORE,false,false,2313,019,46,3,08B,1168460,1929880,2011,02/04/2016 06:33:39 AM,41.963123126,-87.65601675\\
8316872,HT551071,10/19/2011 03:10:00 PM,053XX S CALUMET AVE,0810,THEFT,OVER \$500,RESIDENCE,false,false,0234,002,3,40,06,1179390,1869685,2011,02/04/2016 06:33:39 AM,41.797700881,-87.617676981\\
8316873,HT551063,10/20/2011 11:10:00 AM,003XX E 47TH ST,1811,NARCOTICS,POSS: CANNABIS 30GMS OR LESS,SIDEWALK,true,false,0222,002,3,38,18,1178980,1873925,2011,02/04/2016 06:33:39 AM,41.809345175,-87.619051287\\
8316874,HT550901,10/20/2011 09:11:00 AM,033XX W OGDEN AVE,2022,NARCOTICS,POSS: COCAINE,POLICE FACILITY/VEH PARKING LOT,true,false,1024,010,24,29,18,1154489,1891024,2011,02/04/2016 06:33:39 AM,41.856790413,-87.708424071\\
8316875,HT549739,10/19/2011 01:30:00 PM,002XX E GARFIELD BLVD,0820,THEFT,\$500 AND UNDER,CTA BUS,false,false,0232,002,3,40,06,1178645,1868596,2011,02/04/2016 06:33:39 AM,41.794729551,-87.620442108\\
8316880,HT549802,10/19/2011 12:00:00 PM,011XX W WILSON AVE,0460,BATTERY,SIMPLE,COLLEGE/UNIVERSITY GROUNDS,false,false,2311,019,46,3,08B,1167612,1930696,2011,02/04/2016 06:33:39 AM,41.96538061,-87.659110921\\
8316881,HT431449,08/04/2011 11:00:00 AM,027XX W CHICAGO AVE,0820,THEFT,\$500 AND UNDER,STREET,false,false,1313,012,26,24,06,1157782,1905211,2011,02/04/2016 06:33:39 AM,41.895654523,-87.69595021\\
8316882,HT549162,10/19/2011 06:42:00 AM,105XX S WESTERN AVE,0610,BURGLARY,FORCIBLE ENTRY,TAVERN/LIQUOR STORE,false,false,2211,022,19,72,05,1162255,1834721,2011,02/04/2016 06:33:39 AM,41.702128701,-87.681485145\\
8316884,HT544972,10/16/2011 04:30:00 AM,103XX S HALSTED ST,1310,CRIMINAL DAMAGE,TO PROPERTY,SMALL RETAIL STORE,false,false,2232,022,34,73,14,1172807,1836192,2011,02/04/2016 06:33:39 AM,41.705939683,-87.642803521\\
8316886,HT549777,10/19/2011 02:10:00 PM,014XX W PRATT BLVD,0850,THEFT,ATTEMPT THEFT,SMALL RETAIL STORE,false,false,2431,024,49,1,06,1165565,1945281,2011,02/04/2016 06:33:39 AM,42.005446228,-87.666219555\\
8316887,HT551046,10/20/2011 11:10:00 AM,050XX N WINTHROP AVE,2820,OTHER OFFENSE,TELEPHONE THREAT,RESIDENCE,false,false,2033,020,48,3,26,1167955,1933907,2011,02/04/2016 06:33:39 AM,41.974184283,-87.657756697\\
8316889,HT550997,10/20/2011 09:10:00 AM,041XX N DICKINSON AVE,1121,DECEPTIVE PRACTICE,COUNTERFEITING DOCUMENT,STREET,true,false,1624,016,45,15,10,1142513,1926900,2011,02/04/2016 06:33:39 AM,41.955468935,-87.751489799\\
8316890,HT532649,10/07/2011 11:46:00 PM,062XX S VERNON AVE,2092,NARCOTICS,SOLICIT NARCOTICS ON PUBLICWAY,SIDEWALK,true,false,0313,003,20,42,26,1180324,1863694,2011,02/04/2016 06:33:39 AM,41.781239632,-87.614435596\\
8316893,HT551023,10/20/2011 02:00:00 AM,081XX S STEWART AVE,0810,THEFT,OVER \$500,STREET,false,false,0622,006,21,44,06,1175042,1850896,2011,02/04/2016 06:33:39 AM,41.746239973,-87.634181801\\
8316894,HT550772,10/20/2011 07:10:00 AM,048XX N TALMAN AVE,1320,CRIMINAL DAMAGE,TO VEHICLE,STREET,false,false,2031,020,40,4,14,1157831,1931974,2011,02/04/2016 06:33:39 AM,41.969093078,-87.695038453\\
8316898,HT551055,10/16/2011 09:00:00 AM,018XX S LAFLIN ST,1365,CRIMINAL TRESPASS,TO RESIDENCE,APARTMENT,false,false,1222,012,25,31,26,1166665,1891065,2011,02/04/2016 06:33:39 AM,41.856651049,-87.663730374\\
8316899,HT550695,10/20/2011 05:30:00 AM,122XX S HALSTED ST,1310,CRIMINAL DAMAGE,TO PROPERTY,RESIDENCE PORCH/HALLWAY,false,false,0524,005,34,53,14,1173210,1823636,2011,02/04/2016 06:33:39 AM,41.671475169,-87.641696924\\
8316901,HT549052,10/19/2011 12:01:00 AM,064XX S DR MARTIN LUTHER KING JR DR,1811,NARCOTICS,POSS: CANNABIS 30GMS OR LESS,STREET,true,false,0312,003,20,42,18,1180027,1862432,2011,02/04/2016 06:33:39 AM,41.777783389,-87.615563072\\
8316902,HT550988,10/20/2011 10:25:00 AM,066XX S KENNETH AVE,0486,BATTERY,DOMESTIC BATTERY SIMPLE,RESIDENCE,false,false,0833,008,13,65,08B,1147854,1860166,2011,02/04/2016 06:33:39 AM,41.772241474,-87.733568892\\
8316909,HT550957,10/20/2011 04:45:00 AM,051XX S MONITOR AVE,2825,OTHER OFFENSE,HARASSMENT BY TELEPHONE,RESIDENCE,false,true,0811,008,23,56,26,1138234,1869952,2011,02/04/2016 06:33:39 AM,41.79927468,-87.768598031\\

 

  • Time stamp
  • Geographical Location
  • Nature of infraction
  • Arrests Made

The Underlying Math

  • Predict the spatio-temporal event risk
  • NOT individual people
  • CANNOT be used for individual "pre-arrests"
  • No manual selection of factors
  • No "lists"
  • Uses only de-identified data
  • Can use the models to "audit" the state
  • Identify and reveal enforcement biases
  • Democratization of AI

93% accuracy

87% AUC

~70% specificity at ~80% sensitivity

Chicago Predictive Performance

10 actual crimes:

11 predicted:

8 correct:

2 missed:

3 false alarms

1 Week in advance

Within ~2 city blocks

ONLY Past eventlog as input

Triangles: actual events

 

heatmap: predicted risk 3 days ahead

 Jan 1 2016 - April 1 2019

Philadelphia

100 crimes

Raise 103 flags

90 correct flags

13 false positives

10 missed

Could we have predicted this?

 

Double homicide

Jan 7 2019

Triple homicide incident

Jan 7 2019

https://www.inquirer.com/crime/kensington-triple-shooting-homicide-philadelphia-police-20190107.html

Triangles: actual events

 

heatmap: predicted risk 3 days ahead

Zip code 19132

in Philadelphia

fp       1.840708
tp      21.530973
fn       2.946903
sens     0.878993
ppv      0.909565
fp      0.106195
tp      3.982301
fn      0.522124
sens    0.881521
ppv     0.960446

property

violent

Property:

Other_Larceny-Larceny_from_MV-Auto_Theft-Residential_Burglary-Robbery-Commercial_Burglary    

Violent:

Aggravated_Assault-Rape_&_Attempted-Homicide                                                 
Property:

Other_Larceny-Larceny_from_MV-Auto_Theft-Residential_Burglary-Robbery-Commercial_Burglary    

Violent:

Aggravated_Assault-Rape_&_Attempted-Homicide

Mean AUC

Property crime: 81%

Violent crime:    84%

Spatial tiles:

0.0028 deg latitude, 0.0019 deg longitude

0.2 miles across

(100 x 300 yds)

Time-period:

Training:                      Jan 1 2021 - March 31 2022

Out-of-sample test:  Apr 1 2022 - July 22 2022

Digital Twin

Not just a predictor

Signature of enforcement

inequity

Rotaru, Victor, Yi Huang, Timmy Li, James Evans, and Ishanu Chattopadhyay. "Event-level prediction of urban crime reveals a signature of enforcement bias in US cities." Nature human behaviour 6, no. 8 (2022): 1056-1068.

The Brave New World

Everything is predictable, the only question is by how much?

VeRITaAS

Can A Generative AI Tell if you Are Lying?

Vetting Response Integrity from
cross-Talk in Adversarial
Surveys

Hidden structure of cross-talk between responses to interview items

PTSD diagnostic interview

Number of possible responses

Minimum Performance (n=624)

Average Time: 3.5 min

No. of questions: 20

AUC > 0.95

PPV > 0.86

NPV > 0.92

At least 83.3% sensitivity at 94% specificity

Minimum AUC = \(0.95 \pm 0.005\)

Cannot be coached, or memorized

Datasets for training & validation

1. VA (n=294)

2. Prolific (n=300)

3. Psychiatrists (n=30)

10^{25}

Beat the test!

paraknowledge.ai/veritas

200 participants in

US

100 participants in

UK

30 forensic psychiatrists

10

6

1

Can-You-Fake-PTSD Challenge Results

successful attempts

Future

Vision

Transform bio-surveillance

Democratize AI unleashing its power for social good

Transform early diagnosis

Transform modeling of complex systems

Acknowledgements

The data, ethics, and biases of predictive policing

By Ishanu Chattopadhyay

The data, ethics, and biases of predictive policing

Predictive modeling of crime and rare phenomena using fractal nets

  • 43

More from Ishanu Chattopadhyay