Ishanu Chattopadhyay
Assistant Professor of
Data Science
University of Chicago
Urban Crime Prediction: The data, ethics, and biases of predicting events
Rotaru, V., Huang, Y., Li, T. et al.
Event-level prediction of urban crime reveals a signature of enforcement bias in US cities. Nature Human Behavior 6, 1056–1068 (2022).
Harper Lectures
February 15 2023
AI Awakens
Machines can reach false or dangerous conclusions and human review is essential for responsible AI.
-- De-Arteaga et al.
It is not clear that humans are very effective at overseeing algorithms.
Unless humans get better at overcoming their own conscious and unconscious biases, manual oversight runs the risk of making bias problems worse.
-- Green et al.
Human-in-the-loop is not necessarily better
The Problem of Free Will
Predictive Policing
Broad patterns are easily predictable
Hotspots?
2 weeks, Chicago, 2017
property crime
violent crime
complex urban topology
Some historical patterns are predictive
What Properties Should a Predictive Algorithm have to be acceptable in enforcement?
Lets Predict Crime
ID,Case Number,Date,Block,IUCR,Primary Type,Description,Location Description,Arrest,Domestic,Beat,District,Ward,Community Area,FBI Code,X Coordinate,Y Coordinate,Year,Updated On,Latitude,Longitude,Location\\
8316800,HT550945,08/11/2011 11:00:00 AM,086XX S MARQUETTE AVE,1120,DECEPTIVE PRACTICE,FORGERY,RESIDENCE,false,false,0423,004,7,46,10,1195654,1848294,2011,02/04/2016 06:33:39 AM,41.738615478,-87.558741896\\
8316805,HT550781,10/20/2011 05:00:00 AM,056XX S ABERDEEN ST,0890,THEFT,FROM BUILDING,RESIDENCE,false,false,0712,007,16,68,06,1169943,1867457,2011,02/04/2016 06:33:39 AM,41.791797599,-87.652385205\\
8316806,HT550706,10/20/2011 05:45:00 AM,079XX S LOOMIS BLVD,031A,ROBBERY,ARMED: HANDGUN,STREET,false,false,0612,006,21,71,03,1168370,1852331,2011,02/04/2016 06:33:39 AM,41.750323974,-87.658588247\\
8316811,HT539324,10/12/2011 12:23:52 PM,003XX E 75TH ST,2027,NARCOTICS,POSS: CRACK,SMALL RETAIL STORE,true,false,0323,003,6,69,18,1179641,1855355,2011,02/04/2016 06:33:39 AM,41.758372192,-87.61719416\\
8316822,HT551031,10/19/2011 02:00:00 AM,071XX W DICKENS AVE,0910,MOTOR VEHICLE THEFT,AUTOMOBILE,SIDEWALK,false,false,2512,025,36,25,07,1127877,1913161,2011,02/04/2016 06:33:39 AM,41.918027518,-87.805606689\\
8316824,HT551032,10/20/2011 12:00:00 AM,034XX N NATCHEZ AVE,2825,OTHER OFFENSE,HARASSMENT BY TELEPHONE,RESIDENCE,false,false,1632,016,36,17,26,1132429,1922272,2011,02/04/2016 06:33:39 AM,41.94295109,-87.788669409\\
8316825,HT549690,10/19/2011 12:51:00 PM,079XX S ADA ST,2820,OTHER OFFENSE,TELEPHONE THREAT,APARTMENT,false,false,0612,006,21,71,26,1168711,1852015,2011,02/04/2016 06:33:39 AM,41.749449482,-87.657347764\\
8316826,HT549865,10/19/2011 06:00:00 AM,011XX N LEAMINGTON AVE,0810,THEFT,OVER \$500,RESIDENTIAL YARD (FRONT/BACK),false,false,1531,015,37,25,06,1141821,1907155,2011,02/04/2016 06:33:39 AM,41.9012995,-87.754523767\\
8316827,HT550963,09/01/2011 04:00:00 PM,079XX S LOOMIS BLVD,0610,BURGLARY,FORCIBLE ENTRY,RESIDENCE-GARAGE,false,false,0612,006,21,71,05,1168380,1851969,2011,02/04/2016 06:33:39 AM,41.749330381,-87.658562005\\
8316838,HT548010,10/17/2011 03:20:00 PM,055XX N KEDZIE AVE,0820,THEFT,\$500 AND UNDER,"SCHOOL, PUBLIC, GROUNDS",true,false,1712,017,40,13,06,1154047,1936545,2011,02/04/2016 06:33:39 AM,41.981712678,-87.708829703\\
8316839,HT551049,10/20/2011 08:50:00 AM,102XX S AVENUE N,0430,BATTERY,AGGRAVATED: OTHER DANG WEAPON,STREET,false,false,0432,004,10,52,04B,1201166,1837763,2011,02/04/2016 06:33:39 AM,41.709579698,-87.538903651\\
8316871,HT549680,10/19/2011 01:03:00 PM,044XX N BROADWAY,0460,BATTERY,SIMPLE,DEPARTMENT STORE,false,false,2313,019,46,3,08B,1168460,1929880,2011,02/04/2016 06:33:39 AM,41.963123126,-87.65601675\\
8316872,HT551071,10/19/2011 03:10:00 PM,053XX S CALUMET AVE,0810,THEFT,OVER \$500,RESIDENCE,false,false,0234,002,3,40,06,1179390,1869685,2011,02/04/2016 06:33:39 AM,41.797700881,-87.617676981\\
8316873,HT551063,10/20/2011 11:10:00 AM,003XX E 47TH ST,1811,NARCOTICS,POSS: CANNABIS 30GMS OR LESS,SIDEWALK,true,false,0222,002,3,38,18,1178980,1873925,2011,02/04/2016 06:33:39 AM,41.809345175,-87.619051287\\
8316874,HT550901,10/20/2011 09:11:00 AM,033XX W OGDEN AVE,2022,NARCOTICS,POSS: COCAINE,POLICE FACILITY/VEH PARKING LOT,true,false,1024,010,24,29,18,1154489,1891024,2011,02/04/2016 06:33:39 AM,41.856790413,-87.708424071\\
8316875,HT549739,10/19/2011 01:30:00 PM,002XX E GARFIELD BLVD,0820,THEFT,\$500 AND UNDER,CTA BUS,false,false,0232,002,3,40,06,1178645,1868596,2011,02/04/2016 06:33:39 AM,41.794729551,-87.620442108\\
8316880,HT549802,10/19/2011 12:00:00 PM,011XX W WILSON AVE,0460,BATTERY,SIMPLE,COLLEGE/UNIVERSITY GROUNDS,false,false,2311,019,46,3,08B,1167612,1930696,2011,02/04/2016 06:33:39 AM,41.96538061,-87.659110921\\
8316881,HT431449,08/04/2011 11:00:00 AM,027XX W CHICAGO AVE,0820,THEFT,\$500 AND UNDER,STREET,false,false,1313,012,26,24,06,1157782,1905211,2011,02/04/2016 06:33:39 AM,41.895654523,-87.69595021\\
8316882,HT549162,10/19/2011 06:42:00 AM,105XX S WESTERN AVE,0610,BURGLARY,FORCIBLE ENTRY,TAVERN/LIQUOR STORE,false,false,2211,022,19,72,05,1162255,1834721,2011,02/04/2016 06:33:39 AM,41.702128701,-87.681485145\\
8316884,HT544972,10/16/2011 04:30:00 AM,103XX S HALSTED ST,1310,CRIMINAL DAMAGE,TO PROPERTY,SMALL RETAIL STORE,false,false,2232,022,34,73,14,1172807,1836192,2011,02/04/2016 06:33:39 AM,41.705939683,-87.642803521\\
8316886,HT549777,10/19/2011 02:10:00 PM,014XX W PRATT BLVD,0850,THEFT,ATTEMPT THEFT,SMALL RETAIL STORE,false,false,2431,024,49,1,06,1165565,1945281,2011,02/04/2016 06:33:39 AM,42.005446228,-87.666219555\\
8316887,HT551046,10/20/2011 11:10:00 AM,050XX N WINTHROP AVE,2820,OTHER OFFENSE,TELEPHONE THREAT,RESIDENCE,false,false,2033,020,48,3,26,1167955,1933907,2011,02/04/2016 06:33:39 AM,41.974184283,-87.657756697\\
8316889,HT550997,10/20/2011 09:10:00 AM,041XX N DICKINSON AVE,1121,DECEPTIVE PRACTICE,COUNTERFEITING DOCUMENT,STREET,true,false,1624,016,45,15,10,1142513,1926900,2011,02/04/2016 06:33:39 AM,41.955468935,-87.751489799\\
8316890,HT532649,10/07/2011 11:46:00 PM,062XX S VERNON AVE,2092,NARCOTICS,SOLICIT NARCOTICS ON PUBLICWAY,SIDEWALK,true,false,0313,003,20,42,26,1180324,1863694,2011,02/04/2016 06:33:39 AM,41.781239632,-87.614435596\\
8316893,HT551023,10/20/2011 02:00:00 AM,081XX S STEWART AVE,0810,THEFT,OVER \$500,STREET,false,false,0622,006,21,44,06,1175042,1850896,2011,02/04/2016 06:33:39 AM,41.746239973,-87.634181801\\
8316894,HT550772,10/20/2011 07:10:00 AM,048XX N TALMAN AVE,1320,CRIMINAL DAMAGE,TO VEHICLE,STREET,false,false,2031,020,40,4,14,1157831,1931974,2011,02/04/2016 06:33:39 AM,41.969093078,-87.695038453\\
8316898,HT551055,10/16/2011 09:00:00 AM,018XX S LAFLIN ST,1365,CRIMINAL TRESPASS,TO RESIDENCE,APARTMENT,false,false,1222,012,25,31,26,1166665,1891065,2011,02/04/2016 06:33:39 AM,41.856651049,-87.663730374\\
8316899,HT550695,10/20/2011 05:30:00 AM,122XX S HALSTED ST,1310,CRIMINAL DAMAGE,TO PROPERTY,RESIDENCE PORCH/HALLWAY,false,false,0524,005,34,53,14,1173210,1823636,2011,02/04/2016 06:33:39 AM,41.671475169,-87.641696924\\
8316901,HT549052,10/19/2011 12:01:00 AM,064XX S DR MARTIN LUTHER KING JR DR,1811,NARCOTICS,POSS: CANNABIS 30GMS OR LESS,STREET,true,false,0312,003,20,42,18,1180027,1862432,2011,02/04/2016 06:33:39 AM,41.777783389,-87.615563072\\
8316902,HT550988,10/20/2011 10:25:00 AM,066XX S KENNETH AVE,0486,BATTERY,DOMESTIC BATTERY SIMPLE,RESIDENCE,false,false,0833,008,13,65,08B,1147854,1860166,2011,02/04/2016 06:33:39 AM,41.772241474,-87.733568892\\
8316909,HT550957,10/20/2011 04:45:00 AM,051XX S MONITOR AVE,2825,OTHER OFFENSE,HARASSMENT BY TELEPHONE,RESIDENCE,false,true,0811,008,23,56,26,1138234,1869952,2011,02/04/2016 06:33:39 AM,41.79927468,-87.768598031\\
variables: <location,category>
arrests | |
violent crimes | |
nonviolent crimes |
~3000 location tiles
~9000 variables
~40 million binary interactions
~ 1 billion possible models of binary interaction
The Problem:
1 Week in advance
Within ~2 city blocks
ONLY Past eventlog as input
"crime forecast"
93% accuracy
87% AUC
~70% specificity at ~80% sensitivity
10 actual crimes:
11 predicted:
8 correct:
2 missed:
3 false alarms
Chicago Predictive Performance
Boston Districts: B2 B3 C1
Jan 1 2021 - July 22 2022
Total # of events: 7419
Example: Boston MA, USA
Property:
Other_Larceny-Larceny_from_MV-Auto_Theft-Residential_Burglary-Robbery-Commercial_Burglary
Violent:
Aggravated_Assault-Rape_&_Attempted-Homicide
Mean AUC
Property crime: 81%
Violent crime: 84%
Spatial tiles:
0.0028 deg latitude, 0.0019 deg longitude
0.2 miles across
(100 x 300 yds)
Time-period:
Training: Jan 1 2021 - March 31 2022
Out-of-sample test: Apr 1 2022 - July 22 2022
Prediction Performance
sensitivity 0.88
ppv 0.91
10 crimes
Raise 11 flags
8 correct flags
3 false positives
2 false negatives
example
Mean AUC
Property crime: 81%
Violent crime: 84%
Spatial tiles:
0.003 deg latitude, 0.003 deg longitude
0.25 miles across
Time-period:
Training: Jan 1 2016 - Dec 31 2018
Out-of-sample test: Jan 1 2019 - April 1 2019
Philadelphia
sensitivity 0.90
ppv 0.87
100 crimes
Raise 103 flags
90 correct flags
13 false positives
10 missed
3 day ahead prediction
Jan 1 2019
to
April 1 2019
Play Movie
Triangles: actual events
heatmap: predicted risk 3 days ahead
Could we have predicted this?
Double homicide
Jan 7 2019
Triple homicide incident
Jan 7 2019
https://www.inquirer.com/crime/kensington-triple-shooting-homicide-philadelphia-police-20190107.html
Triangles: actual events
heatmap: predicted risk 3 days ahead
The Underlying Math
Not based on standard "Deep Learning"
Applies to any rare/extreme event phenomena
Ishanu Chattopadhyay, Yi Huang, James Evans et al. Deep Learning Without Neural Networks: Fractal-nets for Rare Event Modeling, 26 October 2020, PREPRINT (Version https://doi.org/10.21203/rs.3.rs-86045/v1
Digital Twin
Not just a predictor
Signature of enforcement
inequity
Results corroborated in signature observed in raw data
Digital Twin of social interactions
Predict policy effects
Precise predictation
Discover and quantify biases
Acknowledgements