Ishanu Chattopadhyay

Assistant Professor of

Data Science

University of Chicago

zed.uchicago.edu

Urban Crime Prediction: The data, ethics, and biases of predicting events

AI Awakens

AI governance: legal framework for ensuring that machine learning (ML) technologies are developed with the goal of helping humanity navigate the adoption of AI systems fairly.

Machines can reach false or dangerous conclusions and human review is essential for responsible AI.

-- De-Arteaga et al.

It is not clear that humans are very effective at overseeing algorithms.

Unless humans get better at overcoming their own conscious and unconscious biases, manual oversight runs the risk of making bias problems worse.

-- Green et al.

Human-in-the-loop is not necessarily better

The Problem of Free Will

Predictive Policing

Broad patterns are easily predictable

Hotspots?

2 weeks, Chicago, 2017

property crime

violent crime

complex urban topology

Not everything is predictable

Not everything is random

Find and use patterns that predict future events

Validate such patterns in out-of-sample data

Some historical patterns are predictive

variables: <location,category>

	arrests
	violent crimes
	nonviolent crimes

~3000 location tiles

~9000 variables

~40 million binary interactions

~ 1 billion possible models of binary interaction

Predictions cannot be self-fulfilling prophecies
Transparency
Explore the interaction between crime, social factors and enforcement

Ability to predict law enforcement response to crime as well
Analyze if such reponses reveal existant policy biases

What Properties Should a Predictive Algorithm have to be acceptable in enforcement?

Patterns from event logs

ID,Case Number,Date,Block,IUCR,Primary Type,Description,Location Description,Arrest,Domestic,Beat,District,Ward,Community Area,FBI Code,X Coordinate,Y Coordinate,Year,Updated On,Latitude,Longitude,Location\\
8316800,HT550945,08/11/2011 11:00:00 AM,086XX S MARQUETTE AVE,1120,DECEPTIVE PRACTICE,FORGERY,RESIDENCE,false,false,0423,004,7,46,10,1195654,1848294,2011,02/04/2016 06:33:39 AM,41.738615478,-87.558741896\\
8316805,HT550781,10/20/2011 05:00:00 AM,056XX S ABERDEEN ST,0890,THEFT,FROM BUILDING,RESIDENCE,false,false,0712,007,16,68,06,1169943,1867457,2011,02/04/2016 06:33:39 AM,41.791797599,-87.652385205\\
8316806,HT550706,10/20/2011 05:45:00 AM,079XX S LOOMIS BLVD,031A,ROBBERY,ARMED: HANDGUN,STREET,false,false,0612,006,21,71,03,1168370,1852331,2011,02/04/2016 06:33:39 AM,41.750323974,-87.658588247\\
8316811,HT539324,10/12/2011 12:23:52 PM,003XX E 75TH ST,2027,NARCOTICS,POSS: CRACK,SMALL RETAIL STORE,true,false,0323,003,6,69,18,1179641,1855355,2011,02/04/2016 06:33:39 AM,41.758372192,-87.61719416\\
8316822,HT551031,10/19/2011 02:00:00 AM,071XX W DICKENS AVE,0910,MOTOR VEHICLE THEFT,AUTOMOBILE,SIDEWALK,false,false,2512,025,36,25,07,1127877,1913161,2011,02/04/2016 06:33:39 AM,41.918027518,-87.805606689\\
8316824,HT551032,10/20/2011 12:00:00 AM,034XX N NATCHEZ AVE,2825,OTHER OFFENSE,HARASSMENT BY TELEPHONE,RESIDENCE,false,false,1632,016,36,17,26,1132429,1922272,2011,02/04/2016 06:33:39 AM,41.94295109,-87.788669409\\
8316825,HT549690,10/19/2011 12:51:00 PM,079XX S ADA ST,2820,OTHER OFFENSE,TELEPHONE THREAT,APARTMENT,false,false,0612,006,21,71,26,1168711,1852015,2011,02/04/2016 06:33:39 AM,41.749449482,-87.657347764\\
8316826,HT549865,10/19/2011 06:00:00 AM,011XX N LEAMINGTON AVE,0810,THEFT,OVER \$500,RESIDENTIAL YARD (FRONT/BACK),false,false,1531,015,37,25,06,1141821,1907155,2011,02/04/2016 06:33:39 AM,41.9012995,-87.754523767\\
8316827,HT550963,09/01/2011 04:00:00 PM,079XX S LOOMIS BLVD,0610,BURGLARY,FORCIBLE ENTRY,RESIDENCE-GARAGE,false,false,0612,006,21,71,05,1168380,1851969,2011,02/04/2016 06:33:39 AM,41.749330381,-87.658562005\\
8316838,HT548010,10/17/2011 03:20:00 PM,055XX N KEDZIE AVE,0820,THEFT,\$500 AND UNDER,"SCHOOL, PUBLIC, GROUNDS",true,false,1712,017,40,13,06,1154047,1936545,2011,02/04/2016 06:33:39 AM,41.981712678,-87.708829703\\
8316839,HT551049,10/20/2011 08:50:00 AM,102XX S AVENUE N,0430,BATTERY,AGGRAVATED: OTHER DANG WEAPON,STREET,false,false,0432,004,10,52,04B,1201166,1837763,2011,02/04/2016 06:33:39 AM,41.709579698,-87.538903651\\
8316871,HT549680,10/19/2011 01:03:00 PM,044XX N BROADWAY,0460,BATTERY,SIMPLE,DEPARTMENT STORE,false,false,2313,019,46,3,08B,1168460,1929880,2011,02/04/2016 06:33:39 AM,41.963123126,-87.65601675\\
8316872,HT551071,10/19/2011 03:10:00 PM,053XX S CALUMET AVE,0810,THEFT,OVER \$500,RESIDENCE,false,false,0234,002,3,40,06,1179390,1869685,2011,02/04/2016 06:33:39 AM,41.797700881,-87.617676981\\
8316873,HT551063,10/20/2011 11:10:00 AM,003XX E 47TH ST,1811,NARCOTICS,POSS: CANNABIS 30GMS OR LESS,SIDEWALK,true,false,0222,002,3,38,18,1178980,1873925,2011,02/04/2016 06:33:39 AM,41.809345175,-87.619051287\\
8316874,HT550901,10/20/2011 09:11:00 AM,033XX W OGDEN AVE,2022,NARCOTICS,POSS: COCAINE,POLICE FACILITY/VEH PARKING LOT,true,false,1024,010,24,29,18,1154489,1891024,2011,02/04/2016 06:33:39 AM,41.856790413,-87.708424071\\
8316875,HT549739,10/19/2011 01:30:00 PM,002XX E GARFIELD BLVD,0820,THEFT,\$500 AND UNDER,CTA BUS,false,false,0232,002,3,40,06,1178645,1868596,2011,02/04/2016 06:33:39 AM,41.794729551,-87.620442108\\
8316880,HT549802,10/19/2011 12:00:00 PM,011XX W WILSON AVE,0460,BATTERY,SIMPLE,COLLEGE/UNIVERSITY GROUNDS,false,false,2311,019,46,3,08B,1167612,1930696,2011,02/04/2016 06:33:39 AM,41.96538061,-87.659110921\\
8316881,HT431449,08/04/2011 11:00:00 AM,027XX W CHICAGO AVE,0820,THEFT,\$500 AND UNDER,STREET,false,false,1313,012,26,24,06,1157782,1905211,2011,02/04/2016 06:33:39 AM,41.895654523,-87.69595021\\
8316882,HT549162,10/19/2011 06:42:00 AM,105XX S WESTERN AVE,0610,BURGLARY,FORCIBLE ENTRY,TAVERN/LIQUOR STORE,false,false,2211,022,19,72,05,1162255,1834721,2011,02/04/2016 06:33:39 AM,41.702128701,-87.681485145\\
8316884,HT544972,10/16/2011 04:30:00 AM,103XX S HALSTED ST,1310,CRIMINAL DAMAGE,TO PROPERTY,SMALL RETAIL STORE,false,false,2232,022,34,73,14,1172807,1836192,2011,02/04/2016 06:33:39 AM,41.705939683,-87.642803521\\
8316886,HT549777,10/19/2011 02:10:00 PM,014XX W PRATT BLVD,0850,THEFT,ATTEMPT THEFT,SMALL RETAIL STORE,false,false,2431,024,49,1,06,1165565,1945281,2011,02/04/2016 06:33:39 AM,42.005446228,-87.666219555\\
8316887,HT551046,10/20/2011 11:10:00 AM,050XX N WINTHROP AVE,2820,OTHER OFFENSE,TELEPHONE THREAT,RESIDENCE,false,false,2033,020,48,3,26,1167955,1933907,2011,02/04/2016 06:33:39 AM,41.974184283,-87.657756697\\
8316889,HT550997,10/20/2011 09:10:00 AM,041XX N DICKINSON AVE,1121,DECEPTIVE PRACTICE,COUNTERFEITING DOCUMENT,STREET,true,false,1624,016,45,15,10,1142513,1926900,2011,02/04/2016 06:33:39 AM,41.955468935,-87.751489799\\
8316890,HT532649,10/07/2011 11:46:00 PM,062XX S VERNON AVE,2092,NARCOTICS,SOLICIT NARCOTICS ON PUBLICWAY,SIDEWALK,true,false,0313,003,20,42,26,1180324,1863694,2011,02/04/2016 06:33:39 AM,41.781239632,-87.614435596\\
8316893,HT551023,10/20/2011 02:00:00 AM,081XX S STEWART AVE,0810,THEFT,OVER \$500,STREET,false,false,0622,006,21,44,06,1175042,1850896,2011,02/04/2016 06:33:39 AM,41.746239973,-87.634181801\\
8316894,HT550772,10/20/2011 07:10:00 AM,048XX N TALMAN AVE,1320,CRIMINAL DAMAGE,TO VEHICLE,STREET,false,false,2031,020,40,4,14,1157831,1931974,2011,02/04/2016 06:33:39 AM,41.969093078,-87.695038453\\
8316898,HT551055,10/16/2011 09:00:00 AM,018XX S LAFLIN ST,1365,CRIMINAL TRESPASS,TO RESIDENCE,APARTMENT,false,false,1222,012,25,31,26,1166665,1891065,2011,02/04/2016 06:33:39 AM,41.856651049,-87.663730374\\
8316899,HT550695,10/20/2011 05:30:00 AM,122XX S HALSTED ST,1310,CRIMINAL DAMAGE,TO PROPERTY,RESIDENCE PORCH/HALLWAY,false,false,0524,005,34,53,14,1173210,1823636,2011,02/04/2016 06:33:39 AM,41.671475169,-87.641696924\\
8316901,HT549052,10/19/2011 12:01:00 AM,064XX S DR MARTIN LUTHER KING JR DR,1811,NARCOTICS,POSS: CANNABIS 30GMS OR LESS,STREET,true,false,0312,003,20,42,18,1180027,1862432,2011,02/04/2016 06:33:39 AM,41.777783389,-87.615563072\\
8316902,HT550988,10/20/2011 10:25:00 AM,066XX S KENNETH AVE,0486,BATTERY,DOMESTIC BATTERY SIMPLE,RESIDENCE,false,false,0833,008,13,65,08B,1147854,1860166,2011,02/04/2016 06:33:39 AM,41.772241474,-87.733568892\\
8316909,HT550957,10/20/2011 04:45:00 AM,051XX S MONITOR AVE,2825,OTHER OFFENSE,HARASSMENT BY TELEPHONE,RESIDENCE,false,true,0811,008,23,56,26,1138234,1869952,2011,02/04/2016 06:33:39 AM,41.79927468,-87.768598031\\

Time stamp
Geographical Location
Nature of infraction
Arrests Made

Lets Predict Crime

No manual selection of factors
No creation of "lists"
Uses only de-identified data

The Problem:

Predicting crime sufficiently ahead of time to be actionable
Prediction precise enough in time and space to be actionable
Use ONLY data that is realistically and cheaply available

1 Week in advance

Within ~2 city blocks

ONLY Past eventlog as input

Prediction Performance

"crime forecast"

Prediction Performance

"crime forecast"

93% accuracy

87% AUC

~70% specificity at ~80% sensitivity

10 actual crimes:

11 predicted:

8 correct:

2 missed:

3 false alarms

Chicago Predictive Performance

Boston Districts: B2 B3 C1

Jan 1 2021 - July 22 2022

Total # of events: 7419

Example: Boston MA, USA

Property:

Other_Larceny-Larceny_from_MV-Auto_Theft-Residential_Burglary-Robbery-Commercial_Burglary    

Violent:

Aggravated_Assault-Rape_&_Attempted-Homicide

Mean AUC

Property crime: 81%

Violent crime: 84%

Spatial tiles:

0.0028 deg latitude, 0.0019 deg longitude

0.2 miles across

(100 x 300 yds)

Time-period:

Training: Jan 1 2021 - March 31 2022

Out-of-sample test: Apr 1 2022 - July 22 2022

Prediction Performance

sensitivity     0.88
ppv             0.91

10 crimes

Raise 11 flags

8 correct flags

3 false positives

2 false negatives

example

Mean AUC

Property crime: 81%

Violent crime: 84%

Spatial tiles:

0.003 deg latitude, 0.003 deg longitude

0.25 miles across

Time-period:

Training: Jan 1 2016 - Dec 31 2018

Out-of-sample test: Jan 1 2019 - April 1 2019

Philadelphia

sensitivity   0.90
ppv           0.87

100 crimes

Raise 103 flags

90 correct flags

13 false positives

10 missed

3 day ahead prediction

Jan 1 2019

April 1 2019

Play Movie

Triangles: actual events

heatmap: predicted risk 3 days ahead

Triple homicide incident

Jan 7 2019

https://www.inquirer.com/crime/kensington-triple-shooting-homicide-philadelphia-police-20190107.html

Triangles: actual events

heatmap: predicted risk 3 days ahead

Could we have predicted this?

Double homicide

Jan 7 2019

The Underlying Math

Not based on standard "Deep Learning"

Forecasting rare events in multi-variable stochastic evolution requires new modeling architecture

Learn local "activation functions" as symbolic probabilistic transducers

Assemble these local predictors into a "fractal net"

Applies to any rare/extreme event phenomena

Ishanu Chattopadhyay, Yi Huang, James Evans et al. Deep Learning Without Neural Networks: Fractal-nets for Rare Event Modeling, 26 October 2020, PREPRINT (Version https://doi.org/10.21203/rs.3.rs-86045/v1

Digital Twin

Not just a predictor

Signature of enforcement

inequity

Results corroborated in signature observed in raw data

Digital Twin of social interactions

Predict policy effects

Precise predictation

Discover and quantify biases

Acknowledgements

High accuracy and actionable
Probe existing inefficiencies and biases
Almost no assumptions on the nature and distribution of crime
No manual feature selection: minimizes implicit bias
Maximize transparency: de-identified data and open source code
Not based on hotspot detection

Performance verified in 8 other US cities

Chicago, LA, Philadelphia, San Francisco, Detroit, Austin, Portland, Atlanta

Optimize resource allocation
Preventive interventions possible
Can testdrive different policies before deployment in high-fidelity simulation

Rotaru, V., Huang, Y., Li, T. et al.

Event-level prediction of urban crime reveals a signature of enforcement bias in US cities. Nature Human Behavior 6, 1056–1068 (2022).

https://doi.org/10.1038/s41562-022-01372-0

For every 10 crimes,

11 flags, 3 false, 2 missed

(1 week advance, with 2 city blocks)

Patterns from event logs

Time stamp
Geographical Location
Nature of infraction
Arrests Made

3 day ahead prediction

Jan 1 2019

April 1 2019

Play Movie

Triangles: actual events

heatmap: predicted risk 3 days ahead

Triple homicide incident

Jan 7 2019

https://www.inquirer.com/crime/kensington-triple-shooting-homicide-philadelphia-police-20190107.html

Triangles: actual events

heatmap: predicted risk 3 days ahead

Past Efforts towards predictive policing

What's the new insight?

Digital twin

Discovery

Use Cases

Where do we go next?

How can this model be extened?

Extreme Event Prediction

Simulate

the world

There is widespread agreement among ethicists and tech advocates that responsible AI principles require fairness, transparency, privacy, human safety, and explanability.

But it is not always clear how to operationalize these broad principles or how to handle situations when conflicts arise between them.... a focus on one goal can come at the detriment of alternative objectives.

-- "Six Steps to Responsible AI in the Federal Government, Brookings Institue, 2022

Allocation/access of resources without bias

No hidden or privileged information that makes decisions on non-consenting citizens

Personal information not accessed or made public

Does not expose individuals to unacceptable risks

AI decisions can be interpreted, and decision chains identified

fairness
transparency
privacy
human safety
explanability

It is difficult to measure fairness in general
Even more difficult to measure equity

Fair housing laws prohibit financial officials from making loan decisions based on race, gender, and marital status.

Yet AI can find proxies that approximate these characteristics
Allow incorporation of information about protected categories without explicit use of demographics.

v_0

v_1

v_2

\omega_0

\omega_1

Example

Autism prevalence is less among Black and Hispanic children.

It is not clear if this under-diagnosis due to biases in heathcare access, or there is an underlying biological mechanism

Tools to make diagnosis more efficient might reflect this same "inequity".

Add Plot on relative prevalence

The Risk of Unchecked State Control and Abrogation of Individual Liberties

Perhaps it is really not a question about AI, but about clarity on the principles by which we wish to govern ourselves

The Risk of AI-mediated Culling of Misinformation

How do we tell the AI what is "mis-information"?

Where is the line between free speech and misformation?

Copy of Prediction_ethics_HARPER

By Ishanu Chattopadhyay

Copy of Prediction_ethics_HARPER

Predictive modeling of crime and rare phenomena using fractal nets

Ishanu Chattopadhyay PRO

ML Data Science Biomedicine Social Science Faculty

Patterns from event logs

Prediction Performance

Prediction Performance

Patterns from event logs

Copy of Prediction_ethics_HARPER

More from Ishanu Chattopadhyay