Societal Implications of Technology

Sarah Dean

CS 6006, October 2023

Societal Implications of Tech

This fall: CS 7340, theme: tech & labor
Last fall: informal reading group, theme: race & technology
- first run in 2020 (without me)

With Haym Hirsch and Anil Damle

Bias: prejudice, usually in a way considered to be unfair

(mathematical) systematic distortion of a statistical result

Prejudice: preconceived opinion

(legal) harm or injury resulting from some action or judgment

Why hold CS seminar on this?

We are trained in CS/math/engineering
- Other disciplines have more expertise on these issues
CS research calls for understanding societal positioning
I came to this view due to my research on bias (in ML)

1. Miscalibration in Recommendation

Designing recommendations which optimize engagement leads to over-recommending the most prevalent types (Steck, 2018)

rom-com 80% of the time

horror 20% of the time

optimize for probable click

The chance of a click depends on whether we recommend a rom-com or horror movie:

$$\mathbb P(\mathsf{click})= \mathbb P(\mathsf{click}~\text{and}~\mathsf{romcom}) + \mathbb P(\mathsf{click}~\text{and}~\mathsf{horror}) $$

$0.8\times \mathbb P(\mathsf{romcom})+0.2\times P(\mathsf{horror}) $

$(1- \mathbb P(\mathsf{romcom}) )$

1. Miscalibration in Recommendation

Designing recommendations which optimize engagement leads to over-recommending the most prevalent types (Steck, 2018)

rom-com 80% of the time

horror 20% of the time

optimize for probable click

recommend rom-com 100% of the time

$$\max_{0\leq x\leq 1}0.8\times x + 0.2 \times (1-x)$$

2. Gender Bias in Translation

Data-driven machine translation perpetuates gender bias.

2. Gender Bias in Translation

Machine translation works by maximizing the probability of an English sentence given Hungarian sentence

(A) She $\leftrightarrow$ Ő
(B) She is a scientist $\leftrightarrow$ Ő egy tudós
(C) My mother's name is Ann. She is a scientist. $\leftrightarrow$ Anyám neve Ann. Ő egy tudós.

$0$

$1$

(A)

(B)

(C)

Naive optimization under uncertainty leads to miscalibration and bias

A broad set of possible ameliorating approaches: UI, uncertainty reduction

3. Gender Bias in Resume Screening

✅

send to recruiter

Data-driven resume screening algorithms repeat pre-existing bias

✅

❌

records

resume

Risk algorithms used in healthcare under-predict the needs of Black patients compared with White patients (Obermeyer et al., 2019).

4. Racial Bias in Healthcare

risk score

medical records

$$$

Pattern recognition (i.e. machine learning) replicates and amplifies bias in the data

✅/❌

data

decisions

Sources of Bias

Naive optimization under uncertainty leads to miscalibration and bias

What does it mean to build good technology in an imperfect world?

Technologies are developed and used within a particular social, economic, and political context. They arise out of a social structure, they are grafted on to it, and they may reinforce it or destroy it, often in ways that are neither foreseen nor foreseeable.”

Ursula Franklin, 1989

I think the computer has from the beginning been a fundamentally conservative force. It has made possible the saving of institutions pretty much as they were, which otherwise might have had to be changed. [...] If it had not been for the computer [...] it might have been necessary to introduce a social invention, as opposed to the technical invention.

Joseph Weizenbaum, 1985