tyranny of the majority?
exploring the effects of recommender systems on minority populations through agent-based modeling
bias in ai · 11/13/2020
matthew sun · mdsun@princeton.edu
roadmap
motivation
user experiences
background research
formal setup & methods
(some) results
motivation
user experiences
background research
methods
results
- societal impacts of recommender systems (RS) frequently discussed in media, academic research
- prominently, filter bubbles/echo chambers (Pariser 2010, Sunstein 2009, Geschke et al. 2019, Liao et al. 2018)
- radicalization pathways (Ribeiro et al. 2020, Ledwich and Zaitsev 2019, Möller et al. 2018)
- in comparison, fewer studies (Ekstrand et al. 2018 & 2019, Mehrotra et al. 2018) have focused on how recommender systems affect minority populations...
- minority users
- minority content creators
- stereotype & prejudice reinforcement
- ...despite increasing interest in how predictive models can harm historically marginalized subpopulations!
motivation
user experiences
background research
methods
results
- directly studying digital platforms is hard! 😢
- proprietary data at YouTube/Instagram/TikTok not accessible to researchers
- demographic data wouldn't be immediately available
- turn to agent-based modeling, an approach that has been used for decades in the social sciences (Bonabeau 2002)
- System of agents that interact with the environment over many time steps ➡ simple rules can lead to complex behavior
- Example: Thomas Schelling's model of segregation (1971)
motivation
user experiences
background research
methods
results
motivating question: can we apply this technique to understand new forms of "digital segregation" induced by recommendation systems?
"move if less than a third of my neighbors are like me"
motivation
user experiences
background research
methods
results
"all of the technology that powers social media sites that we use...is racially biased. I am not the default on social media...and I want to talk about how I work around that."
"It’s an app that’s dominated by white people and of course you’re going to like the content of people you can relate to. I think that, along with the algorithm suppressing content from Black creators, it’s also worth talking about the fact that a lot of white users on this app don’t support Black creators.” - @chinforshort
motivation
user experiences
background research
methods
results
what evidence is there to show that users might be significantly biased in the types of content they choose to consume online?
-
Social identity theory: group membership plays an important role in one's self-concept (Tajfel 1978)
- People favor ingroups, discriminate against outgroups
- Motivated by a desire to maintain a positive self-concept
- Social cognitive theory: individuals are drawn to media with characters similar to themselves (Bandura 2001)
this is all pretty intuitive, yet these assumptions have not been explicitly modeled in RS literature!
motivation
user experiences
background research
methods
results
Social identity theory
- Appiah et al. 2013: members of marginalized racial group more likely to select and read news stories featuring members of their own race
- Salience of race can moderate ingroup preference
- Weaver 2011: White study participants showed significantly less interest in seeing movies with mostly Black casts; effects moderated by genre of movie
- Knoblock and Hastall 2010: older adults were more likely to choose to read negative news articles about younger individuals
motivation
user experiences
background research
methods
results
Social cognitive theory
- Knobloch et al. 2005: showed that Chinese, German, and American boys and girls aged 4-6 years old preferred to consume stories featuring protagonists of their gender (selective exposure)
- Abrams and Giles 2007: minority group members more likely to avoid TV shows where they were not well-represented or highly stereotyped
motivation
user experiences
background research
methods
results
Content creators and algorithmic visibility
- Cotter 2018: examined influencer communities
- Some influencers believe algorithmic platforms reward conformity, most popular content
- Constant strategizing for how to succeed with the algorithm
- Influencers are acutely aware of algorithmic power and pursue visibility as if "playing a game"
- Bishop 2017: argues that YouTube algorithm encourages beauty vloggers to create highly gendered videos
- Cultural content on YouTube is highly gendered
- Provocation: digital influencers are selling their own personalities, social identity theory will play a greater role in their success than perhaps in the domain of movies or other cultural items
motivation
user experiences
background research
results
users
creators
RS
methods
user experiences
background research
results
users
creators
RS
items
motivation
methods
user experiences
background research
results
users
creators
RS
motivation
methods
user experiences
background research
results
users
creators
RS
motivation
methods
user experiences
background research
results
users
creators
RS
= recommendations
motivation
methods
user experiences
background research
results
users
creators
RS
motivation
methods
user experiences
background research
results
users
creators
RS
motivation
methods
user experiences
background research
results
users
creators
RS
motivation
methods
motivation
user experiences
background research
results
users
creators
RS
methods
motivation
user experiences
background research
methods
results
s = group affinity strength
user-item utility matrix: U x I
\begin{bmatrix}
a_{11} & a_{12} & \cdots & 0 & s \\
a_{21} & a_{22}& \cdots & s & 0 \\
\vdots & \vdots & \ddots & \vdots & \vdots\\
a_{N1} & a_{N2} & \cdots & s & 0
\end{bmatrix}
Users matrix U:
(N x A)
member of majority group?
member of minority group?
a_{ij},i_{jk}\sim \text{Uniform}(0,1)
Items matrix I:
(A x I)
\begin{bmatrix}
i_{11} & i_{21} & \cdots & i_{I1} \\
i_{12} & i_{22}& \cdots & i_{I2} \\
\vdots & \vdots & \ddots & \vdots \\
0 & 0 & \cdots & 1\\
1 & 1 & \cdots & 0
\end{bmatrix}
created by minority group?
created by majority group?
motivation
user experiences
background research
methods
results
Experimental setup
- 1,000 users (80% majority group, 20% minority group)
- 10,000 items (50% created by majority group, 50% created by minority group)
- Each simulation is run for 100s of iterations, we also run multiple simulations to average out results
- Vary group strength s in {0, 0.1, 0.5, 1.0}
- Vary type of RS algorithm (popularity, content filtering)
Outcomes of interest
- "Dominance" of majority-group items
- At timestep t, what percentage of the recommended items were generated by the minority group?
- Utility experienced by minority group versus utility experienced by majority group
motivation
user experiences
background research
methods
results
Popularity Recommender System
motivation
user experiences
background research
methods
results
Popularity RS
Content Filtering RS
Insight: as group preferences become stronger, recommendations become dominated by majority-group items, even though the content distribution was initially evenly split between groups.
motivation
user experiences
background research
methods
results
Popularity RS
Content Filtering RS
In a popularity-based recommender system, members of majority groups experience higher utility than minority groups. Choice of algorithmic system makes a difference!
- Run experiments with more types of algorithms (matrix factorization, social filtering)
- Model more nuanced group affinity (different levels of group affinity for majority/minority group members)
- Use sparse, binarized item attributes
- Model content creators as dynamic over time, shifting the types of content produced
next steps
thank you!
Bias in AI 11/13
By Matthew Sun
