AI, Society, and Human Behavior

Research Methods in Context

Carina I Hausladen

Topics

Four cutting-edge topics at the frontier of computation social science:

1. Measuring Bias in AI
2. Social Choice for LLM Alignment
3. Clustering Multidimensional Time Series — Modeling Human Behavior
4. Modeling Social Dilemmas through Reinforcement Learning

• Research Skills

  • Design your own research question

  • Replicate, extend, or reinterpret topics we discuss

• Applied Methods

  • Analyze real data using computational tools

  • Code in teams to explore your question

  • Build a GitHub repository for open, replicable research

• Communication & Impact

  • Write a short research-style paper

  • Present your insights to others

  • Discussion & active participation

Skills

Activities & Assessment

• 10:00 – 11:30:
  • Carina introducing topics and methods
• 11:30 – 12:30:
  • Lunch (together)
• 12:30 – 14:00:
  • 30 minutes discussants
  • 30 min explaining of core concepts to each other
  • 30 min coding

1. Reading Response

1. Reading Response

• Your response should answer the following:
  1. What is the core idea or contribution?
  2. What questions would you like to ask in class?
  3. What parts of the paper are interesting to you and why?
  4. How would you replicate or extend the paper?
      
• These responses are not graded.
• Responses are contributed via Overleaf.

• Serve as a discussant for one paper (only once!)

• Probably in pairs of two

• Deliver a brief (~7–10 min) presentation, focusing on:

  • Summarize the core idea of the paper

  • Does it introduce an interesting dataset we could utilize?

  • Is there an analysis worth replicating? How could this work be extended*?

    • *who did recently cite this paper?

  • Encourage discussion with your classmates

• Graded (20%)

• Deadline: Thursdays, 10 PM

2. Discussant Role

3. Group Project

• Group Project, delivered as
  • presentation (30%)
  • paper (50%)
• The paper should have around 8 pages and 4,000-8,000 words, and should be structured like a paper.
  • You should include a 'contributions' section outlining what group member did what.
• You should link a Github repo with the code you developed.

Code Clinic

• Checking analysis choices; assessing whether additional statistical tests are needed.
• Do the figures make the point?
• Does your GitHub repository support replication?

Writing Clinic

• Good writing
  • specifically focusing on abstract, figure captions, title
• Good presentations: what makes a talk effective

Presentation and Paper

• Writing is thinking
  • Ideally, the core of your paper is in a good shape before the presentation
  • When do you want to hand in your final paper? 
• Your presentation should also include a short introduction to your GitHub repository

Topics

• No advanced math or ML required

  • Focus on intuition, discussion, and conceptual understanding.

• Choose what interests you

  • You can catch up on background knowledge as needed.

  • Work in groups to support and complement each other’s skills.

• Recommended:

  • Interest in machine learning, social science, or AI ethics

  • Basic probability and statistics

  • Introductory Python programming

Prerequisites

1. Measuring Bias in AI

• Where Bias in AI Appears

  • Hiring

  • Predictive policing

  • Ad targeting

• Sources of Bias

  • Human bias & feedback loops

  • Sample imbalance / unreliable data

  • Model & deployment effects

• ​Fairness Criteria

• Bias and Embeddings

  • ​Word embeddings encode stereotypes

  • Embedding geometry

• Causality

  • Simpson’s Paradox

  • Causal inference

• ​Case Study

Carina I. Hausladen, Manuel Knott, Colin F. Camerer, Pietro Perona

Social perception of faces in a vision-language model

• Preference elicitation
  • Ordinal vs cardinal preferences
  • Methods of elicitation
• From individual to collective choice
  • Fairness and proportionality principles
  • Key properties: monotonicity ...
• Committee elections

• Participatory budgeting (PB)
  • PB as generalization of committee elections
  • Aggregation methods for PB: proportional and cost-aware
• Human-centered LLMs
  • Learning from human preferences (RLHF)
  • Pluralistic alignment

2. Social Choice and
LLM Alignment

Guest Lecture

3. Clustering Multidimensional Time Series

• Behavioral data as multidimensional time series
• Distance Metrics
  • Local
    • e.g. Euclidean Distance
  • Global
    • Dynamic Time Warping (DTW)

• Clustering Methods
  • Hierarchical clustering:
  • PAM (Partitioning Around Medoids)
  • DBSCAN/HDBSCAN: density-based
• Evaluation & Validation
  • Internal indices 
  • External validation 

4. Modeling Social Dilemmas

• Social Dilemma Games
  • Prisoner’s Dilemma, Stag Hunt, Public Goods Game.
  • Emergent dynamics.
• Reinforcement Learning
  • Agents learn from rewards and punishments over time.
• Markov Decision Processes
  • Sequential decision-making under uncertainty.

• Q-Learning
  • learning state–action values through trial and error
  • latest literature on social dilemmas
• Inverse Reinforcement Learning
  • Infer the hidden reward function.
  • Useful in social science: recover fairness concerns, reciprocity, etc.

Identifying Latent Intentions
via
Inverse Reinforcement Learning
in
Repeated Public Good Games

Carina I Hausladen, Marcel H Schubert, Christoph Engel

MAX PLANCK INSTITUTE
FOR RESEARCH ON COLLECTIVE GOODS

Your Next toDos

• Sign up for the course (or not)

• Wait for an announcement from my side

• Select the paper you want to serve as discussant on in class

carinah@ethz.ch

slides.com/carinah

github.com/carinahausladen

S