explaining for the best intervention

yuan meng

april 25, 2022

exit talk 🐻

exit talk

how do we learn about the world?

learning by observing

"a painting of 20 athletes playing a brand new sport"

dall·e 2 (openai, 2022)

palm ( google ai, 2022 )

trained on broad data at scale + adaptable to wide range of downstream tasks

how else do people learn?

anti-depressants

mood

thoughts

anti-depressants

mood

thoughts

🤔

learning by experimenting

intervene on mood (e.g., exercise) without anti-depressants 👉 are thoughts affected?

no

yes

data isn't always sufficient or clear...

learning by thinking

image credit: unsplash @maxsaeling

two well-known facts

explanation

the night sky is not filled with starlight; there's darkness in between

if light has travelled infinitely long, it will have filled the night sky

why?🤨

light has only travelled for a finite amount of time 👉 the universe may have a beginning

don't always need new data!

designing good experiments is challenging...

can thinking lead to better experimenation?

intervention

explaining (cause: why, mechanism: how...)

can explaining lead to better intervention?

where the quest began...

evidence on self-explaining effect

explaining for the best inference

walker et al. (2014)

walker and lombrozo (2017)

williams and lombrozo (2010)

explaining helps children and adults acquire generalizable abstractions useful for future scenarios (lombrozo, 2016)

inductive rich properties

the moral of the story

subtle yet broad patterns

explaining may help learners pinpoint own uncertainty or grasp the difference-making principle behind effective intervention 

"trouble-shooting"

intervention design

legare (2012)

why is it suddenly working?

why does it stop working?

explanations generated by reinforcement learning agent 

lampinen et al. (2021)

other domains: question-asking (ruggeri et al., 2019), choosing to observe more in multi-armed bandit problem (liquin & lombrozo, 2017)...

explaining for the best intervention?

outline of talk

  • study 1: natural intervention strategies of children (5- to 7-year-olds) + adults
  • study 2: ask to explain interventions (5- to 7-year-olds + adults) 
  • study 3: train to explain interventions (7- to 11-year-olds)

can explaining lead to better intervention?

task desiderata

  • easy to quantify intervention quality

adapted from coenen et al. (2015)

😵‍💫

light bulb game

task desiderata

  • easy to quantify intervention quality
  • suitable for wide age range

adapted from coenen et al. (2015)

optimal strategy

expected information gain (EIG)

\mathrm{I}(x_i) =-\log_2 {\mathrm{P}(x_i)} = -\log_2 \frac{1}{2} = 1

1. surprisal of each hypothesis

2. average surprisal = information entropy (shannon, 1948)

\mathrm{H}(X) = \sum_{i}^{n}\mathrm{P}(x_i)\mathrm{I}(x_i) = -\sum_{i}^{n}\mathrm{P}(x_i)\log \mathrm{P}(x_i) = 1

3. reduction in entropy after intervention (e.g., turning on green) and outcome = information gain

\mathrm{IG}(a) = \mathrm{H}(X) - \mathrm{H}(X|a,y) = 1 - 0 = 1

expected information gain over all outcomes 👉 guides intervention

don't know what's gonna happen!

\mathrm{EIG}(a) = \mathrm{H}(X) - \sum_{j}^{m}\mathrm{P}(y_i|a)\mathrm{H}(X|a,y_j) = 1

difference making

1

0

0

suboptimal strategy

positive test strategy (PTS)

test one hypothesis at a time

  1. begin with one hypothesis  
  2. score each intervention by the proportion of links it can affect

(coenen et al., 2015; mccormack et al., 2016; meng et al., 2018; meng & xu, 2019; nussenbaum et al., 2020; steyvers et al. 2003)

not being considered yet

1

0.5

0

why suboptimal?

  • EIG: 1 intervention
  • PTS: 1 or more

suboptimal strategy

positive test strategy (PTS)

\mathrm{PTS}(a) = \max_{g}[\frac{DescendantLinks{a,g}}{TotalLinks_g}]

test one hypothesis at a time

  1. begin with one hypothesis  
  2. score each intervention by the proportion of links it can affect
  3. repeat for the other hypothesis

take the maximum across hypotheses

: yellow

(coenen et al., 2015; mccormack et al., 2016; meng et al., 2018; meng & xu, 2019; nussenbaum et al., 2020; steyvers et al. 2003)

1

0.5

0

1

0

0

computational modeling:

infer weight of EIG

V(a) = \theta \cdot \mathrm{EIG}(a) + (1 - \theta) \cdot \mathrm{PTS}(a)

1. each person's weight of EIG: θ

2. value of an intervention

3. probability choosing intervention

\mathrm{P}(a) = \frac{\exp(V(a)/\tau)}{\sum_{a \in A}\exp(V(a)/\tau)}
  • τ 👉 ∞: maximize V(a)
  • τ 👉 ∞: choose randomly

infer each person's θ and τ from  interventions they chose

n × 6 × 3 tensor (unobserved): probability of each of n people choosing each of 3 interventions in each of 6 puzzles

n-vector (unobserved): each of n people’s EIG weight

n-vector (unobserved): each of n people’s "temperature"

6 × 3 matrix (observed): EIG score of each of 3 interventions in each of 6 puzzles

6 × 3 matrix (observed): PTS score of each of 3 interventions in each of 6 puzzles

n × 6 matrix (observed): each of n people’s choices in each of 6 puzzles

adapted from coenen et al. (2015)

study 1

baseline startegy

5- to 7-year-olds: n = 39

adults: n = 29

meng, bramley, and xu (2018)

mccormack et al. (2016)

intervention strategy

tl;dr: adults mainly used EIG vs. children mainly PTS

0.86

0.38

0.77

0.24

averaged across all participants and all puzzles: chance = 1/3

learn from interventions

tl:dr: children didn't learn from informative interventions like adults did

0.91

0.45

only included trials where EIG = 1 (same later)

does explaining intervention choices help?

5- to 7-year-olds didn't choose informative interventions or learn from outcomes

study 2

asked to explain

5- to 7-year-olds: n = 59 (explain); n = 58 (report)

adults: n = 30 (explain); n = 27 (report)

meng and xu (2019)

before choosing

"after deciding on which light bulb you wanna use, don't turn it on yet! point to it and I'll ask you…"

  • explanation: "… why you choose that one to find the answer."
  • report (control): "… which one you choose to find the answer."

afterwards

"cool, can you tell me…"

  • explanation: "… why do you wanna use that light bulb to help?"
  • report (control): "… which light bulb do you wanna use to help?"

if choice changed, use last one for modeling

intervention strategy

tl:dr: explaining had no significant impact on intervention strategies

0.86

0.38

0.95

0.94

0.44

0.39

baseline 👉 explanation 👉 report

0.76, 0.86, 0.84

0.24, 0.28, 0.21

learn from interventions

tl:dr: children who either explained or reported were able to learn from interventions 

0.91

0.97

0.94

0.45

0.80

0.65

*

*

*

why no effect on intervention strategies?

hypothesis: explanation quality matters?

wrong explanations likely relevant to the task

  • ​explain an outcome: wrong explanations are likely still relevant to the task
  • explain an intervention: recognize uncertainty, simulate possible interventions + outcomes, reassess uncertainty after an intervention-outcome pair...

wrong explanations don't target intervention selection at all

why no effect on intervention strategies?

hypothesis: does the quality of explanations matter?

  • study older children who explain better

confounding!

intelligence, education...

explanation type

intervention strategy

why no effect on intervention strategies?

hypothesis: does the quality of explanations matter?

  • study older children who explain better
  • train to generate EIG-based explanations 👉 does it change intervention strategies?

study 3

trained to explain

7- to 8-year-olds: n = 31 (explain), n = 27 (training)

9- to 11-year-olds: n = 30 (explain), n = 30 (training)

(control condition in progress)

same as in study 2

first trial as training

EIG training 

"if alex turns on the {yellow, green, red} light bulb, can they find out the answer?" (correct if wrong)

explanation condition: explanation types

tl:dr: without training, PTS is the most common type in 7- to 8-year-olds and EIG in 9- to 11-year-olds

true rationale unknown

tl:dr: EIG was the dominant type during training; only remained so in 9- to 11-year-olds after training

training condition:

explanation types

training trials (first puzzle)

non-training trials (last 5)

explanation condition 

EIG score by explanation type

tl;dr: EIG-based explanations 👉 informative interventions; PTS-based explanations 👉 uninformative interventions

condition: explanation

condition: training

no 9- to 11-year-olds explained this way

intervention strategy

tl:dr: 9- to 11-year-olds relied far more on EIG after training; 7- to 8-year-olds weren't affected as much

0.44

0.86

0.64

0.59

on a par with adult baseline (study 1)

explanation 👉 training

0.26, 0.35

0.45, 0.71

*

*

(training trials excluded from all analyses)

*

learn from interventions

tl:dr: all children chose correct structures after informative interventions (EIG = 1) most of the time

0.90

0.89

0.94

0.95

key findings

  • didn't help (study 1: natural strategies) 👉 adults mainly used EIG vs. 5- to 7-year-olds mainly used PTS to select interventions
  • "socrates method" (studies 2 & 3: ask to explain) 👉 did not change adults' or children's (5- to 11-year-olds) intervention strategies 
  • teach the principle (study 3: train to explain) 👉 brief training in EIG maximization led 9- to 11-year-olds to rely as much on EIG as adults

can explaining lead to better intervention?

6 years later...

thinking alone doesn't lead to better experimentation, but training the prepared mind to think might

open question: why use PTS at all?

  • resource rationality (griffiths et al. 2015; lieder & griffiths, 2020)?
    • quality: EIG takes fewer steps to find the answer than PTS
    • ​cost: EIG requires more costly computations than PTS
3 ^ {4 \choose 2} = 3 ^ 6 = 729

# of structures:

# of costly computations (hold all structures in mind)

  • EIG: 4 vs. PTS: 1

future research: manipulate cost-quality balance

  • can learners reach an optimal trade-off between computational cost and information gain?

explain the "inexplicable" 

worse case 👉 test one edge at a time: 6 × 2 = 12 interventions

gonna be complicated...

each edge: 2 interventions at most

explain the "inexplicable" 

open question: what does PTS really mean in causal learning?

  • goals: make everything vs. same thing happen?
  • formalizations: 4 ways to define PTS 
    • ​proportion of links: max (y = g), mean (y < g)
    • number of links: max (y > g), mean (y = g)

make everything happen? e.g., "yellow is most useful because it may turn on every single one."

make same thing happen? e.g., "green always goes to red, so it helps in both examples."

future research: unpack PTS

  • personalization: model PTS according to PTS explanations given
  • model comparison: fit different formalizations to human data

examples of PTS-based explanations

1 (0.5)

1 (1)

2 (1)

0 (0)

0 (0)

0 (0)

explain the "inexplicable" 

open question: why did older children benefit more from training?

  • EIG training may require readiness in counterfactual reasoning, working memory, etc.

nussenbaum et al. (2020)

EIG weight

decision noise

~0.38

future research: more elaborative training for younger children?

  • train children to think beyond why, but also what if, why else... (nyhout et al. 2019; engle & walker, 2021)

acknowledgments

  • committee: fei xu, alison gopnik, mahesh srinivasan, neil bramley
  • labmates: rongzhi liu, ruthe foushee, shaun o'grady, elena luchkina, stephanie alderete, roya baharloo, kaitlyn tang, emma roth, gwyneth heuser, harmonie strohl, phyllis lun...
  • research assistants & interns: eliza huang, anqi li, sophie peeler, shengyi wu, jessamine li, selena cheng, chelsea leung, nina li, stella rue...
  • for your kindness: silvia bunge, julie aranda

"it is the usual fate of mankind to get things done in some boggling way first, and find out afterward how they could have been done much more easily and perfectly."

— charles s. pierces (1882)

the phd grind 

"i procrastinated more than i had ever done in my life thus far: i watched lots of tv shows, took many naps, and wasted countless hours messing around online.

unlike my friends with nine-to-five jobs, there was no boss to look over my shoulder day to day, so i let my mind roam free without any structure in my life.

having full intellectual freedom was actually a curse, since i was not yet prepared to handle it." (p. 16-17)

guo (2012, 2015)

"so, was it fun?

i'll answer using another f-word: it was fun at times, but more importantly, it was fulfilling.

fun is often frivolous, ephemeral, and easy to obtain, but true fulfillment comes only after overcoming significant and meaningful challenges." (p. 107)

well into my 3rd year: hooked by data science 👉re-realized coolness & importance of self-supervised learning + bayes + information theory... or just scientific computing in general (post on data science)

questions?