Epidemiology of representations

an empirical approach

École Doctorale Cerveau-Cognition-Comportement (ED3C, ED n°158)
Centre d’Analyse et de Mathématique Sociales (CAMS, UMR 8557 EHESS-CNRS)

Sébastien Lerique

Pr. Russell Gray
Pr. Fiona Jordan
Dr. Márton Karsai
Pr. Jean-Pierre Nadal
Pr. Sharon Peperkamp
Pr. Camille Roth
Dr. Mónica Tamariz

Reviewer
Reviewer
Examiner
Supervisor
Examiner
Co-supervisor
Examiner

Thesis submitted for the degree of Ph.D. in Cognitive Science

27th October 2017

Cultural evolution

significant transformations

an epidemiology of representations

\(\Longrightarrow\)

Evolutionary principles for culture

Parallel genetic and cultural change

Cultural Attraction Theory (Sperber 1996)

Standard evolutionary theory

Cognitive science

Anthropology

Standard Cultural Evolution

Cavalli-Sforza and Feldman (1981)

Boyd and Richerson (1985, 2005)

Claidière and Sperber (2007)

Claidière, Scott-Phillips, and Sperber (2014)

Transformations of linguistic representations

Morin (2013)

Baumard et al. (2015)

IN VIVO

Claidière et al. (2014)

Moussaïd et al. (2015)

IN VITRO

Adamic et al. (2016)

Historical data

Online data

Transmission chains

Empirical study of cultural evolution

Mesoudi and Whiten (2004)

In vivo online data

Corpus of quotations from a large body of (8.5m) blog posts

August '08 to April '09 (Leskovec et al., 2009)

Groups (and dynamics) of sentences

Clusters 71.6k 45.7k
Quotes 310k 128k
Occurrences 8.16m 2.43m

Raw

Cleaned

frequency

age of acquisition

#letters

#synonyms

clustering

orthographic neighbourhood density

Pakistani President Asif Ali Zardari:

  – “we will not be scared of these cowards”

  ⭢ “we will not be afraid of these cowards”

 

US Senator McCain:

  – “I admire Senator Obama and his accomplishments”

  ⭢ “I respect Senator Obama and his accomplishments”

Sentence reformulations

Word features

#phonemes

#syllables

phonological neighbourhood density

betweenness

degree

pagerank

frequency

age of acquisition

#letters

#synonyms

clustering

orthographic neighbourhood density

#phonemes

#syllables

phonological neighbourhood density

betweenness

degree

pagerank

And word list recall

Deese (1959), Roediger and McDermott (1995) paradigm

Zaromb et al. (2006)

Similar to sentence recall

Potter and Lombardi (1990)

Address the word frequency paradox

Mandler et al. (1982)

Psycholinguistics

Semantic network

Substitution model

Time: continuous / discrete

Source: all / majority

Past: all / last bin

Destination: all / exclude past

Time: continuous / discrete

Source: all / majority

Past: all / last bin

Destination: all / exclude past

6177 substitutions

Susceptibility

\sigma_g = \frac{s_g}{s_g^0}

“This crisis did not develop overnight and it will not be solved overnight”

US President Bush

Replacement word

\nu_{\phi}(f) = \left<\phi(w')\right>_{\left\lbrace w \rightarrow w' | \phi(w) = f \right\rbrace}

“This crisis did not develop overnight and it will not be solved overnight”

US President Bush

“This problem did not develop overnight and it will not be solved overnight”

Sentence context

Susceptibility

Feature variation

“Senior general Than Shwe is foolish with power”

crazy

Stepping back

In vivo psycholinguistics experiment

Transformations \(\rightarrow\) Substitutions

Attractors

Contractile behaviour

Towards words easier to recall

IN VIVO

IN VITRO

Historical data

Transmission chains

Long term evolution

Control over data generation

No complexity sacrifice

Low-level cognitive biases in the wild

Online data

Empirical bind

Realistic content

Control over data-generation

Computational analysis

Already coded

Do-it-by-hand

Simple setting

Danescu-Niculescu-Mizil et al. (2012)

Moussaïd et al. (2015)

Lauf et al. (2013)

Cornish et al. (2013)

Claidière et al. (2014)

Web experiments

Sequence alignments

Case study 1

Requirements fulfilled

Control over experimental setting

Fast iterations

Scale

Exp. 1

MemeTracker, WikiSource,

12 Angry Men, Tales,

News stories

Exp. 2

Memorable/non-memorable quote pairs

(Danescu-Niculescu-Mizil et al., 2012)

Exp. 3

Nouvelles en trois lignes

(Fénéon, 1906)

Web-based experiments

Experiment setup

reading and writing time \(\propto\) number of words

Transformation model

At Dover, the finale of the bailiffs' convention. Their duties, said a speaker, are "delicate, dangerous, and insufficiently compensated."

depth in branch

At Dover, the finale of the bailiffs convention,their duty said a speaker are delicate, dangerous and detailed

At Dover, at a Bailiffs convention. a speaker said that their duty was to patience, and determination

In Dover, at a Bailiffs convention, the speaker said that their duty was to patience.

In Dover, at a Bailiffs Convention, the speak said their duty was to patience

At Dover, the finale of the bailiffs' convention. Their duties, said a speaker, are "delicate, dangerous, and insufficiently compensated."

At Dover, the finale of the bailiffs convention,their duty said a speaker are delicate, dangerous and detailed

Sequence alignments

Needleman and Wunsch (1970)

AGAACT-
 | ||
-G-AC-G


AGAACT

GACG


Finding her son, Alvin, 69, hanged, Mrs Hunt, of Brighton, was so depressed she could not cut him down
Finding her son Arthur 69 hanged Mrs Brown from Brighton was so upset she could not cut him down
Finding her son Alvin  69 hanged Mrs Hunt of -     -    Brighton, was so depressed she could not cut him down
Finding her son Arthur 69 hanged Mrs -    -  Brown from Brighton was so upset she could not cut him down

Apply to utterances using NLP

At Dover, the finale of the bailiffs convention, their duty said a speaker are delicate, dangerous and detailed
At Dover, at a Bailiffs convention. a speaker said that their duty was to patience, and determination
At Dover the finale of the -  - bailiffs convention - -       -    -    their duty
At Dover -   -      -  -   at a Bailiffs convention a speaker said that their duty 


said a speaker are delicate dangerous -   -  -        and detailed -
-    - -       -   -        -         was to patience and -        determination
At Dover the finale of the -  - bailiffs convention |-Exchange-1------| their duty
At Dover -   -      -  -   at a Bailiffs convention a speaker said that their duty 


said a speaker are delicate dangerous -   -  -        and detailed -
|-Exchange-1------------------------| was to patience and -        determination
said a speaker are delicate dangerous |-E2----|
|E2| a speaker -   -        -         said that
said -
said that

\(\hookrightarrow E_1\)

\(\hookrightarrow E_2\)

Extend to build recursive deep alignments

Insertion-deletion chunks

Bursts in branches

Detailed behaviours

Frequency

Frequency

Frequency

|chunk|

|chunk|

Deletion

Insertion

Replacement

Position in \(u\)

\(|u|_w\)

Number of operations vs. utterance length

Susceptibility vs. position in utterance

Deletions tend to gate other operations

Insertions relate to preceding deletions

Stubbersfield et al. (2015)

Bebbington et al. (2017)

Links the low-level with contrasted outcomes

Substitutions in online quotations

Complete transformations in chains

IN VIVO

IN VITRO

Recover susceptibility and variation results

Extend to insertions and deletions

Measure lexical evolution in the long term

Recover online quotations

Realistic content

Control over data-generation

Computational analysis

Already coded

Do-it-by-hand

Simple setting

Empirical (un)bind

Quantitative analysis of complex meaning change

Structural changes from exchanges

Relating insertion and deletion chunks

Inner structure of transformations

Sequence alignments of semantic parses

Further work

In vivo applications to more complete data sets (social networks)

Sentence processing \(\leftrightarrow\) Higher level evolution

Feedback loops: utterance distribution \(\leftrightarrow\) detailed transformations

Long-lived chains with recurring changes

Semantic parsing and NLP methods on the inner structure

Connect to the constitution of meaning in interaction and context

Openings

Semantics

Thank you

Jury

Supervision

Support

Jean-Pierre Nadal & Camille Roth

Family & friends

Substitution models

POS Susceptibility

Feature interactions

\vec{\phi}(w') - \vec{\phi}(w) = \vec{A} + B \cdot \vec{\phi}(w)

Burmese poet Saw Wai (Nov 2008):
  – “Senior general Than Shwe is foolish with power”
  ⭢ “Senior general Than Shwe is crazy with power”


  – "foolish": 8.94 y.o., 675 times, cc of .0082
  ⭢ "crazy": 5.22 y.o., 4100 times, cc of .0017

Live experiment

Data quality

#participants
#root utterances
tree size
Duration
Spam rate
Usable reformulations
53 49 2 x 70
54 50 25/batch
48 49 70
64min 43min 37min/batch
22.4% + 3.5% 0.8% + 0.6% 1% + 0.1%
1980 2411 3506
Exp. 1 Exp. 2 Exp. 3
  • First large-scale launch
    • Bugs and customer service
    • Mistaken UI affordances
  • Extensive rewrite
    • Automated tests
    • Pilots evaluating the UI
    • Pilots sampling root utterances

– “There is no hope for peace, it is a lost cause”

⭢ “There is no lose hope that rara ra to op”

– “My Government's overriding priority is to ensure the stability of the British economy”

⭢ “My governments overall liability is to sort out the... not sure.”

Example data

Immediately after I become president I will confront this economic challenge head-on by taking all necessary steps

immediately after I become a president I will confront this economic challenge

Immediately after I become president, I will tackle this economic challenge head-on by taking all the necessary steps

This crisis did not develop overnight and it will not be solved overnight

the crisis did not developed overnight, and it will be not solved overnight

original

This, crisis, did, not, develop, overnight, and, it, will, not, be, solved, overnight

this, crisis, did, not, develop, overnight, and, it, will, not, be, solved, overnight

this, crisis, did, not, develop, overnight, and, it, will, not, be, solved, overnight

crisi, develop, overnight, solv, overnight

tokenize

lowercase & length > 2

stopwords

stem

The crisis didn't happen today won't be solved by midnight.

crisi, happen, today, solv, midnight

d = 0,6

Utterance-to-utterance distance

Aggregate trends

Size reduction

Transmissibility

Variability

Sequence alignments

Needleman and Wunsch (1970)

AGAACT-
 | ||
-G-AC-G


AGAACT

GACG


Finding her son, Alvin, 69, hanged, Mrs Hunt, of Brighton, was so depressed she could not cut him down

Finding her son Arthur 69 hanged Mrs Brown from Brighton was so upset she could not cut him down
Finding her son Alvin  69 hanged Mrs Hunt of -     -    Brighton, was so depressed she could not cut him down

Finding her son Arthur 69 hanged Mrs -    -  Brown from Brighton was so upset she could not cut him down

Gap open cost \(\rightarrow \theta_{open}\)

Gap extend cost \(\rightarrow \theta_{extend}\)

Item match-mismatch

Applied to utterances

similarity(w, w') = \begin{cases} S_C \left( w, w' \right) & if\ we\ have\ word\ vectors\ for\ both\ w\ and\ w' \\ \delta_{lemma(w), lemma(w')} & otherwise \end{cases}
score(w, w') = similarity(w, w') + \theta_{mismatch}
At Dover, the finale of the bailiffs convention, their duty said a speaker are delicate, dangerous and detailed

At Dover, at a Bailiffs convention. a speaker said that their duty was to patience, and determination
At Dover the finale of the -  - bailiffs convention - -       -    -    their duty 
At Dover -   -      -  -   at a Bailiffs convention a speaker said that their duty 

said a speaker are delicate dangerous -   -  -        and detailed -
-    - -       -   -        -         was to patience and -        determination
At Dover the finale of the -  - bailiffs convention |-Exchange-1------| their duty 
At Dover -   -      -  -   at a Bailiffs convention a speaker said that their duty 

said a speaker are delicate dangerous -   -  -        and detailed -
|-Exchange-1------------------------| was to patience and -        determination
said a speaker are delicate dangerous |-E2----|
|E2| a speaker -   -        -         said that
said -
said that

\(\hookrightarrow E_1\)

\(\hookrightarrow E_2\)

Deep alignments

Alignment optimisation

\(\theta_{open}\)

\(\theta_{extend}\)

\(\theta_{mismatch}\)

\(\theta_{exchange}\) by hand

All transformations

Hand-coded training set size?

Train the \(\theta_*\) on hand-coded alignments

Simulate the training process: imagine we know the optimal \(\theta\)

1. Sample \(\theta^0 \in [-1, 0]^3\) to generate artificial alignments for all transformations

2. From those, sample \(n\) training alignments

3. Brute-force \(\hat{\theta}_1, ..., \hat{\theta}_m\) estimators of \(\theta_0\)

4. Evaluate the number of errors per transformation on the test set

Test set

10x

10x

\(\Longrightarrow\) 100-200 hand-coded alignments yield \(\leq\) 1 error/transformation

Transformation model

B = \frac{\sigma_{intervals} - \mu_{intervals}}{\sigma_{intervals} + \mu_{intervals}}
.22 \leq B \leq .33

Lexical evolution (1)

\sigma_g^- = \frac{s_g^-}{s_g^0}
\sigma_g^+ = \frac{s_g^+}{s_g^0}

Step-wise

Susceptibility

Feature variation \(\nu_{\phi}\)

Lexical evolution (2)

Along the branches

Challenges with meaning

Can you think of anything else, Barbara, they might have told me about that party?

I've spoken to the other children who were there that day.

S

B

Abuser

The Devil's Advocate (1997)

?

Strong pragmatics (Scott-Phillips, 2017)

Access to context

Theory of the constitution of meaning

Challenges