Anticipating changes:
Decision making with temporal expectations
Dimitrije Marković
Bernstein Conference 2020
Satellite Workshop: "Dynamic probabilistic inference in the brain"
Dynamic probabilistic inference
- How is uncertainty represented and updated?
- How is approximate inference implemented?
- How is spatio-temporal structure of our natural environment represented?
Introduce a computational model that represents temporal structure of a dynamic environment.
Infer learned temporal structure from human behaviour.
Anticipating changes
Recent empirical evidence of neuronal circuitry supporting anticipatory behaviour:
- A Vilà-Balló, et al. Journal of Neuroscience (2017).
- VD Costa, et al. Journal of Neuroscience (2015).
Accurate temporal representation → anticipating events.
Anticipating changes
Marković, et al. PLoS computational biology (2019).
Outline
- Probabilistic reversal learning task
- Behavioural model
- Model-based data analysis
- Learning the hidden temporal structure
- Conclusion
Probabilistic reversal learning
Probabilistic reversal learning
Probabilistic reversal learning
- Probabilistic reversal learning task
- Behavioural model
- Model-based data analysis
- Learning the hidden temporal structure
- Conclusion
Representing duration statistics
two hidden states
st∈{A,B}
Hidden semi-Markov model
Transition probability
p(st+1∣st,ft)={I2,J2−I2, for ft<n+1 for ft=n+1
Duration probability
p(ft+1∣ft)→p(d)
Discrete phase-type distribution
Phase transitions
p(ft∣ft−1)
πi=(i−1n)(1−δ)n−i−1δi−1,π0=1−∑i=1nπi
\pi_i = {n \choose i - 1} (1 - \delta)^{n - i -1}\delta^{i-1}, \:\: \pi_0 = 1 - \sum_{i=1}^n \pi_i
M Varmazyar, et al., Journal of Industrial Engineering International (2019).
Discrete phase-type distribution
Phase transitions
p(ft∣ft−1)
Duration distribution
p(d)=(d−1d+n−2)(1−δ)d−1δn
E[d]=δn(1−δ)+1=μ+1
\begin{aligned}
E[d]&=\frac{n (1-\delta)}{\delta}+1 \\
&= \mu + 1
\end{aligned}
Var[d]=μ+nμ2
Var[d] = \mu + \frac{\mu^2}{n}
πi=(i−1n)(1−δ)n−i−1δi−1,π0=1−∑i=1nπi
\pi_i = {n \choose i - 1} (1 - \delta)^{n - i -1}\delta^{i-1}, \:\: \pi_0 = 1 - \sum_{i=1}^n \pi_i
M Varmazyar, et al., Journal of Industrial Engineering International (2019).
Behavioural model
K Friston, et al., Neural computation (2017).
history of past outcomes and choices Ht−1=(ot−1:1,at−1:1)
belief updating (Bayes rule)
p(st,ft∣Ht) = p(ot∣at,Ht−1)p(ot∣st,at)p(st,ft∣Ht−1)
p(at=a∣Ht−1)=∑ue−β(Vtu+αIGtu)e−β(Vta+αIGta)
p(a_{t} = a|H_{t-1}) = \frac{e^{-\beta \left(V_{t}^a + \alpha IG^a_{t} \right)}}{\sum_u e^{-\beta \left( V_{t}^{u} + \alpha IG^u_{t} \right) }}
bt
b_t
ot
o_t
at
a_t
st′
s'_t
bt+1
b_{t+1}
ot+1
o_{t+1}
at+1
a_{t+1}
st+1′
s'_{t+1}
bt−1
b_{t-1}
ot−1
o_{t-1}
at−1
a_{t-1}
st−1′
s'_{t-1}
Generative process
Action selection
bt+1≡∑st,ftp(st+1,ft+1∣st,ft)p(st,ft∣Ht)
b_{t+1} \equiv \sum_{s_t, f_t} p(s_{t+1}, f_{t+1}|s_t, f_t) p(s_t, f_t|H_t)
- Probabilistic reversal learning task
- Behavioural model
- Model-based data analysis
- Learning the hidden temporal structure
- Conclusion
Behavioural data
- 50 healthy volunteers (20-30 years old):
- 27 subjects in the condition with regular reversals
- 23 subjects in the condition with irregular reversal
- 40 trials long training with a single reversal
learning phase
model fitting
model testing
Model selection
Inferred duration distribution
Trials until correct (TUC)
Perfomance
Group level trajectories
Posterior samples vs data
- Probabilistic reversal learning task
- Behavioural model
- Model-based data analysis
- Learning the hidden temporal structure
- Conclusion
Learning temporal structure
Condition with regular reversals
Condition with irregular reversals
duration [d]
duration [d]
- Probabilistic reversal learning task
- Behavioural model
- Model-based data analysis
- Learning the hidden temporal structure
- Conclusion
Conclusion
- Modelling and assessing influence of temporal-expectations on decision-making in dynamic environments.
- How people learn temporal expectations could also be addressed with this approach, but some challenges remain.
- Linking the underlying representation of the temporal structure to behaviour provides a novel method for computational cognitive phenotyping.
Thanks to:
- Stefan Kiebel
- Andrea Reiter
- Thomas Parr
- Karl Friston
- Sebastian Bitzer
Anticipating changes: Decision making with temporal expectations Dimitrije Marković Bernstein Conference 2020 Satellite Workshop: "Dynamic probabilistic inference in the brain"
Anticipating changes
By dimarkov
Anticipating changes
- 101
