Anticipating changes:
Decision making with temporal expectations
Dimitrije Marković
Bernstein Conference 2020
Satellite Workshop: "Dynamic probabilistic inference in the brain"
Dynamic probabilistic inference
- How is uncertainty represented and updated?
- How is approximate inference implemented?
- How is spatio-temporal structure of our natural environment represented?
Introduce a computational model that represents temporal structure of a dynamic environment.
Infer learned temporal structure from human behaviour.
Anticipating changes
Recent empirical evidence of neuronal circuitry supporting anticipatory behaviour:
- A Vilà-Balló, et al. Journal of Neuroscience (2017).
- VD Costa, et al. Journal of Neuroscience (2015).
Accurate temporal representation → anticipating events.
Anticipating changes
Marković, et al. PLoS computational biology (2019).
Outline
- Probabilistic reversal learning task
- Behavioural model
- Model-based data analysis
- Learning the hidden temporal structure
- Conclusion
Probabilistic reversal learning
Probabilistic reversal learning
Probabilistic reversal learning
- Probabilistic reversal learning task
- Behavioural model
- Model-based data analysis
- Learning the hidden temporal structure
- Conclusion
Representing duration statistics
two hidden states
st∈{A,B}
Hidden semi-Markov model
Transition probability
p(st+1∣st,ft)={I2,J2−I2, for ft<n+1 for ft=n+1
Duration probability
p(ft+1∣ft)→p(d)
Discrete phase-type distribution
Phase transitions
p(ft∣ft−1)
M Varmazyar, et al., Journal of Industrial Engineering International (2019).
Discrete phase-type distribution
Phase transitions
p(ft∣ft−1)
Duration distribution
p(d)=(d−1d+n−2)(1−δ)d−1δn
M Varmazyar, et al., Journal of Industrial Engineering International (2019).
Behavioural model
K Friston, et al., Neural computation (2017).
history of past outcomes and choices Ht−1=(ot−1:1,at−1:1)
belief updating (Bayes rule)
p(st,ft∣Ht) = p(ot∣at,Ht−1)p(ot∣st,at)p(st,ft∣Ht−1)
Generative process
Action selection
- Probabilistic reversal learning task
- Behavioural model
- Model-based data analysis
- Learning the hidden temporal structure
- Conclusion
Behavioural data
- 50 healthy volunteers (20-30 years old):
- 27 subjects in the condition with regular reversals
- 23 subjects in the condition with irregular reversal
- 40 trials long training with a single reversal
learning phase
model fitting
model testing
Model selection
Inferred duration distribution
Trials until correct (TUC)
Perfomance
Group level trajectories
Posterior samples vs data
- Probabilistic reversal learning task
- Behavioural model
- Model-based data analysis
- Learning the hidden temporal structure
- Conclusion
Learning temporal structure
Condition with regular reversals
Condition with irregular reversals
duration [d]
duration [d]
- Probabilistic reversal learning task
- Behavioural model
- Model-based data analysis
- Learning the hidden temporal structure
- Conclusion
Conclusion
- Modelling and assessing influence of temporal-expectations on decision-making in dynamic environments.
- How people learn temporal expectations could also be addressed with this approach, but some challenges remain.
- Linking the underlying representation of the temporal structure to behaviour provides a novel method for computational cognitive phenotyping.
Thanks to:
- Stefan Kiebel
- Andrea Reiter
- Thomas Parr
- Karl Friston
- Sebastian Bitzer
