Goal

Local minima

Consultant

We can do better than random or exhaustive search

Multi-Armed bandit

Maximize some reward function through balancing exploration & exploitation

... failure and invention are inseparable twins. To invent you have to experiment, and if you know in advance that it's going to work, it's not an experiment. Most large organizations embrace the idea of invention, but are not willing to suffer the string of failed experiments necessary to get there.

Jeff Bezos

Uncertainty

Aleatoric & Epistemic

Uncertainty

natural randomness vs incomplete model/map of situation

How do we improve this map?

Epistemic uncertainty

Uncertainty

Aleatory variability 

P(tiger | friend saw tiger) ?

P(terrorist attack | terrorism attack happened yesterday) ?

Local vs Global knowledge 

Loss aversion due to global knowledge hampers exploration within decision space !

P(terrorist attack | terrorism attack happened yesterday) ?

-

-

-

-

+

- use two-way door decisions as exploration tool 

- This reduces epistemic uncertainty in your environment

- A more precise map allows for long-term planning

- Beware loss aversion

- Backtracking is good!

Key Takeaways