Perception-based optimal control

$\displaystyle \min_{\pi\in\Pi} ~~\mathrm{cost}(x_0,u_0,x_1,\dots)$

$~~~\mathrm{s.t.}~~ u_t = \pi_t(z_{0:t})$

$~~~~~~~~~~x_{t+1} = \mathrm{dynamics}_t(x_t,u_t)$

$~~~~~~~~~~z_t = \mathrm{observation}_t(Cx_t)~$

Observation-feedback optimal control problem

Perception-based optimal control

$\displaystyle \min_{\pi\in\Pi} ~~\mathrm{cost}(\widehat x_0,u_0,\widehat x_1,\dots)$

$~~~\mathrm{s.t.}~~ u_t = \pi_t(\widehat x_{0:t})$

$~~~~~~~~~~\widehat x_{t+1} = \mathrm{dynamics}_t(\widehat x_t,u_t)$

$~~~~~~~~~~\widehat x_t =\mathrm{EKF}(z_{0:t})~$

Classic approach: physical models and filtering

Perception-based optimal control

$\displaystyle \min_{\pi\in\Pi} ~~\mathrm{cost}(x_0,u_0,x_1,\dots)$

$~~~\mathrm{s.t.}~~ u_t = \widehat \pi_t(z_{0:t})$

End-to-end approach: learn everything from data

Perception-based optimal control

$\displaystyle \min_{\pi\in\Pi} ~~\mathrm{cost}(x_0,u_0,x_1,\dots)$

$~~~\mathrm{s.t.}~~ u_t = \pi_t(y_{0:t})$

$~~~~~~~~~~x_{t+1} = \mathrm{dynamics}_t(x_t,u_t)$

$~~~~~~~~~~y_t = \mathrm{perception}(\mathrm{observation}_t(Cx_t))~$

Our focus: learned perception map

Closed-loop guarantees

How does imperfect perception affect system evolution?

Main Result

$\mathrm{cost}(\widehat\pi) - \mathrm{cost}(\pi_\star) \lesssim$ $rL_g L_h \left(\frac{p^2\sigma_\eta^4}{T}\right)^{\frac{1}{p+4}}$ $\left\|\begin{bmatrix} Q\mathbf \Phi_{xn}\\ R\mathbf \Phi_{un} \end{bmatrix}\right\|_{\mathcal L_1}$

The certainty-equivalent controller has bounded suboptimality w.h.p.

Ingredients

1. Uniform convergence of $\widehat h$

bounded errors

2. Closed-loop performance

propagation of errors

Simulation Experiments

Data-driven perception

Data collected with linear control and periodic reference signal:

Perception errors

Thanks for your attention!

Certainty-Equivalent Perception-Based Control
Sarah Dean and Benjamin Recht

Read more at arxiv.org/abs/2008.12332
Code at github.com/modestyachts/certainty_equiv_perception_control