RTI and MLI at MORFAE Workshop Clausthal

Video from:
Kehrle, F., et. al. (2011), Optimal control of formula 1 race cars in a VDrift based virtual environment

Real-Time Iterations and Multi-Level Iterations

Ihno Schrot

MORFAE Workshop at TU Clausthal on September 21, 2023

OUTLINE

1

Optimal Control Problem Formulation and Nonlinear Model Predictive Control

2

Problem Discretization using Direct Multiple Shooting

3

The Sequential Quadratic Programming Method as Solution Method for the Nonlinear Programs

4

Real - Time Iterations

5

Multi - Level Iterations

1

Optimal Control Problem Formulation and Nonlinear Model Predictive Control

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

\newcommand{\ud}{\mathrm{d}} \begin{aligned} & 0 = g\left(x(t),u(t),p\right),&& t\in[0,T],\\ & 0 \leq h\left(x(t),u(t),p\right),&& t\in[0,T],\\ \end{aligned}

\newcommand{\ud}{\mathrm{d}} \begin{aligned} \min_{x(\cdot),u(\cdot)}~ & \int_{0}^{T} L\left(x(t),u(t),p\right) \ud t + E\left(x(T)\right)\\ \text{s.\,t. } & \end{aligned}

\newcommand{\ud}{\mathrm{d}} \begin{aligned} & \dot{x}(t) = f\left(x(t),u(t),p\right),&& t\in[0,T],\\ & x(0) = x_0,&&\\ \end{aligned}

\newcommand{\ud}{\mathrm{d}} \begin{aligned} & 0 = r^{\mathrm{e}} \left(x(T)\right),&&\\ & 0 \leq r^{\mathrm{i}} \left(x(T)\right)&& \end{aligned}

Objective Function

Dynamical System

Continuous Constraints

Terminal Constraints

State $$x$$

Control $$u$$

Parameter $$p$$

x(t)

\dot{x}(t)

x(0)

u(t)

1. OCP AND NMPC

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

\newcommand{\ud}{\mathrm{d}} \begin{aligned} \min_{x(\cdot),u(\cdot)}~ & \int_{0}^{T} L\left(x(t),u(t),p\right) \ud t + &&E\left(x(T)\right)\\ \text{s.\,t. } \end{aligned}

1. OCP AND NMPC

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

\newcommand{\ud}{\mathrm{d}} \begin{aligned} \min_{\textcolor{#fca311}{x(\cdot)},u(\cdot)}~ & \int_{0}^{T} L\left(\textcolor{#fca311}{x(t)},u(t),p\right) \ud t + &&E\left(x(T)\right)\\ \text{s.\,t. } \end{aligned}

$x$: States

1. OCP AND NMPC

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

\newcommand{\ud}{\mathrm{d}} \begin{aligned} \min_{x(\cdot),\textcolor{#fca311}{u(\cdot)}}~ & \int_{0}^{T} L\left(x(t),\textcolor{#fca311}{u(t)},p\right) \ud t + &&E\left(x(T)\right)\\ \text{s.\,t. } \end{aligned}

1. OCP AND NMPC

$x$: States

$u$: Controls

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

\newcommand{\ud}{\mathrm{d}} \begin{aligned} \min_{x(\cdot),u(\cdot)}~ & \int_{0}^{T} L\left(x(t),u(t),\textcolor{#fca311}{p}\right) \ud t + &&E\left(x(T)\right)\\ \text{s.\,t. } \end{aligned}

1. OCP AND NMPC

$x$: States

$u$: Controls

$p$: Parameters

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

1. OCP AND NMPC

$x$: States

$u$: Controls

$p$: Parameters

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

\newcommand{\ud}{\mathrm{d}} \begin{aligned} \min_{x(\cdot),u(\cdot)}~ & \int_{0}^{T} L\left(x(t),u(t),p\right) \ud t + &&E\left(x(T)\right)\\ \text{s.\,t. } & \dot{x}(t) = f\left(x(t),u(t),p\right),&& t\in[0,T],\\ & x(0) = x_0,&&\\ & 0 = g\left(x(t),u(t),p\right),&& t\in[0,T],\\ & 0 \leq h\left(x(t),u(t),p\right),&& t\in[0,T],\\ \end{aligned}

1. OCP AND NMPC

$x$: States

$u$: Controls

$p$: Parameters

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

1. OCP AND NMPC

$x$: States

$u$: Controls

$p$: Parameters

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

t_k

t_k~~ + T

1. Measure current state $$x_k$$

Repeat:

Illustration based on Grüne and Pannek, Nonlinear Model Predictive Control

1. OCP AND NMPC

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

t_k

t_k~~ + T

2. Solve OCP for horizon $$[t_k,t_k+T]$$

1. Measure current state $$x_k$$

Repeat:

Illustration based on Grüne and Pannek, Nonlinear Model Predictive Control

\newcommand{\ud}{\mathrm{d}} \begin{aligned} \min_{x(\cdot),u(\cdot)}~ & \int_{\textcolor{#FCA311}{t_k}}^{\textcolor{#FCA311}{t_k+T}} L\left(x(t),u(t),p\right) \ud t && + E\left(x(\textcolor{#FCA311}{t_k+T})\right)\\ \text{s.\,t. } & \dot{x}(t) = f\left(x(t),u(t),p\right),&& t\in[\textcolor{#FCA311}{t_k,t_k+T}],\\ & x(\textcolor{#FCA311}{t_k}) = \textcolor{#FCA311}{x_k},&&\\ & 0 = g\left(x(t),u(t),p\right),&& t\in[\textcolor{#FCA311}{t_k,t_k+T}],\\ & 0 \leq h\left(x(t),u(t),p\right),&& t\in[\textcolor{#FCA311}{t_k,t_k+T}],\\ & 0 = r^{\mathrm{e}} \left(x(\textcolor{#FCA311}{t_k+T})\right),&&\\ & 0 \leq r^{\mathrm{i}} \left(x(\textcolor{#FCA311}{t_k+T})\right)&& \end{aligned}

1. OCP AND NMPC

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

t_k

t_k~~ + T

2. Solve OCP for horizon $$[t_k,t_k+T]$$

1. Measure current state $$x_k$$

3. Apply first part of computed contol

Repeat:

Illustration based on Grüne and Pannek, Nonlinear Model Predictive Control

1. OCP AND NMPC

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

t_k

t_k~~ + T

2. Solve OCP for horizon $$[t_k,t_k+T]$$

1. Measure current state $$x_k$$

3. Apply first part of computed contol

4. Progress to time $$t_{k+1}$$

t_{k+1}

Repeat:

Illustration based on Grüne and Pannek, Nonlinear Model Predictive Control

1. OCP AND NMPC

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

Illustration based on Grüne and Pannek, Nonlinear Model Predictive Control

1. OCP AND NMPC

t_k

t_k~~ + T

2. Solve OCP for horizon $$[t_k,t_k+T]$$

4. Progress to time $$t_{k+1}$$

1. Measure current state $x_k$

3. Apply first part of computed contol

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

1. OCP AND NMPC

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

OUTLINE

1

Optimal Control Problem Formulation and Nonlinear Model Predictive Control

2

Problem Discretization using Direct Multiple Shooting

3

The Sequential Quadratic Programming Method as Solution Method for the Nonlinear Programs

4

Real - Time Iterations

5

Multi - Level Iterations

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

1. Introduce Shooting Grid

State $$x(\cdot)$$

Control $$u(\cdot)$$

2. Replace state trajectory by points

3. Replace control trajectory by, e.g., piecewise constant controls

s_0

s_1

s_2

s_N

q_0

q_1

q_2

q_{N-1}

4. Introduce Matching Conditions

\underbrace{x(\tau_{i+1};s_i,q_i)} - s_{i+1} = 0

Infinite-Dimensional

Bock, H. G., & Plitt, K. J. (1984). A multiple shooting algorithm for direct solution of optimal control problems.

\tau_0

\tau_1

\tau_N

2. MULTIPLE SHOOTING

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

Finite-dimensional nonlinear program

2. MULTIPLE SHOOTING

\newcommand{\ud}{\mathrm{d}} \begin{aligned} \min_{x(\cdot),u(\cdot)}~ & \int_{t_k}^{t_k+T} L\left(x(t),u(t),p\right) \ud t && + E\left(x(t_k+T)\right) \end{aligned}

\newcommand{\ud}{\mathrm{d}} \begin{aligned} \min_{\substack{s_0,\ldots,s_N,\\q_0,\ldots,q_N}}~ & \sum_{j=0}^{N} L_j\left(s_j,q_j,p\right) + E\left(s_N\right) && \end{aligned}

\newcommand{\ud}{\mathrm{d}} \begin{aligned} \text{s.\,t. } & \dot{x}(t) = f\left(x(t),u(t),p\right),&& t\in[t_k,t_k+T],\\ & x(t_k) = x_k,&&\\ \end{aligned}

\newcommand{\ud}{\mathrm{d}} \begin{aligned} & 0 = g\left(x(t),u(t),p\right),&& t\in[t_k,t_k+T],\\ & 0 \leq h\left(x(t),u(t),p\right),&& t\in[t_k,t_k+T],\\ & 0 = r^{\mathrm{e}} \left(x(t_k+T)\right),&&\\ & 0 \leq r^{\mathrm{i}} \left(x(t_k+T)\right)&& \end{aligned}

\newcommand{\ud}{\mathrm{d}} \begin{aligned} \text{s.\,t. } & 0 = x(\tau_{j+1};s_j,q_j) - s_{j+1}, && j=0,\ldots,N-1\\ & 0 = x_k - s_0,&&\\ & 0 = q_N - q_{N-1} && \end{aligned}

\newcommand{\ud}{\mathrm{d}} \begin{aligned} & 0 = g\left(s_j,q_j,p\right), && j=0,\ldots,N,\\ & 0 \leq h\left(s_j,q_j,p\right), && j=0,\ldots,N,\\ & 0 = r^{\mathrm{e}}(s_N),&&\\ & 0 \leq r^{\mathrm{i}}(s_N),&&\\ \end{aligned}

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

OUTLINE

1

Optimal Control Problem Formulation and Nonlinear Model Predictive Control

2

Problem Discretization using Direct Multiple Shooting

3

The Sequential Quadratic Programming Method as Solution Method for the Nonlinear Programs

4

Real - Time Iterations

5

Multi - Level Iterations

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

\begin{aligned} &\min_x & & f(x)\\ & ~~\text{s.t.} & & g(x) = 0 \end{aligned}

F(x)=0

Nonlinear System of Equations

SQP Method

Step of Newton's Method

Optimality Conditions

Quadratic Approximation

Quadratic Program

\begin{aligned} &\min_{\Delta x} & &\frac{1}{2}\Delta x ^T \nabla_{xx}^2\mathcal{L}\left(x_k,\lambda_k\right)\Delta x + \nabla_x f(x_k)^T\Delta x\\ & ~~\text{s.t.} & & g(x_k) + \nabla_x g(x_k)^T \Delta x_k = 0 \end{aligned}

\text{Lagrangian:}\\ \mathcal{L}(x,\lambda)\coloneqq f(x) - \lambda^T g(x)

Step $$\Delta x$$

Current Guess $$x_k$$

Solving the QP

Update

3. SQP METHOD

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

In the case of Multiple Shooting:

QPs are highly structured
QPs can be condensed so that they only depend on $\Delta s_0, \Delta q_0,\ldots,\Delta q_{N-1} $

\begin{aligned} &\min_{\textcolor{#FCA311}{\Delta w}} & &\frac{1}{2}\textcolor{#FCA311}{\Delta w} ^T B_k \textcolor{#FCA311}{\Delta w} + b_k ^T \textcolor{#FCA311}{\Delta w}\\ & ~~\text{s.t.} & & 0 = C_k \textcolor{#FCA311}{\Delta w} + c_k,\\ & & & 0 \leq D_k \textcolor{#FCA311}{\Delta w} + d_k,\\ & & & 0 = \textcolor{#FCA311}{\Delta s_0} + (s_0-x_k) \end{aligned}

\begin{aligned} &\min_{\Delta w} & &\frac{1}{2}\Delta w ^T \textcolor{#FCA311}{B_k} \Delta w + \textcolor{#FCA311}{b_k^T} \Delta w\\ & ~~\text{s.t.} & & 0 = C_k \Delta w + c_k,\\ & & & 0 \leq D_k \Delta w + d_k,\\ & & & 0 = \Delta s_0 + (s_0-x_k) \end{aligned}

\begin{aligned} &\min_{\Delta w} & &\frac{1}{2}\Delta w ^T B_k \Delta w + b_k ^T \Delta w\\ & ~~\text{s.t.} & & 0 = \textcolor{#FCA311}{C_k} \Delta w + \textcolor{#FCA311}{c_k},\\ & & & 0 \leq \textcolor{#FCA311}{D_k} \Delta w + \textcolor{#FCA311}{d_k},\\ & & & 0 = \Delta s_0 + (s_0-x_k) \end{aligned}

\begin{aligned} &\min_{\Delta w} & &\frac{1}{2}\Delta w ^T B_k \Delta w + b_k ^T \Delta w\\ & ~~\text{s.t.} & & 0 = C_k \Delta w + c_k,\\ & & & 0 \leq D_k \Delta w + d_k,\\ & & & \textcolor{#FCA311}{0 = \Delta s_0 + (s_0-x_k)} \end{aligned}

$\Delta w $: (Primal) Variables including $\Delta s_0 $

$B_k $: Hessian of Lagrangian

$b_k $: Objective gradient

$C_k, D_k $: Jacobians of (in-)equality constraints

$c_k, d_k $: Residuals of (in-)equality constraints

QP at sampling time $t_k$:

3. SQP METHOD

Initial Value Embedding Constraint

Depend only on $w_k$

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

OUTLINE

1

Optimal Control Problem Formulation and Nonlinear Model Predictive Control

2

Problem Discretization using Direct Multiple Shooting

3

The Sequential Quadratic Programming Method as Solution Method for the Nonlinear Programs

4

Real - Time Iterations

5

Multi - Level Iterations

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

Observations:

Exact controls cannot be computed without delay
$\rightarrow$ Rather compute approximate solutions quickly!
Subsequent OCPs parametrized by current state $x_k$

\begin{aligned} &\min_{\Delta w} & &\frac{1}{2}\Delta w ^T B_k \Delta w + b_k ^T \Delta w\\ & ~~\text{s.t.} & & 0 = C_k \Delta w + c_k,\\ & & & 0 \leq D_k \Delta w + d_k,\\ & & & 0 = \Delta s_0 + (s_0-\textcolor{#FCA311}{x_k}) \end{aligned}

Efficient solution method: Parametric Active-Set Method implemented in qpOASES

Diehl, M., et. al. (2002). Real-time optimization and nonlinear model predictive control of processes governed by differential-algebraic equations.

1st key idea of RTI:

Perform only a single SQP iteration per sample time

Solve:

Ferreau, H. J., et. al. (2014). qpOASES: A parametric active-set algorithm for quadratic programming.

4. REAL-TIME ITERATIONS

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

Observation: Current state $x_k$ enters OCPs linearly

2nd key idea of RTI:

Split computations in preparation and feedback phase

\begin{aligned} &\min_{\Delta w} & &\frac{1}{2}\Delta w ^T \textcolor{#FCA311}{B_k} \Delta w + \textcolor{#FCA311}{b_k ^T} \Delta w\\ & ~~\text{s.t.} & & 0 = \textcolor{#FCA311}{C_k} \Delta w + \textcolor{#FCA311}{c_k},\\ & & & 0 \leq \textcolor{#FCA311}{D_k} \Delta w + \textcolor{#FCA311}{d_k},\\ & & & \textcolor{#14213D}{0 = \Delta s_0 + (s_0-x_k)} \end{aligned}

Preparation Phase for $t_k$

t_{k-1}

t_{k}

Feedback

4. REAL-TIME ITERATIONS

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

OUTLINE

1

Optimal Control Problem Formulation and Nonlinear Model Predictive Control

2

Problem Discretization using Direct Multiple Shooting

3

The Sequential Quadratic Programming Method as Solution Method for the Nonlinear Programs

4

Real - Time Iterations

5

Multi - Level Iterations

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

Observation: Linearizations can be valid for a longer time period

Key idea of MLI:

Reduce expensive evaluations by updating the QPs in 4 levels

Wirsching, L. (2018). Multi-Level Iteration Schemes with Adaptive Level Choice for Nonlinear Model Predictive Control.

Level D = RTI

Level C

Level B

Level A

Computational Complexity

Accuracy

Level D = RTI

Level C

Level B

Level A

Levels can run in parallel and communicate with each other.

5. MULTI-LEVEL ITERATIONS

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

Level D = RTI

Computational Complexity

Accuracy

Level D = RTI

Level C

Level B

Level A

Level C

Level B

Level A

5. MULTI-LEVEL ITERATIONS

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

\begin{aligned} &\min_{\Delta w} & &\frac{1}{2}\Delta w ^T \textcolor{#FCA311}{B_k} \Delta w + \textcolor{#FCA311}{b_k ^T} \Delta w\\ & ~~\text{s.t.} & & 0 = \textcolor{#FCA311}{C_k} \Delta w + \textcolor{#FCA311}{c_k},\\ & & & 0 \leq \textcolor{#FCA311}{D_k} \Delta w + \textcolor{#FCA311}{d_k},\\ & & & 0 = \Delta s_0 + (s_0-\textcolor{#FCA311}{x_k}) \end{aligned}

Computational Complexity

Accuracy

Level D = RTI

Level C

Level B

Level A

Level D = RTI

Level C

Level B

Level A

Full linearization iterations

5. MULTI-LEVEL ITERATIONS

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

Computational Complexity

Accuracy

Level D = RTI

Level C

Level B

Level A

Level D = RTI

Full linearization iterations

Level C

Level B

Level A

Optimality Iterations

\begin{aligned} &\min_{\Delta w} & &\frac{1}{2}\Delta w ^T B_k \Delta w + \textcolor{#FCA311}{b_k ^T} \Delta w\\ & ~~\text{s.t.} & & 0 = C_k \Delta w + \textcolor{#FCA311}{c_k},\\ & & & 0 \leq D_k \Delta w + \textcolor{#FCA311}{d_k},\\ & & & 0 = \Delta s_0 + (s_0-\textcolor{#FCA311}{x_k}) \end{aligned}

No new Hessian and (full) Jacobians!

New Jacobians enter as matrix-vector products only

5. MULTI-LEVEL ITERATIONS

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

Computational Complexity

Accuracy

Level D = RTI

Level C

Level B

Level A

Level D = RTI

Full linearization iterations

Level C

Level B

Level A

Optimality Iterations

Feasibility Iterations

No new Jacobians involved

5. MULTI-LEVEL ITERATIONS

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

Computational Complexity

Accuracy

Level D = RTI

Level C

Level B

Level A

Level D = RTI

Full linearization iterations

Level C

Level B

Level A

Optimality Iterations

Feasibility Iterations

Feedback Iterations

5. MULTI-LEVEL ITERATIONS

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

4. MLI: Reduce expensive evaluations by updating the QPs in 4 levels

3. RTI: Solve only one QP per sampling time and split computations in preparation and feedback phase

2. Apply an SQP-type method to the NLPs

1. The $\infty$-dim. OCP is discretized using Multiple Shooting

SUMMARY

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

4. REAL-TIME ITERATIONS

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023

5. MULTI-LEVEL ITERATIONS

Uncontrolled chain

Chain controlled with MLI

Videos from: Wirsching, L. (2018). Multi-Level Iteration Schemes with Adaptive Level Choice for Nonlinear Model Predictive Control.

Ihno Schrot — Real-Time Iterations and Multi-Level Iterations — MORFAE Workshop Clausthal — September 21, 2023