Teaching Modern Control
with Modern Tools

Insights from ETH Zurich's Computational Control Course

Saverio Bolognani

24 October 2024

IfA - Automatic Control Laboratory

3 faculty members (Lygeros, Dörfler, Smith) +1

a few Senior Scientists

15 Postdocs, 30+ PhD students

Computational Control

First taught: Spring 2023
Approx 70 students for credits, 100+ attending
Last-year B.Sc. students + first-year M.Sc. students
- Electrical Engineering
- Mechanical Engineering
- Robotics, Syst. & Control
- Energy Science and Technology
- Computer Science
- Biomedical Eng., Quantum Eng., ...
- Space next year?

What goes into the

"second control course"?

Process automation, concept of control. Modelling of dynamical systems - examples, state space description, linearisation, analytical/numerical solution. Laplace transform, system response for first and second order systems - effect of additional poles and zeros. Closed-loop control - idea of feedback. PID control, Ziegler - Nichols tuning. Stability, Routh-Hurwitz criterion, root locus, frequency response, Bode diagram, Bode gain/phase relationship, controller design via "loop shaping", Nyquist criterion. Feedforward compensation, cascade control. Multivariable systems (transfer matrix, state space representation), multi-loop control, problem of coupling, Relative Gain Array, decoupling, sensitivity to model uncertainty. State space representation (modal description, controllability, control canonical form, observer canonical form), state feedback, pole placement - choice of poles. Observer, observability, duality, separation principle. LQ Regulator, optimal state estimation.

What control design tasks can you already solve?

What is blocking you from solving more complicated ones?

What goes into the

"second control course"?

Group activity (first day of class)

For each design problem:

Do you think you have the necessary competences?
Would you take up the challenge?
If yes, what methods would you use?
If no, what’s blocking you?

Insight: Bring props.

Show students what a controller is.

Systems in which the controller is an embedded computer that can sense and actuate a physical plant.

Insight: Tell the students what kind of experts they will become

Suitable for systems where model uncertainty, stringent constraints, and complex dynamics call for advanced control solutions.

Course Content

(13 weeks)

Dynamic Programming and LQR
Model Predictive Control
Economic MPC
Robust MPC
Data-Driven Predictive Control
Markov Decision Processes
Monte Carlo Learning (episodic)
Reinforcement Learning (online)

https://bsaver.io/COCO-slides

LTI

Discrete-time

state space

representation

Markov decision process

LQR

- State-space representation

- Optimal control problems

- Markovianity and Bellman principle

- Concept of value function

- Closed-form solution of linear-quadratic problems

Model Predictive Control

- Receding horizon principle

- Online computation and resulting static feedback

- Linear quadratic case + constraints

- Closed-loop stability (Lyapunov)

- Steady-state selection, disturbance rejection

Economic MPC

- Economic cost of trajectories

- Average performance guarantees

Robust MPC

- Robust satisfaction of constraints

- Closed-form solution for worst-case LQR

- MPC with disturbance feedback

Data-Driven Predictive Control

- State-space identification (Kalman Ho)

- System trajectories as behavioral representation (LTI)

- Data-Driven Predictive Control vs MPC

- Regularization and noise

Markov Decision Processes

- Dynamic programming on MDPs

- Value iteration and policy iteration algorithms

Monte Carlo Learning

- Q function

- Experimental policy evaluation (episodic)

- Approximations: linear approximants

Reinforcement Learning

- Stochastic approximations and stochastic gradient

- TD-learning / SARSA

- Q learning

- Policy gradient

Competencies,

not content

Taxonomy of competences

Insight: Design course around competencies, not content

Learning objectives

Competence-based training

constructive alignment

of learning activities

with learning objectives

Example:

How to recommend a controller?

"Control Engineer Flowchart"

In-class activity
At the end of every module (LQR, MPC, Economic MPC...)
Every student identifies the deciding factors that make them recommend one control strategy compared to another one, and organize them in a flowchart
- "Are constraints an important part of the problem?"
- "Do we have a reliable model for the system?"
- "Do we have a simulator for the system?"
A few students present their flowchart
and defend it in front of the class.
The flowchart grows week after week.

Insight: Train competencies, don't expect them to "appear"

Project

You are a control expert, and you are asked to act as a consultant for an aerospace company. This company wants to hear your opinion on their rocket landing control scheme.

You are provided with

a simulator of the rocket landing dynamics,
a linearized model of the rocket derived by their engineers,
specifications of the control problem (limits on the actuators, constraints on the landing),
a basic working controller (PID-type) that is already implemented in the simulator.

Insight: Back to the promise, treat students as experts

2023 Project

First deliverable

You need to prepare a 5-slide presentation for the company's Chief Technology Officer to explain what type of controller you would recommend and show how it outperforms their current controller in an important failure scenario.

Current state: Show how the current basic controller performs in the standard operating condition
Failure mode: Motivate the need for a better controller. Find a compelling failure scenario that is plausible but that their controller cannot handle well.
Your recommendation: Explain what type of controller you would recommend, providing the three most important reasons.
Demonstration: Illustrate how your controller outperforms their current controller in the failure scenario.
Deployment plan: Plan the necessary steps needed in order to deploy the controller that you proposed.

Constructive alignment

Second deliverable

You also need to prepare a Jupyter notebook for the company's technical team so that they can understand what you are proposing.

How the failure scenario has been modeled in the simulation.
What parameters are available to define the failure scenario.
How to implement the proposed controller.
What parameters need to be tuned to design the controller.
How do you recommend tuning it.

+ public repos

Insight: Plenty of open tools available

Constructive alignment

You are a control expert, and you are asked to act as a consultant for InsulinCo, a company that provides artificial pancreas (AP) care to patients with diabetes. Their AP uses an insulin pump to inject insulin in response to glucose measurements and meal predictions. InsulinCo would like to know if an advanced control method can improve the performance of their AP.