Teaching Modern Control
with Modern Tools

 

Insights from ETH Zurich's Computational Control Course

Saverio Bolognani

24 October 2024

IfA - Automatic Control Laboratory

3 faculty members (Lygeros, Dörfler, Smith) +1

a few Senior Scientists

15 Postdocs, 30+ PhD students

Computational Control

  • First taught: Spring 2023
  • Approx 70 students for credits, 100+ attending
  • Last-year B.Sc. students + first-year M.Sc. students
    • Electrical Engineering
    • Mechanical Engineering
    • Robotics, Syst. & Control
    • Energy Science and Technology
    • Computer Science
    • Biomedical Eng., Quantum Eng., ...
    • Space next year?

What goes into the

"second control course"?

Process automation, concept of control. Modelling of dynamical systems - examples, state space description, linearisation, analytical/numerical solution. Laplace transform, system response for first and second order systems - effect of additional poles and zeros. Closed-loop control - idea of feedback. PID control, Ziegler - Nichols tuning. Stability, Routh-Hurwitz criterion, root locus, frequency response, Bode diagram, Bode gain/phase relationship, controller design via "loop shaping", Nyquist criterion. Feedforward compensation, cascade control. Multivariable systems (transfer matrix, state space representation), multi-loop control, problem of coupling, Relative Gain Array, decoupling, sensitivity to model uncertainty. State space representation (modal description, controllability, control canonical form, observer canonical form), state feedback, pole placement - choice of poles. Observer, observability, duality, separation principle. LQ Regulator, optimal state estimation.

Process automation, concept of control. Modelling of dynamical systems - examples, state space description, linearisation, analytical/numerical solution. Laplace transform, system response for first and second order systems - effect of additional poles and zeros. Closed-loop control - idea of feedback. PID control, Ziegler - Nichols tuning. Stability, Routh-Hurwitz criterion, root locus, frequency response, Bode diagram, Bode gain/phase relationship, controller design via "loop shaping", Nyquist criterion. Feedforward compensation, cascade control. Multivariable systems (transfer matrix, state space representation), multi-loop control, problem of coupling, Relative Gain Array, decoupling, sensitivity to model uncertainty. State space representation (modal description, controllability, control canonical form, observer canonical form), state feedback, pole placement - choice of poles. Observer, observability, duality, separation principle. LQ Regulator, optimal state estimation.

What control design tasks can you already solve?

What is blocking you from solving more complicated ones?

What goes into the

"second control course"?

Group activity (first day of class)

For each design problem:

  • Do you think you have the necessary competences?
  • Would you take up the challenge?
  • If yes, what methods would you use?
  • If no, what’s blocking you?

Insight: Bring props.

Show students what a controller is.

Systems in which the controller is an embedded computer that can sense and actuate a physical plant.

Insight: Tell the students what kind of experts they will become

Suitable for systems where model uncertainty, stringent constraints, and complex dynamics call for advanced control solutions.

Course Content

(13 weeks)

  • Dynamic Programming and LQR

  • Model Predictive Control

  • Economic MPC

  • Robust MPC

  • Data-Driven Predictive Control

  • Markov Decision Processes

  • Monte Carlo Learning (episodic)

  • Reinforcement Learning (online)

LTI

Discrete-time

state space

representation

Markov decision process

LQR

- State-space representation

- Optimal control problems

- Markovianity and Bellman principle

- Concept of value function

- Closed-form solution of linear-quadratic problems

Model Predictive Control

- Receding horizon principle

- Online computation and resulting static feedback

- Linear quadratic case + constraints

- Closed-loop stability (Lyapunov)

- Steady-state selection, disturbance rejection

Economic MPC

- Economic cost of trajectories

- Average performance guarantees

Robust MPC

- Robust satisfaction of constraints

- Closed-form solution for worst-case LQR

- MPC with disturbance feedback

Data-Driven Predictive Control

- State-space identification (Kalman Ho)

- System trajectories as behavioral representation (LTI)

- Data-Driven Predictive Control vs MPC

- Regularization and noise

Markov Decision Processes

- Dynamic programming on MDPs

- Value iteration and policy iteration algorithms

Monte Carlo Learning

- Q function

- Experimental policy evaluation (episodic)

- Approximations: linear approximants

Reinforcement Learning

- Stochastic approximations and stochastic gradient

- TD-learning / SARSA

- Q learning

- Policy gradient

Competencies,

not content

Taxonomy of competences

Insight: Design course around competencies, not content

Learning objectives

Competence-based training

constructive alignment

of learning activities

with learning objectives

Example:

How to recommend a controller?

"Control Engineer Flowchart"

  • In-class activity
  • At the end of every module (LQR, MPC, Economic MPC...)
  • Every student identifies the deciding factors that make them recommend one control strategy compared to another one, and organize them in a flowchart
    • "Are constraints an important part of the problem?"
    • "Do we have a reliable model for the system?"
    • "Do we have a simulator for the system?"
  • A few students present their flowchart
    and defend it in front of the class.
  • The flowchart grows week after week.

Insight: Train competencies, don't expect them to "appear"

Project

You are a control expert, and you are asked to act as a consultant for an aerospace company. This company wants to hear your opinion on their rocket landing control scheme.

You are provided with

  • a simulator of the rocket landing dynamics,
  • a linearized model of the rocket derived by their engineers,
  • specifications of the control problem (limits on the actuators, constraints on the landing),
  • a basic working controller (PID-type) that is already implemented in the simulator.

Insight: Back to the promise, treat students as experts

2023 Project

First deliverable

You need to prepare a 5-slide presentation for the company's Chief Technology Officer to explain what type of controller you would recommend and show how it outperforms their current controller in an important failure scenario.

  1. Current state: Show how the current basic controller performs in the standard operating condition
  2. Failure mode: Motivate the need for a better controller. Find a compelling failure scenario that is plausible but that their controller cannot handle well.
  3. Your recommendation: Explain what type of controller you would recommend, providing the three most important reasons.
  4. Demonstration: Illustrate how your controller outperforms their current controller in the failure scenario.
  5. Deployment plan: Plan the necessary steps needed in order to deploy the controller that you proposed.

Constructive alignment

Second deliverable

You also need to prepare a Jupyter notebook for the company's technical team so that they can understand what you are proposing.

  • How the failure scenario has been modeled in the simulation.
  • What parameters are available to define the failure scenario.
  • How to implement the proposed controller.
  • What parameters need to be tuned to design the controller.
  • How do you recommend tuning it.

+ public repos

Insight: Plenty of open tools available

Constructive alignment

You are a control expert, and you are asked to act as a consultant for InsulinCo, a company that provides artificial pancreas (AP) care to patients with diabetes. Their AP uses an insulin pump to inject insulin in response to glucose measurements and meal predictions. InsulinCo would like to know if an advanced control method can improve the performance of their AP.

2024 Project

InsulinCo has provided you with

  • the ReplayBg simulator, which simulates the nonlinear dynamics of a human glucose regulation mechanism.
  • the time series data for a single sample patient.

Insight: Provide modern control design tools

Behçet Açikmeşe

Professors at U Washington

Past member of NASA's Mars Science Laboratory G&C team

Insight: Treat students with excellent guests and resources

Simone Del Favero

Professors at U Padova

Where the first FDA-approved pancreas simulator was developed

Exam questions

AI tools

ETH Zurich FAQ on ChatGPT

Insight: discuss the role of AI tools for an expert

Reception by the students

2023 Golden Owl
teaching award


awarded by the ETH Zurich student association VSETH

Sophie Hall

Nicolas Lanzetti

Alberto Padoan

Keith Moffat

Public course material

https://bsaver.io/COCO

+ open github code

Insight: Bring props. Show students what a controller is.

Insight: Tell the students what kind of experts they will become

Insight: Design course around competencies, not content

Insight: Train competencies, don't expect them to "appear"

Insight: Back to the promise, treat students as experts

Insight: Provide modern control design tools

Insight: Treat students with excellent guests and resources

Insight: discuss the role of AI tools for an expert

Insight: Plenty of open tools available

Insight: discuss the role of AI tools for an expert

Constructive alignment between learning activities, assessment techniques, and learning objectives (competences).

Teaching Modern Control with Modern Tools

By Saverio Bolognani

Teaching Modern Control with Modern Tools

  • 79