Adrien Durier PRO
Enseignant-Chercheur en informatique @ Université Paris-Saclay
Concurrency Theory for formal verification of
Cyber-Physical Systems
Adrien Durier
LMF, séminaire au vert
3 juin 2025
Work in collaboration with
Paolo Crisafulli, Benjamin Puyobro, Safouan Taha, Burkhart Wolff
Context:
Autonomous vehicles
Collision-avoiding driving strategies
and driving control
RSS: Responsibility-Sensitive Safety
On a formal model of safe and scalable self-driving cars
S. Shalev-Shwartz, S. Shammah, A. Shashua
Mobileye, 2017:
RSS: Responsibility-Sensitive Safety
When far enough:
Cars may pick whatever speed they choose
RSS: Responsibility-Sensitive Safety
RSS 'Safe' Distance
RSS: Responsibility-Sensitive Safety
RSS 'Safe' Distance
Safety Violation: Rear has to start braking
as soon as it can react
(according to its reaction time)
RSS: Responsibility-Sensitive Safety
A car only cares about its own 'safety' violation
RSS: Responsibility-Sensitive Safety
RSS: Responsibility-Sensitive Safety
RSS: Responsibility-Sensitive Safety
Possible safety violation
(from blue point of view)
RSS: Responsibility-Sensitive Safety
Green goes 'straight':
its responsibility is not engaged
RSS: Responsibility-Sensitive Safety
Mobileye claim:
when car can change lanes,
it's still fine.
RSS: Responsibility-Sensitive Safety
2 Dimensions: Parking
RSS: Responsibility-Sensitive Safety
RSS: Responsibility-Sensitive Safety
Possible 'lateral' safety violation
But not 'longitudinal'
Possible 'lateral' safety violation
But not 'longitudinal'
Mobileye Claim:
Everything is fine ("no need to brake")
As long as safe distance is maintained in one dimension
This is fine.
Goals
- Formally verify Mobileye claims
- Develop a general framework for formal verification of Cyber-Physical Systems
(Almost) All that follows is implemented and verified in Isabelle/HOL
The Framework
Car 1 (Front)
Car 2 (Rear)
Position (x)
Time
Discrete + Continuous
Non-Deterministic Time-Triggered Model
Initial 'Scene'
For each agent:
- Initial Position
- Initial Speed
- Initial Acceleration
- ...
Non-Deterministic Time-Triggered Model
Initial 'Scene'
...
...
...
...
...
...
...
...
...
...
Agree on
\sigma
0,1s
0,01s
0,1s
Pick
\delta t\in\R
- Compute new positions/speeds based on previous scene (deterministic)
- Every agent picks (non-deterministic) a new speed
- Everybody has to agree
The whole tree is called a scenario
Car 1 (Front)
Car 2 (Rear)
Position (x)
Time
Discrete + Continuous
Please forgive the segments instead of hyperboles...
Car 1 (Front)
Car 2 (Rear)
Position (x)
Time
Two cars on a lane...
Please forgive the segments instead of hyperboles...
\delta t_1
\delta t_2
\delta t_3
Car 1 (Front)
Car 2 (Rear)
Position (x)
Time
Two cars on a lane...
Please forgive the segments instead of hyperboles...
\delta t_1
\delta t_2
\delta t_3
Safety distance
violation
Rear car starts braking
Car 1 (Front)
Car 2 (Rear)
Two cars on a lane...
\mathbf{Car}_1 (\sigma_i) = e_{time}\boldsymbol ? (\delta t) \rightarrow e_{{scene}} \boldsymbol ? (\sigma) \rightarrow \mathbf{Car}_1 (\sigma)
Why CSP?
- Uncountable non-determinism
- Well suited for communications between agents
- Simulation generation (from scenario)
- Can be plugged to FDR for model-checking
- Already implemented in Isabelle/HOL
\mathbf{Car}_1 (\sigma_i) = e_{time}\boldsymbol ? (\delta t) \rightarrow e_{{scene}} \boldsymbol ? (\sigma) \rightarrow \mathbf{Car}_1 (\sigma)
e\boldsymbol ? (x)=\square \mathbf{event}\in \{e(x)|x \in \dots\}
e\boldsymbol ! (x)=\sqcap \mathbf{event}\in \{e(x)|x \in \dots\}
\boldsymbol\square :
Non-deterministic choice (external)
\boldsymbol\sqcap :
Non-deterministic choice (internal)
\boldsymbol{\rightarrow} :
'Sequential composition'
\boldsymbol{\shortmid\shortmid} :
Parallel Composition (with forced synchronization)
CSP Crash Course
Menu
Pick a meal
\mathbf{Car}_1 (\sigma_i) = e_{time}\boldsymbol ? (\delta t) \rightarrow e_{{scene}} \boldsymbol ? (\sigma) \rightarrow \mathbf{Car}_1 (\sigma)
CSP Crash Course
Not any
\sigma
\sigma\in \mathcal M (\sigma_i,\Delta t)
\text{Where }\mathcal M\text{ is a `\textbf{Motion}'}
Contains both standard kinematics, driving strategy, and even possibly scheduling
\mathcal M = \mathbf{drive}\circ \mathbf{std\_kinematics} \circ\dots
\mathbf{Car}_1 (\sigma_i) = e_{time}\boldsymbol ? (\delta t) \rightarrow e_{{scene}} \boldsymbol ? (\sigma) \rightarrow \mathbf{Car}_1 (\sigma)
Motions
\sigma\in \mathcal M (\sigma_i,\Delta t)
\mathcal M = \mathbf{drive}\circ \mathbf{std\_kinematics} \circ\dots
Algebra of Motions:
Some operations preserve Invariants & Safety Properties...
\mathcal M_1\circ \mathcal M_2,~\mathcal M_1\cup\mathcal M_2,~\mathcal M_1\cap\mathcal M_2,~\mathcal M_1\otimes\mathcal M_2\dots
\mathbf{Car}_1 (\sigma_i) = e_{time}\boldsymbol ? (\delta t) \rightarrow e_{{scene}} \boldsymbol ? (\sigma|\sigma\in \mathcal M (\sigma_i,\Delta t)) \rightarrow \mathbf{Car}_1 (\sigma)
Motions
Remember the Menu!
Car 1 (Front)
Car 2 (Rear)
Two cars on a lane...
\mathbf{Car}_1 (\sigma_i) = e_{time}\boldsymbol ? (\delta t) \rightarrow e_{{scene}} \boldsymbol ? (\sigma) \rightarrow \mathbf{Car}_1 (\sigma)
\mathbf{Car}_2 (\sigma_i) = e_{time}\boldsymbol ? (\delta t) \rightarrow e_{{scene}} \boldsymbol ? (\sigma) \rightarrow \mathbf{Car}_2 (\sigma)
\mathbf{Demon} (\Delta t)= e_{time}\boldsymbol !(\delta t | \delta t\in ]0;\Delta t[)\rightarrow e_{scene}\boldsymbol ?(\sigma)\rightarrow \mathbf{Demon} (\Delta t)
Maxwell's 'Demon'
Scenarios
\mathbf{Car}_1 (\sigma_i)
\mathbf{Car}_2 (\sigma_i)
\mathbf{Demon} (\Delta t)
\mathbf{scenario}_{\mathcal M,\Delta t}(\sigma_i) =
\boldsymbol{\shortmid\shortmid}
\boldsymbol{\shortmid\shortmid}
Everybody has to agree on timings & scenes
Scenarios
\mathbf{Car}_1 (\sigma_i)
\mathbf{Car}_2 (\sigma_i)
\mathbf{Demon} (\Delta t)
\mathbf{scenario}_{\mathcal M,\Delta t}(\sigma_i) =
\boldsymbol{\shortmid\shortmid}
\boldsymbol{\shortmid\shortmid}
\delta t
\delta t
\sigma
\sigma
Car 1 (Front)
Car 2 (Rear)
Position (x)
Time
Two cars on a lane...
\delta t_1
\delta t_2
\delta t_3
Car 1 (Front)
Car 2 (Rear)
Position (x)
Time
Two cars on a lane...
The demon ability to 'oversample'
makes it impossible to miss a collision!
Polychrony
Problems
Car 1 (Front)
Car 2 (Rear)
Position (x)
Time
Every discretized time is a possible
point of decision
\delta t_1
\delta t_2
\delta t_3
Safety distance
violation
Rear car starts braking
Car 1 (Front)
Car 2 (Rear)
Position (x)
Time
\delta t_1
\delta t_2'
\delta t_3
Rear car starts braking
\delta t_2''
New Braking Point
Every discretized time is a possible
point of decision
Car 1 (Front)
Car 2 (Rear)
Position (x)
Time
\delta t_1
\delta t_2'
\delta t_3
\delta t_2''
Oversampling may make it impossible to recover previous curve !!!!
Every discretized time is a possible
point of decision
Car 1 (Front)
Car 2 (Rear)
Position (x)
Time
\delta t_1
\delta t_2'
\delta t_3
Rear car starts braking
\delta t_2''
Might not be able to take a decision (yet)
Every discretized time is a possible
point of decision
Car 1 (Front)
Car 2 (Rear)
Position (x)
Time
Solution: Scheduling
(Cycle Times, Reaction Times)
Cycle time: car may only react at these points (exactly)
Car 1 (Front)
Car 2 (Rear)
Position (x)
Time
Solution: Scheduling
(Cycle Times, Reaction Times)
Bad
Car 1 (Front)
Car 2 (Rear)
Position (x)
Time
Solution: Scheduling
(Cycle Times, Reaction Times)
Can be applied directly in the 'Motions'
(by 'killing' bad branches)
Non-Deterministic Time-Triggered Model
Initial 'Scene'
...
...
...
...
...
...
...
...
...
...
0,1s
0,01s
0,1s
Non-Deterministic Time-Triggered Model
Initial 'Scene'
...
...
...
...
...
...
...
...
...
...
0,1s
0,01s
0,1s
Car 1 (Front)
Car 2 (Rear)
Position (x)
Time
Solution: Scheduling
(Cycle Times, Reaction Times)
Reaction time: car have to be able to react before it ends
Car 1 (Front)
Car 2 (Rear)
Position (x)
Time
Solution: Scheduling
(Cycle Times, Reaction Times)
Reaction time: car have to be able to react before it ends
Position (x)
Time
Killing branches: effect on Bouncing Ball
Safety Properties
Safety Properties
Safety Property:
- Something bad never happens
Liveness Properties:
- Something good eventually happens
\mathbf{safe}_P (\sigma_i) = e_{time}\boldsymbol ? (\delta t) \rightarrow e_{{scene}} \boldsymbol ? (\sigma | P(\sigma)) \rightarrow \mathbf{safe}_P (\sigma)
Safety Properties
\mathbf{safe}_P = e_{time}{\boldsymbol ?} (\delta t) \rightarrow e_{{scene}} \boldsymbol ? (\sigma | P(\sigma)) \rightarrow \mathbf{safe}_P
\mathbf{safe}_{\mathtt{no\_collision}}:
Safety Properties
\mathbf{safe}_P = e_{time}{\boldsymbol ?} (\delta t) \rightarrow e_{{scene}} \boldsymbol ? (\sigma | P(\sigma)) \rightarrow \mathbf{safe}_P
\mathbf{safe}_{\mathtt{no\_collision}}:
Safety Properties
\mathbf{safe}_P = e_{time}{\boldsymbol ?} (\delta t) \rightarrow e_{{scene}} \boldsymbol ? (\sigma | P(\sigma)) \rightarrow \mathbf{safe}_P
No collision!
\mathbf{safe}_{\mathtt{no\_collision}}:
Safety Properties
\mathbf{safe}_P = e_{time}{\boldsymbol ?} (\delta t) \rightarrow e_{{scene}} \boldsymbol ? (\sigma | P(\sigma)) \rightarrow \mathbf{safe}_P
Monotony of safe processes:
If
Then
\forall \sigma. P(\sigma)\Rightarrow P'(\sigma)
\mathbf{safe}_{P'} \leq \mathbf{safe}_P
Safety Properties
\mathbf{safe}_P = e_{time}{\boldsymbol ?} (\delta t) \rightarrow e_{{scene}} \boldsymbol ? (\sigma | P(\sigma)) \rightarrow \mathbf{safe}_P
Safety of a scenario:
A scenario is defined as safe relative to P if:
\mathbf{safe}_P \leq \mathbf{scenario}_{\mathcal M,\Delta t} (\sigma_i)
In other words, every scene of every trace of the scenario verifies P.
\mathbf{scenario}_{\mathcal M,\Delta t} (\sigma_i)= \mathbf{Demon}(\Delta t) ~||~ \mathbf{Agent}_1 (\mathcal M, \sigma_i)~||~ \dots
Safety Properties
\mathbf{safe}_P = e_{time}{\boldsymbol ?} (\delta t) \rightarrow e_{{scene}} \boldsymbol ? (\sigma | P(\sigma)) \rightarrow \mathbf{safe}_P
\mathbf{scenario}_{\mathcal M,\Delta t} (\sigma_i)= \mathbf{Demon}(\Delta t) ~||~ \mathbf{Agent}_1 (\mathcal M, \sigma_i)~||~ \dots
Inductive proof by invariant
If, for some Pinv
Then scenario is P-safe
P_{inv}(\sigma_i)
\forall\sigma.P_{inv}(\sigma)\Rightarrow P(\sigma)
\forall\sigma.P_{inv}(\sigma)\Rightarrow P_{inv}(\sigma')\text{ for }\sigma'\in\mathcal M(\sigma,\delta t)
Example: RSS (2 cars, longi)
\mathbf{scenario}_{\mathcal M,\Delta t} (\sigma_i)= \mathbf{Demon}(\Delta t) ~||~ \mathbf{Car}_1 (\mathcal M, \sigma_i)~||~ \mathbf{Car}_2 (\mathcal M, \sigma_i)
P_{inv}(\sigma_i)
\forall\sigma.P_{inv}(\sigma)\Rightarrow P(\sigma)
\forall\sigma.P_{inv}(\sigma)\Rightarrow P_{inv}(\sigma')\text{ for }\sigma'\in\mathcal M(\sigma,\delta t)
Inductive proof by invariant
P_{inv}= |\mathbf{Car}_2.\mathtt{pos} - \mathbf{Car}_1.\mathtt{pos} | > d_{min}
d_{min}=d_{brake_2} - d_{brake_1}
RSS 'Safe' Distance
\mathbf{d}_{min}
Invariant Proof:
If you start braking at RSS safe distance,
then you never break minimum distance
\mathbf{d}_{min}
Liveness Properties
'Something good eventually happens'
Liveness Properties
'Something good eventually happens'
Something good?
- "Destination is reached"
- "The car advances X km on the road"
- ...
"Eventually":
- Usually, trace-related (at some point in the trace...)
- But for us... Time-related
Liveness Properties
'Something good eventually happens'
Zenon's Paradox:
For something good to eventually happen, time needs to progress
(on at least some branches)
- Killing branches can make time progress impossible !
Theorem:
If M is a non-empty Motion, then time can progress in
\mathbf{scenario}_{\mathcal M,\Delta t}(\sigma_i)
Concurrency Theory for formal verification of Cyber-Physical Systems
By Adrien Durier
Concurrency Theory for formal verification of Cyber-Physical Systems
Concurrency Theory for formal verification of Cyber-Physical Systems - LMF - Séminaire au vert 2025
