Some applications of stochastic control in epidemiology
A framework based on stochastic optimal control and dynamic programming.
Gabriel Salcedo-Varela, David González-Sánchez,
Saúl Díaz-Infante Velasco,
saul.diazinfante@unison.mx,
March 29, 2024
MathBio seminar, ASU
Stochastic control and epidemiology.
What’s the buzz?
x˙(t)=F(x(t),θ,a(t)),x(0)=x0,x0, x(t)∈Γ⊂Rd,θ:=(θ1,⋯,θm),a(t):=(aθ1(t),⋯,aθi(t)),1≤j≤i,t∈[0,T].
\begin{aligned}
& \dot{x}(t) =
F(x(t), \theta, a(t)), \quad x(0) = x_0,
\\
& x_0, \ x(t) \in \Gamma \subset \mathbb{R}^d,
\quad \theta:= (\theta_1,\cdots, \theta_m),
\\
& a(t):= (a_{\theta_1}(t), \cdots, a_{\theta_i}(t)),
\\
&1\leq j\leq i, \quad t \in [0, T].
\end{aligned}
x˙(t)=F(x(t),θ),x(0)=x0,t∈[0,T],x0, x(t)∈Γ⊂Rdθ:=(θ1,⋯,θm),F:Γ→Γ.
\begin{aligned}
& \dot{x}(t) =
F(x(t), \theta), \quad x(0) = x_0, \quad t\in[0, T],
\\
& x_0, \ x(t) \in \Gamma \subset\mathbb{R}^d
\quad \theta:= (\theta_1,\cdots, \theta_m),
\\
&F: \Gamma \to \Gamma.
\end{aligned}
c(x(t),a(t)):=∫0tc0(s,x(s))ds+∫0tc1(a(s))ds.
c(x(t), a(t)):=
\int_0^t c_{0}(s, x(s)) ds + \int_0^t c_1( a(s))ds.
OCMT:s.t.a⋅∈Aminc(x(⋅),a(⋅))=∫0Tc0(s,x(s))ds+∫0Tc1(s,a(s))dsx(t)˙=F(x(t),θ,a(t)),x(0)=x0.
\begin{aligned}
\text{OCM}^T:&
\\
&
\min_{a_{\cdot} \in \mathcal{A}}
c(x(\cdot), a(\cdot)) =
\int_0 ^ T
c_{0}(s, x(s))
ds
+
\int_0 ^ T
c_1(s, a(s))ds
\\
\text{s.t.}&
\\
&
\dot{x(t)} = F(x(t),
\theta, a(t)),
\qquad x(0)= x_0.
\end{aligned}
Spread Dynamics
Controller
x(t)
x(t)
at∈A
a_t \in \mathcal{A}
Data
- Modelling
Control
OCMT:s.t.a⋅∈Aminc(x(⋅),a(⋅))=∫0Tc0(s,x(s))ds+∫0Tc1(s,a(s))dsx(t)˙=F(x(t),θ,a(t)),x(0)=x0.
\begin{aligned}
\text{OCM}^T:&
\\
&
\min_{a_{\cdot} \in \mathcal{A}}
c(x(\cdot), a(\cdot)) =
\int_0 ^ T
c_{0}(s, x(s))
ds
+
\int_0 ^ T
c_1(s, a(s))ds
\\
\text{s.t.}&
\\
&
\dot{x(t)} = F(x(t),
\theta, a(t)),
\qquad x(0)= x_0.
\end{aligned}
Spread Dynamics
Controller
x(t)
x(t)
at∈A
a_t \in \mathcal{A}
- Bayesian Methods MCMC
- Expectation Maximization
- Maximum likelihood
- Deterministic: ODE, FD, FDD, PDE
- Stochastic: SDEs
- Pontryagin
- Dynamic Programming
Data
- Modelling
Control
To fix ideas:
- TYLCV Disease
- is spread by the insect whitefly (Bemisia tabaci)
-
How to Manage TYLCV?
-
Replanting infected plants suffers random fluctuations due to is leading identification because a yellow plant is not necessarily infected.
- The protocol to manage infected plants suggests replanting neighbors. Then naturally, a farmer could randomly replant a healthy plant instead of a latent one.
- Thus strategies like fumigation with insecticide suffer random fluctuationsin their efficiency.
Controls
- Replanting
- Fumigation
Sp˙Lp˙Ip˙Sv˙Iv˙=−βpSpIv+r1Lp+r2Ip,=βpSpIv−(b+r1)Lp,=bLp−r2Ip,=−βvSvIp−(γ+γf)Sv+(1−θ)μ,=βvSvIp−(γ+γf)Iv+θμ.
\begin{aligned}
\dot{S_p} &=
-\beta_p S_p I_v +\textcolor{blue}{r_1}L_p + \textcolor{blue}{r_2} I_p,\\
\dot{L_p} &=
\beta_p S_p I_v -(b +\textcolor{blue}{r_1}) L_p,\\
\dot{I_p} &=
b L_p - \textcolor{blue}{r_2} I_p,\\
\dot{S_v} &=
-\beta_v S_v I_p - (\gamma +\textcolor{orange}{ \gamma_f}) S_v +(1-\theta)\mu,\\
\dot{I_v} &=
\beta_v S_v I_p -(\gamma+\textcolor{orange}{\gamma_f}) I_v +\theta\mu.
\end{aligned}
Subject toSp˙Lp˙Ip˙Sv˙Iv˙uˉ(⋅)∈Ux0[t0,T]minJ(u)=−βpSpIv+(r1+u1)Lp+(r2+u2)Ip,=βpSpIv−(b+(r1+u1))Lp,=bLp−(r2+u2)Ip,=−βvSvIp−(γ+(γf+u3))Sv+(1−θ)μ,=βvSvIp−(γ+(γf+u3))Iv+θμ,Sp(0)=Sp0,Lp(0)=Lp0,Ip(0)=Ip0,Sv(0)=Sv0,Iv(0)=Iv0,ui(t)∈[0,uimax].
\begin{aligned}
& \min_{\bar{u}(\cdot)\in \mathcal{U}_{x_0}[t_0,T]}J(u)
\\
\text{Subject to} &
\\
\dot{S_p} &=
- \beta_p S_p I_v
+ \textcolor{blue}{(r_1 +u_1)} L_p
+ \textcolor{blue}{(r_2 + u_2)} I_p,
\\
\dot{L_p} &=
\beta_p S_p I_v -(b +
\textcolor{blue}{(r_1 + u_1))} L_p,
\\
\dot{I_p} &=
b L_p -
\textcolor{blue}{(r_2 + u_2)} I_p,
\\
\dot{S_v} &=
- \beta_v S_v I_p
- (\gamma+ \textcolor{orange}{(\gamma_f + u_3)} ) S_v
+ (1-\theta)\mu,
\\
\dot{I_v} &=
\beta_v S_v I_p
- (\gamma + \textcolor{orange}{(\gamma_f + u_3)}) I_v
+ \theta\mu,
\\
&S_p(0) = S_{p_0}, L_p(0) = L_{p_0}, I_p(0) = I_{p_0},
\\
&S_v(0) = S_{v_0}, I_v(0) = I_{v_0},
u_i(t) \in [0, u_i ^ {max}].
\end{aligned}
J(u)=∫0T[A1Ip(t)+A2Lp(t)+A3Iv(t)+i=1∑3ciui(t)2]dt,
\begin{aligned}
J(u) &= \int_{0}^T \Big[A_1 I_p(t) + A_2 L_p(t) + A_3 I_v(t)
+ \sum^3_{i=1}c_i u_i(t)^2 \Big] dt,
\end{aligned}
Performing parameter calibration using MCMC (Markov Chain Monte Carlo).
Simulation: Counterfactual vs Controlled.
OCMT:s.t.a⋅∈Aminc(x(⋅),a(⋅))=∫0Tc0(s,x(s))ds+∫0Tc1(s,a(s))dsx(t)˙=F(x(t),θ,a(t)),x(0)=x0.
\begin{aligned}
\text{OCM}^T:&
\\
&
\min_{a_{\cdot} \in \mathcal{A}}
c(x(\cdot), a(\cdot)) =
\int_0 ^ T
c_{0}(s, x(s))
ds
+
\int_0 ^ T
c_1(s, a(s))ds
\\
\text{s.t.}&
\\
&
\dot{x(t)} = F(x(t),
\theta, a(t)),
\qquad x(0)= x_0.
\end{aligned}
x(t)
x(t)
Spread Dynamics
Controller
at∈A
a_t \in \mathcal{A}
ξt
\xi_t
sto-OCMT:s.t.a⋅∈Aminc(x(⋅),a(⋅))=Eπ[∫0Tc0(s,x(s))dWs+∫0Tc1(s,a(s))ds]x(t)˙=F(x(t),θ,a(t),ξt),x(0)=x0.
\begin{aligned}
\text{sto-OCM}^T:&
\\
&
\min_{a_{\cdot} \in \mathcal{A}}
c(x(\cdot), a(\cdot)) =
\textcolor{orange}{\mathbb{E}^{\pi}}
\Big[
\int_0 ^ T
c_{0}(s, x(s))
dW_s
+
\int_0 ^ T
c_1(s, a(s))ds
\Big]
\\
\text{s.t.}&
\\
&
\dot{x(t)} =
F(
x(t),
\theta,
a(t),
\textcolor{orange}{\xi_t}
),
\qquad x(0)= x_0.
\end{aligned}
SDEs, CTMC, Sto-Per
Data
Sto-Modelling
Control
Stochastic extension
dtdx(t)=f(t,x(t))α⇝dx(t)=f(t,x(t))dt+g(t,x(t))dB(t)⇝α+P(x(t))dtdB(t)
\begin{aligned}
\frac{d}{dt}x(t) = f(t,x(t)) &
\rightsquigarrow dx(t) = f(t,x(t))dt + g(t,x(t))dB(t)
\\
\alpha &
\rightsquigarrow \alpha + P(x(t))\frac{dB(t)}{dt}
\end{aligned}
dtdSpdtdLpdtdIpdtdSvdtdIv=−βpSpNvIv+r1Lp+r2Ip,=βpSpNvIv−(b+r1)Lp,=bLp−r2Ip,=−βvSvNpIp−(γ+γf)Sv+(1−θ)μ,=βvSvNpIp−(γ+γf)Iv+θμ,
\begin{aligned}
\frac{dS_p}{dt} &=
-\beta_p S_p \frac{I_v}{N_v} + \textcolor{blue}{r_1} L_p
+ \textcolor{blue}{r_2} I_p,\\
\frac{dL_p}{dt} &=
\beta_p S_p \frac{I_v}{N_v}
- (b + \textcolor{blue}{r_1}) L_p,\\
\frac{dI_p}{dt} &=
b L_p - \textcolor{blue}{r_2} I_p,\\
\frac{dS_v}{dt} &=
-\beta_v S_v \frac{I_p}{N_p}
- (\gamma + \textcolor{orange}{\gamma_f}) S_v + (1-\theta)\mu,\\
\frac{dI_v}{dt} &=
\beta_v S_v \frac{I_p}{N_p}
- (\gamma + \textcolor{orange}{\gamma_f}) I_v +\theta\mu,\\
\end{aligned}
r1dt⇝r1dt+σLNpSpdBp(t),r2dt⇝r2dt+σINpSpdBp(t),γfdt⇝γfdt+σvdBv(t).
\begin{aligned}
r_1 dt \rightsquigarrow r_1 dt + \sigma_L\frac{S_p}{N_p}dB_p(t),
\\
r_2 dt \rightsquigarrow r_2 dt + \sigma_I\frac{S_p}{N_p} dB_p(t),
\\
\gamma_f dt \rightsquigarrow \gamma_f dt + \sigma_v dB_v(t).
\end{aligned}
dSpdLpdIpdSvdIv=(−βpSpNvIv+r1Lp+r2Ip)dt+NpSp(σLLp+σIIp)dBp(t),=(βpSpNvIv−(b+r1)Lp)dt−σLNpSpLpdBp(t),=(bLp−r2Ip)dt−σINpSpIpdBp(t),=(−βvSvNpIp−(γ+γf)Sv+(1−θ)μ)dt−σvSvdBv(t),=(βvSvNpIp−(γ+γf)Iv+θμ)dt−σvIvdBv(t).
\begin{aligned}
d S_p &=
\left(
-\beta_p S_p \frac{I_v}{N_v} + \textcolor{blue}{r_1} L_p
+ \textcolor{blue}{r_2} I_p
\right)dt
+ \textcolor{blue}{\frac{S_p(\sigma_L L_p
+
\sigma_I I_p)}{N_p}} dB_p(t),
\\
dL_p &=
\left(
\beta_p S_p \frac{I_v}{N_v}
- (b + \textcolor{blue}{r_1}) L_p \right) dt
- \textcolor{blue}{\sigma_L \frac{S_p L_p}{N_p}} dB_p(t),
\\
d I_p &=
\left(
b L_p - \textcolor{blue}{r_2} I_p
\right) dt
- \textcolor{blue}{\sigma_I \frac{S_pI_p}{N_p}} dB_p(t),
\\
dS_v &=
\left(
-\beta_v S_v \frac{I_p}{N_p} - (\gamma + \textcolor{orange}{\gamma_f}) S_v
+ (1-\theta) \mu \right)dt
- \textcolor{orange}{\sigma_v S_v} dB_v(t),
\\
d I_v &=
\left(
\beta_v S_v \frac{I_p}{N_p} - (\gamma + \textcolor{orange}{\gamma_f}) I_v
+ \theta \mu
\right) dt
- \textcolor{orange}{\sigma_v I_v} dB_v(t).
\end{aligned}
Enhancing parameter calibration through noise
dx(t)x(0)J(u(⋅))=f(t,x(t),u(t))dt+g(t,x(t),u(t))dW(t),=x0∈Rn,=E{∫0Tc(t,x(t),u(t))dt+h(x(T))}.
\begin{aligned}
dx(t) &= f(t, x(t), \textcolor{orange}{u(t)})dt + g(t, x(t), u(t)) dW(t),\\
x(0) &= x_0 \in \mathbb{R}^n,
\\
\\
J(u(\cdot)) &= \mathbb{E} \left\{ \int^T_0 c(t,x(t),u(t)) dt + h(x(T))\right\}.
\end{aligned}
Suject todx(t)x(0)J(πˉ)=π∈U[0,T]infJ(π)=f(t,x(t),u(t))dt+g(t,x(t),u(t))dW(t),=x0∈Rn,
\begin{aligned}
&J(\bar{\pi}) = \inf_{\pi \in\, \mathcal{U}[0,T]} J(\pi)\\
\text{Suject to}&\\
dx(t) &= f(t, x(t), u(t))dt + g(t, x(t), u(t)) dW(t),\\
x(0) &= x_0 \in \mathbb{R}^n,
\end{aligned}
∀t∈[0,T], x, x^∈Rn, u∈U,∣φ(t,x,u)−φ(t,x^,u)∣≤L∣x−x^∣,
\begin{aligned}
\forall t & \in[0,T],
\ x, \ \hat{x} \in \mathbb{R}^n,
\ u \in U,
\\
&|\varphi(t,x,u) - \varphi(t, \hat{x},u)|\leq L|x-\hat{x}|,
\end{aligned}
f:[0,T]×Rn×Ug:[0,T]×Rn×Uc:[0,T]×Rn×Uh:Rn→R→Rn,→Rn×m,→R,
\begin{aligned}
f:[0,T] \times \mathbb{R} ^ n \times U
&\to \mathbb{R}^n,
\\
g:[0,T] \times\mathbb{R} ^ n \times U
&\to \mathbb{R}^{n\times m},
\\
c:[0,T] \times\mathbb{R} ^ n \times U
&\to \mathbb{R},
\\
h:\mathbb{R}^n
\to \mathbb{R}&
\end{aligned}
(s,y)∈[0,T)V(s,y)V(T,y)×Rn,=u(⋅)∈U[s,T]infJ(s,y;u(⋅)),=h(y),∀y∈Rn,
\begin{aligned}
(s,y) \in [0,T) &\times \mathbb{R}^n,
\\
V(s,y)
&= \inf_{ u(\cdot) \in \mathcal{U} [s,T]}
J(s, y; u(\cdot) ),
\\
V(T,y) &= h(y), \qquad \forall y\in \mathbb{R}^n,
\end{aligned}
−Vt+u∈UsupHG(t,x,u,−Vx,−Vxx)=0,(t,x)∈[0,T)×Rn,Vt=T=h(x),x∈Rn,HG(t,x,u,p,P)=⟨p,f(t,x,u)⟩−c(t,x,u)+21trace(g⊤(t,x,u)Pg(t,x,u))=Hdet(t,u,x,p)+21trace(g⊤(t,x,u)Pg(t,x,u))
\begin{aligned}
& -V_t +
\sup_{u \in U} H_G (t ,x, u, -V_x, -V_{xx})=0,
(t,x) \in [0,T)\times\mathbb{R}^n,
\\
& V \Big|_{t=T} = h(x), \qquad x\in \mathbb{R} ^ n,
\\
& H_G(t,x,u,p,P) = \langle p, f(t,x,u) \rangle - c(t,x,u)
\\
&+\dfrac{1}{2} \mathtt{trace}\Big( g^\top(t,x,u)Pg(t,x,u)\Big)
\\
& = H_{det}(t,u,x,p) +\dfrac{1}{2} \mathtt{trace}\Big( g^\top(t,x,u)Pg(t,x,u)\Big)
\end{aligned}
∀(t,u)∈[0,T]×U∣φ(t,0,u)∣≤L
\begin{aligned}
\forall (t,u) & \in [0,T]\times U
\\
&|\varphi(t,0,u)|\leq L
\end{aligned}
OPC:
Value Function
HJB
dx(t)x(0)J(u(⋅))=f(t,x(t),u(t))dt+g(t,x(t),u(t))dW(t),=x0∈Rn,=E{∫0Tc(t,x(t),u(t))dt+h(x(T))}.
\begin{aligned}
dx(t) &= f(t, x(t), \textcolor{orange}{u(t)})dt + g(t, x(t), u(t)) dW(t),\\
x(0) &= x_0 \in \mathbb{R}^n,
\\
J(u(\cdot)) &= \mathbb{E} \left\{ \int^T_0 c(t,x(t),u(t)) dt + h(x(T))\right\}.
\end{aligned}
J(u)=E∫0T[A1Ip(t)+A2Lp(t)+A3Iv(t)+c1u12(t)+c2u22(t)+c3u32(t)]dt,
\begin{aligned}
J(u) = \mathbb{E}\int_{0}^{T}
& \Bigg[
A_1 I_p(t) + A_2 L_p(t) + A_3 I_v (t)
\\
& + c_1 u ^ 2_1(t) + c_2 u ^ 2_2(t) + c_3 u ^ 2_3(t)
\Bigg] dt,
\end{aligned}
u∈U[0,T]minJ(u)subject to dSp=(−βpSpNvIv+(r1+u1)Lp+(r2+u2)Lp)dt+NpSp(σLLp+σIIp)dBp,dLp=(βpSpNvIv−(b+r1+u1)Lp)dt−σLNpSpLpdBp,dIp=(bLp−(r2+u2)Ip)dt−σINpSpIpdBp,dSv=(−NpβvSvIp−(γ+γf+u3)Sv+(1−θ)μ)dt−σvSvdBv,dIv=(NpβvSvIp−(γ+γf+u3)Iv+θμ)dt−σvSvdBv.
\begin{aligned}
&\qquad\qquad\qquad\qquad\min_{u\in \mathcal{U}[0,T]} J(u) \\
&\qquad \text{subject to } \\
&dS_p
=
\left(
-\beta_pS_p\frac{I_v}{N_v}+(r_1 +u_1) L_p + (r_2 + u_2)L_p
\right)dt
\\
&+
\frac{S_p}{N_p}\left(
\sigma_L L_p+\sigma_I I_p \right)dB_p,\\
&dL_p
=
\left(
\beta_pS_p\frac{I_v}{N_v}-(b+r_1 +u_1)L_p
\right)dt
-\sigma_L\frac{S_p}{N_p}L_pdB_p,
\\
&dI_p
= \left(
b L_p - (r_2 + u_2) I_p
\right)dt - \sigma_I\frac{S_p}{N_p}I_p dB_p,
\\
&dS_v
=
\left( -\frac{\beta_v}{N_p}S_vI_p
-(\gamma +\gamma_f+ u_3) S_v + (1-\theta)\mu\right)dt
-\sigma_v S_vdB_v,
\\
&dI_v
=
\left( \frac{\beta_v}{N_p}S_vI_p - (\gamma +\gamma_f+u_3) I_v +\theta \mu \right) dt
- \sigma_v S_v dB_v.
\end{aligned}
V(s,x)=u(⋅)∈U[s,T]infE{∫sT[A1Ip(t)+A2Lp(t)+A3Iv(t)+c1u12(t)+c2u22(t)+c3u32(t)]dt}
\begin{aligned}
V(s,x) =
\inf_{u(\cdot) \in \mathcal {U}[s,T]}
&\mathbb{E}
\Bigg\{
\int_{s}^{T}
\Big[
A_1 I_p(t) + A_2 L_p(t) + A_3 I_v (t)
\\
& + c_1 u ^ 2_1(t) + c_2 u ^ 2_2(t) + c_3 u ^ 2_3(t)
\Big]
dt
\Bigg\}
\end{aligned}
V(s,x)=u(⋅)∈U[s,T]infE{∫sT[A1Ip(t)+A2Lp(t)+A3Iv(t)+c1u12(t)+c2u22(t)+c3u32(t)]dt}
\begin{aligned}
V(s,x) =
\inf_{u(\cdot) \in \mathcal {U}[s,T]}
&\mathbb{E}
\Bigg\{
\int_{s}^{T}
\Big[
A_1 I_p(t) + A_2 L_p(t) + A_3 I_v (t)
\\
& + c_1 u ^ 2_1(t) + c_2 u ^ 2_2(t) + c_3 u ^ 2_3(t)
\Big]
dt
\Bigg\}
\end{aligned}
−Vt+u∈UsupHG(t,x,u,−Vx,−Vxx)=0
\begin{aligned}
&-V_t + \sup_{u \in U} H_G(t, x, u, -V_x, -V_{xx})=0
\end{aligned}
HG(t,x,u,p,P)=⟨p,f(t,x,u)⟩−c(t,x,u)+21trace(g⊤(t,x,u)Pg(t,x,u))
\begin{aligned}
H_G(t,x,u,p,P) &= \langle p, f(t,x,u) \rangle - c(t,x,u)
\\
&+\dfrac{1}{2} \mathtt{trace}\Big( g^\top(t,x,u)Pg(t,x,u)\Big)
\end{aligned}
Hdet(t,u,x,−Vx)=A1Iv+A2Lp+A3Iv+i=1∑3ciui2−Vx1(−βpSpIv+(r1+u1)Lp+(r2+u2)Ip)−Vx2(βpSpIv−(b+r1+u1)Lp)−Vx3(bLp−(r2+u2)Ip)−Vx4(−βvSvIp−(γ+γf+u3)Sv+(1−θ)μ)−Vx5(βvSvIp−(γ+γf+u3)Iv+θμ).
\begin{aligned}
H_{det} &(t,u,x,-V_x) = A_1I_v+A_2L_p+A_3I_v
+\sum_{i=1}^{3}c_iu_i^2\\
&-V_{x_1}(-\beta_p S_p I_v +(r_1 +u_1)L_p
+ (r_2 + u_2) I_p)\\
& -V_{x_2}(\beta_p S_p I_v-(b +r_1 + u_1)L_p)\\
&-{V_{x_3}}(b L_p - (r_2 + u_2) I_p)\\
&-V_{x_4}(-\beta_v S_v I_p - (\gamma+\gamma_f+u_3) S_v
+(1-\theta)\mu)\\
&-V_{x_5}(\beta_v S_v I_p
-(\gamma+\gamma_f+u_3) I_v+\theta\mu).
\end{aligned}
HJB (Dynamic Programming)- Course of dimensionality
≈ HJB
\approx \
\mathtt{HJB}
HJB(Neuro-Dynamic Programming)
Abstract dynamic programming.
Athena Scientific, Belmont, MA, 2013. viii+248 pp.
ISBN:978-1-886529-42-7
ISBN:1-886529-42-6
Rollout, policy iteration, and distributed reinforcement learning.
Revised and updated second printing
Athena Sci. Optim. Comput. Ser.
Athena Scientific, Belmont, MA, [2020], ©2020. xiii+483 pp.
ISBN:978-1-886529-07-6
Reinforcement learning and optimal control
Athena Sci. Optim. Comput. Ser.
Athena Scientific, Belmont, MA, 2019, xiv+373 pp.
ISBN: 978-1-886529-39-7
Modeling a traffic light warning system for acute respiratory infections as an optimal control problem
Saul Diaz Infante Velasco
Adrian Acuña Zegarra
Jorge Velasco Hernandez
sauldiazinfante@gmail.com
Guidelines for estimating the risk of the epidemiological traffic light
Vaccination camping
Omicron variant
Introduction
We propose an extension of the classic Kermack-McKendrick mathematical model. 𝑁(𝑡) is constant and is split into four compartments:
- susceptible (𝑆(𝑡)),
- infected (𝐼(𝑡)),
- recovered (𝑅(𝑡)),
- and vaccinated (𝑉 (𝑡)).
Susceptible individuals can become infected when interacting with an infectious individual. After a period of time (1∕𝛾), infected people recover. Recovered people lose their natural immunity after a period of time 1∕𝜃.
Ck(t∗)=1−(1−N(t∗)I(t∗))k
C_k(t^*) = 1 - \left(1 - \frac{I(t^*)}{N(t^*)}\right)^k
Risk index
Another closely related index that has been used to monitor and evaluate the development of ARI is the event gathering risk, developed by Chande (2020).
Chande, A., Lee, S., Harris, M. et al. Real-time, interactive website for US-county-level COVID-19 event risk assessment. at Hum Behav, 1313–1319 (2020). https://doi.org/10.1038/s41562-020-01000-9
S′=I′=V′=R′=μN−λ(t)S−(ϕ+μ)S+ωV+θRλ(t)(S+(1−σ)V)−(γ+μ)IϕS−(1−σ)λ(t)V−(ω+μ)VγI−(θ+μ)R
\begin{aligned}
S' =& \mu N - \lambda(t)S
- (\phi + \mu) S + \omega V + \theta R
\\
I' =& \lambda(t) (S + (1 - \sigma)V)
- (\gamma + \mu) I
\\
V' =& \phi S - (1 - \sigma) \lambda(t)V - (\omega + \mu) V
\\
R' =& \gamma I - (\theta + \mu) R
\end{aligned}
C′=k(1−C)(1−k1)(1−(1−C)k1)×(β(t)(1−C(t))(S+(1−σ)V)N1−(γ+μ)).
\begin{aligned}
C^{\prime} =& k (1 - C)^{\left(1 - \frac{1}{k}\right)}
\left(1 - (1 - C)^{\frac{1}{k}}\right)
\\
& \times
\left(
\beta(t) (1 - C(t))
(S + (1 - \sigma)V) \frac{1}{N} - (\gamma + \mu)
\right).
\end{aligned}
β(t)λ(t)=(1+acos(3652πt))β0=β(t)(1−C(t))NI(t)
\begin{aligned}
\beta(t) & =
\left(
1 +
a \cos
\left(
\frac{2\pi}{365} t
\right)
\right) \beta_0
\\
\lambda(t) & = \beta(t) (1 - C(t)) \frac{I(t)}{N}
\end{aligned}
# MODEL FORMULATION
C(t)=1−(1−NI(t))k
C(t) = 1 - \left(1 - \frac{I(t)}{N}\right)^k
Risk index
gives the probability of finding at time t an infected person in a group of k individuals.
C′=k(1−C)(1−k1)(1−(1−C)k1)×(β(t)(1−C(t))(S+(1−σ)V)N1−(γ+μ))
\begin{aligned}
C' =& k (1 - C)^{\left(1 - \frac{1}{k}\right)}
\left(1 - (1 - C)^{\frac{1}{k}}\right)
\\
&
\times
\left(
\beta(t) (1 - C(t))
(S + (1 - \sigma)V) \frac{1}{N} - (\gamma + \mu)
\right)
\end{aligned}
s′i′v′r′C′=μ−λ^(t)s−(ϕ+μ)s+ωv+θr=λ^(t)(s+(1−σ)v)−(γ+μ)i=ϕs−(1−σ)λ^(t)v−(ω+μ)v=γi−(θ+μ)r=k(1−C)(1−k1)(1−(1−C)k1)×(β(t)(1−C(t))(s+(1−σ)v)−(γ+μ))
\begin{aligned}
s^{\prime} &= \mu - \hat{\lambda}(t)s - (\phi + \mu) s + \omega v + \theta r
\\
i^{\prime} &= \hat{\lambda}(t) (s + (1 - \sigma)v) - (\gamma + \mu) i
\\
v^{\prime} &= \phi s - (1 - \sigma) \hat{\lambda}(t)v - (\omega + \mu) v
\\
r^{\prime} &= \gamma i - (\theta + \mu) r
\\
C^{\prime} &= k (1 - C)^{\left(1 - \frac{1}{k}\right)}
\left(
1 - (1 - C)^{\frac{1}{k}}
\right)
\\
& \times
\left(
\beta(t) (1 - C(t)) (s + (1 - \sigma) v) - (\gamma + \mu)
\right)
\end{aligned}
# REPRODUCTIVE NUMBER
s=NS;i=NI;v=NV;r=NR
s = \frac{S}{N};
\quad i = \frac{I}{N};
\quad v = \frac{V}{N};
\quad r = \frac{R}{N}
Normalization
Ω={(s,i,v,r,C)∈R+5:0≤C≤1,0≤y≤1,∀y∈{s,i,v,r},s+i+v+r=1}
\begin{aligned}
\Omega =
\Big \{
&
(s, i, v, r, C)\in \mathbb{R}^5_+
:
\\
&
0\leq C\leq 1,
\\
&
0 \leq y \leq 1,
\forall y \in \{s, i, v, r \},
\\
&
s + i + v + r = 1
\Big \}
\end{aligned}
Invariance
R0=(γ+μ)β0(1−C∗)(s∗+(1−σ)v∗)
\mathcal{R}_0 = \frac{\beta_0(1 - C^*)(s^* + (1 - \sigma)v^*)}{(\gamma + \mu)}
Basic Reproductive number
s∗v∗C∗=ϕ+ω+μω+μ=ϕ+ω+μϕ=0
\begin{aligned}
s^* &= \frac{\omega + \mu}{\phi + \omega + \mu}
\\
v^* &= \frac{\phi}{\phi + \omega + \mu}
\\
C^* &= 0
\end{aligned}
FDE
Effective reproduction number
Rt=(γ+μ)β(t)(1−C(t))(s(t)+(1−σ)v(t))
\mathcal{R}_t = \frac{\beta(t)(1 - C(t))(s(t) + (1 - \sigma)v(t))}{(\gamma + \mu)}
# Light Traffic Policies and Optimal Control
x(t)⊤:=(s(t),i(t),v(t),r(t),C(t))
x(t)^\top:= (s(t), i(t),v(t),r(t), C(t))
a(tk)φ
a(t_k) \xrightarrow{\varphi}
A(ξ):={green,yellow,orange,red}
A(\xi):= \Big \{green, yellow, orange, red \Big\}
(uβ4,uk4)
(u_{\beta}^{4}, u_k^{4})
(uβ1,uk1)
(u_{\beta}^{1}, u_k^{1})
(uβ3,uk3)
(u_{\beta}^{3}, u_k^{3})
(uβ2,uk2)
(u_{\beta}^{2}, u_k^{2})
- A decision-maker can apply a strategy from a finite set of actions.
- The set of actions implies a particular effect accordingly to the light semaphore.
- We describe this effect by modulating the transmission rate and size of gathering :
β(t), k.
\beta(t),\ k.
- A decision-maker can apply a strategy from a finite set of actions.
- The set of actions implies a particular effect accordingly to the light semaphore.
- We describe this effect by modulating the transmission rate and size of gathering.
λ(uβ,Tj):=k(uk,Tj):=β(t)(1−uβ)(1−C(t))y(t)(1−uk)k
\begin{aligned}
\widehat{\lambda}(u_{\beta}, T_j):=
&
\beta(t)(1 - u_{\beta}) (1 - C(t)) y(t)
\\
k(u_k, T_j):=
& (1 - u_k) k
\end{aligned}
x′=y′=v′=z′=C′=μ−λ(uβ,t)x−(ϕ+μ)x+ωv+θzλ(uβ,t)(x+(1−σ)v)−(γ+μ)yϕx−(1−σ)λ(uβ,t)v−(ω+μ)vγy−(θ+μ)zk(uk,Tj)(1−C)(1−k(uk,Tj)1)(1−(1−C)k(uk,Tj)1)×[β(t)(1−uβ)(1−C(t))(x+(1−σ)v)−(γ+μ)]
\begin{aligned}
x^{\prime} =&
\mu - \widehat{\lambda}(u_{\beta}, t) x - (\phi + \mu) x + \omega v + \theta z
\\
y^{\prime} =&
\widehat{\lambda}(u_{\beta}, t) (x + (1 - \sigma)v) - (\gamma + \mu) y
\\
v^{\prime} =&
\phi x - (1 - \sigma)
\widehat{\lambda}(u_{\beta}, t) v - (\omega + \mu) v
\\
z^{\prime} =&
\gamma y - (\theta + \mu) z
\\
C^{\prime} =&
k(u_k, T_j)
(1 - C) ^ {
\left(
1 - \frac{1}{k(u_k, T_j)}
\right)
}
\left(
1 - (1 - C) ^ \frac{1}{k(u_k, T_j)}
\right)
\\
& \times
\left[
\beta(t) (1 - u_{\beta}) (1 - C(t)) (x + (1 - \sigma) v)
- (\gamma + \mu)
\right]
\end{aligned}
Ti
T_i
Ti+1
T_{i+1}
Ti−1
T_{i-1}
decision periodpj:=tj−tj−1
\overbrace{\text{decision period}}^{p_j := t_j - t_{j-1}}
\underset{
\substack{
j \in A \\
A:=\{\text{green, yellow, orange, red} \}
}
}
%}
{
J
\left(
\xi^{[p_i]}, u_{\beta} ^{[j]}, u_k^{[j]}
\right)
}
:=
\underbrace{
\int_{0}^T
a_I y(s)
}_{YLD}
ds
+
\underbrace{
\int_{0}^T
a_C C(s)
+
a_{\beta} u_{\beta}^2(s)
+
a_k u_k^2(s)
}_{
\substack{
\text{political and economic}
\\
\text{implications}
}
}
ds.
β⇝β(1−uβ)
\beta \rightsquigarrow \beta (1 -u_{\beta})
k⇝k(1−uk)
k \rightsquigarrow k (1 - u_{k})
\underset{
\substack{
j \in A \\
A:=\{\text{green, yellow, orange, red} \}
}
}
%}
{
J
\left(
\xi^{[p_i]}, u_{\beta} ^{[j]}, u_k^{[j]}
\right)
}
:=
\underbrace{
\int_{0}^T
a_I y(s)
}_{YLD}
ds
+
\underbrace{
\int_{0}^T
a_C C(s) +
a_{\beta} u_{\beta}^2(s)
+
a_k u_k^2(s)
}_{
\substack{
\text{political and economic}
\\
\text{implications}
}
}
ds.
- The decision maker only chooses the strategy from a finite set of actions--as mentioned above-- and according to the light traffic protocol. Thus, the controller decides which color from the plausible light-traffic actions for the next period.
- The corresponding authorities periodically meet every few weeks and make a decision.
- The decision taken minimizes the functional J subject to the mentioned dynamics.
Hypothesis
Ti
T_i
Ti+1
T_{i+1}
Ti−1
T_{i-1}
decision periodpj:=tj−tj−1
\overbrace{\text{decision period}}^{p_j := t_j - t_{j-1}}
(OCP) Decide in each stage (a week), the light color that minimize functional cost J
β(t):=λ(uβ,tj):=k(uk,tj):=(1+acos(3652πt))β0β(t)(1−uβ)(1−C(t))i(t)(1−uk)k
\begin{aligned}
\beta(t) := &
\left(
1 +
a \cos
\left(
\frac{2\pi}{365} t
\right)
\right) \beta_0
\\
\widehat{\lambda}(u_{\beta}, t_j):=
&
\beta(t)(1 - u_{\beta}) (1 - C(t)) i(t)
\\
k(u_k, t_j):=
& (1 - u_k) k
\end{aligned}
j∈AA:={green, yellow, orange, red}minJ(ξ[pi],uβ[j],uk[j]):=YLD∫Ti−1TiaIi(s)ds+political and economicimplications∫Ti−1TiaCC(s)+aβuβ2(s)+akuk2(s)ds.
\underset{
\substack{
j \in A \\
A:=\{\text{green, yellow, orange, red} \}
}
}
{
\min
}
J
\left(
\xi^{[p_i]},
u_{\beta} ^ {[j]},
u_k ^ {[j]}
\right)
:=
\underbrace{
\int_{T_{i-1}} ^ {T_{i}}
a_I i(s)
}_{YLD}
ds
+
\underbrace{
\int_{T_{i-1}} ^{T_{i}}
a_C C(s) +
a_{\beta} u_{\beta}^2(s)
+
a_k u_k ^ 2(s)
}_{
\substack{
\text{political and economic}
\\
\text{implications}
}
} ds.
subject to:
s′i′v′r′C′=μ−λ^(t)s−(ϕ+μ)s+ωv+θr=λ^(t)(s+(1−σ)v)−(γ+μ)i=ϕs−(1−σ)λ^(t)v−(ω+μ)v=γi−(θ+μ)r=k(1−C)(1−k1)(1−(1−C)k1)×(β(t)(1−C(t))(s+(1−σ)v)−(γ+μ))
\begin{aligned}
s^{\prime} &= \mu - \hat{\lambda}(t)s - (\phi + \mu) s + \omega v + \theta r
\\
i^{\prime} &= \hat{\lambda}(t) (s + (1 - \sigma)v) - (\gamma + \mu) i
\\
v^{\prime} &= \phi s - (1 - \sigma) \hat{\lambda}(t)v - (\omega + \mu) v
\\
r^{\prime} &= \gamma i - (\theta + \mu) r
\\
C^{\prime} &= k (1 - C)^{\left(1 - \frac{1}{k}\right)}
\left(
1 - (1 - C)^{\frac{1}{k}}
\right)
\\
& \times
\left(
\beta(t) (1 - C(t)) (s + (1 - \sigma) v) - (\gamma + \mu)
\right)
\end{aligned}
Counterfactual vs controlled dynamics
Counterfactual vs controlled dynamics
Counterfactual vs controlled dynamics
The influence of mobility restriction expenses over prevalence and cost
j∈AA:={green, yellow, orange, red}J(ξ[pi],uβ[j],uk[j]):=
\underset{
\substack{
j \in A \\
A:=\{\text{green, yellow, orange, red} \}
}
}
{
J
\left(
\xi^{[p_i]},
u_{\beta} ^ {[j]},
u_k ^ {[j]}
\right)
}
:=
%
\int_{T_{i-1}} ^ {T_{i}}
a_I i(s) + a_C C(s)
ds
+
\int_{T_{i-1}} ^{T_{i}}
a_{\beta}
u_{\beta}^2(s)
+
a_k u_k ^ 2(s)
ds.
Expenses
due to mobility restrictions
aβ↘
a_{\beta} \searrow
The influence of decision period span over prevalence and cost
j∈AA:={green, yellow, orange, red}J(ξ[pi],uβ[j],uk[j]):=
\underset{
\substack{
j \in A \\
A:=\{\text{green, yellow, orange, red} \}
}
}
{
J
\left(
\xi^{[p_i]},
u_{\beta} ^ {[j]},
u_k ^ {[j]}
\right)
}
:=
%
\int_{T_{i-1}} ^ {T_{i}}
a_I i(s) + a_C C(s)
ds
+
\int_{T_{i-1}} ^{T_{i}}
a_{\beta}
u_{\beta}^2(s)
+
a_k u_k ^ 2(s)
ds.
decisionperiodpi:=Ti+1−Ti
\underbrace{p_i:=T_{i+1} - T_i}_{\substack{decision \\ period}}
pi↗
p_i \nearrow
- Our model suggest that this kind of Policies follows a delicate balanced between health benefit and economic cost
Perspectives
- Uncertainty quantification
- Partial information
- Games
References
- Chande, A., Lee, S., Harris, M. et al. Real-time, interactive website for US-county-level COVID-19 event risk assessment. Nat Hum Behav 4, 1313–1319 (2020). https://doi.org/10.1038/s41562-020-01000-9
- Chan, H.F., Skali, A., Savage, D.A. et al. Risk attitudes and human mobility during the COVID-19 pandemic. Sci Rep 10, 19931 (2020). https://doi.org/10.1038/s41598-020-76763-2
- P. van den Driessche, James Watmough, Reproduction numbers and sub-threshold endemic equilibria for compartmental models of disease transmission, Mathematical Biosciences, Volume 180, Issues 1–2, 2002,Pages 29-48,ISSN 0025-5564, https://doi.org/10.1016/S0025-5564(02)00108-6
Gracias
- Salcedo-Varela, G. & Diaz-Infante, S. Threshold behaviour of a stochastic vector plant model for Tomato Yellow Curl Leaves disease: a study based on mathematical analysis and simulation. Int J Comput Math 1–0 (2022) doi:10.1080/00207160.2022.2152680.
- Salcedo‐Varela, G. A., Peñuñuri, F., González‐Sánchez, D. & Díaz‐Infante, S. Synchronizing lockdown and vaccination policies for COVID‐19: An optimal control approach based on piecewise constant strategies. Optim. Control Appl. Methods (2023) doi:10.1002/oca.3032.
- Diaz-Infante, S., Gonzalez-Sanchez, D. & Salcedo-Varela, G. Handbook of Visual, Experimental and Computational Mathematics, Bridges through Data. 1–19 (2023) doi:10.1007/978-3-030-93954-0_37-1.
Some applications of stochastic control in epidemiology A framework based on stochastic optimal control and dynamic programming. Gabriel Salcedo-Varela, David González-Sánchez,
Saúl Díaz-Infante Velasco,
saul.diazinfante @unison.mx, March 29, 2024 MathBio seminar, ASU