$$\theta_t = \underbrace{\Big(\sum_{k=1}^{t-1}x_k x_k^\top  + \lambda I\Big)^{-1}}_{A_{t-1}^{-1}}\underbrace{\sum_{k=1}^{t-1}x_ky_k }_{b_{t-1}}$$

Sherman-Morrison formula: $\displaystyle (A+uv^\top)^{-1} = A^{-1} - \frac{A^{-1}uv^\top A^{-1}}{1+v^\top A^{-1}u}$

$$s_{t+1} = F(s_t)$$

(Autonomous) discrete-time dynamical system where $F:\mathcal S\to\mathcal S$

$\mathcal S$ is the state space. The state is sufficient for predicting its future.

Given initial state $s_0$, the solutions to difference equations, i.e. trajectories: $$(s_0, F(s_0), F(F(s_0)), ... )$$

What might trajectories look like?

• converging $(1, 0.1, 0.001, 0.0001, ...)$
• diverging $(1, 10, 100, 1000,...)$
• oscillating $(1, -1, 1, -1, 1, ...)$
• converging towards oscillation $(0.9, -0.99, 0.999, -0.9999,...)$

An equilibrium point $s_\mathrm{eq}$ satisfies

$s_{eq} = F(s_{eq})$

examples:

• $s_{t+1} = s_t$
• $s_{t+1} = 2s_t$
• $s_{t+1} = 0.5 s_t$

Suppose that $s_0=v$ is an eigenvector of $A$

$$s_{t} =\lambda^t v$$

$\displaystyle s_t = \sum_{i=1}^n v_i \lambda_i^t (u_i^\top s_0)$ is a weighted combination of (right) eigenvectors

General case: real eigenvalues with geometric multiplicity equal to algebraic multiplicity

$0<\lambda_2<\lambda_1<1$

$0<\lambda_2<1<\lambda_1$

$1<\lambda_2<\lambda_1$

Exercise: what do trajectories look like when $\lambda_1$ and/or $\lambda_2$ is negative? (demo notebook)

$0<\alpha^2+\beta^2<1$

$1<\alpha^2+\beta^2$

Exercise: what do trajectories look like when $\alpha$ is negative? (demo notebook)

General case: pair of complex eigenvalues

$\lambda = \alpha \pm i \beta$

$$\begin{bmatrix}1\\0\end{bmatrix} \to \begin{bmatrix}\alpha\\ \beta\end{bmatrix}$$

rotation by $\arctan(\beta/\alpha)$

scale by $\sqrt{\alpha^2+\beta^2}$

$0<\lambda<1$

$1<\lambda$

Exercise: what do trajectories look like when $\lambda$ is negative? (demo notebook)

General case: eigenvalues with geometric multiplicity $>1$

$$\left(\begin{bmatrix} \lambda & \\  & \lambda\end{bmatrix} + \begin{bmatrix}  & 1\\  & \end{bmatrix} \right)^t$$

$$=\begin{bmatrix} \lambda^t & t\lambda^{t-1}\\  & \lambda^t\end{bmatrix}$$

All matrices are similar to a matrix of Jordan canonical form

where $J_i = \begin{bmatrix}\lambda_i & 1 & &\\ & \ddots & \ddots &\\ &&\ddots &1\\ && &\lambda_i \end{bmatrix}\in\mathbb R^{m_i\times m_i}$

Reference: Ch 3d and 4 in Callier & Desoer, "Linear Systems Theory"

$\begin{bmatrix} J_1&&\\&\ddots&\\&&J_p\end{bmatrix}$

$m_i$ is geometric multiplicity of $\lambda_i$

Theorem: Let $\{\lambda_i\}_{i=1}^n\subset \mathbb C$ be the eigenvalues of $A$.
Then for $s_{t+1}=As_t$, the equilibrium $s_{eq}=0$ is

• asymptotically (exponentially, globally) stable $\iff \max_{i\in[n]}|\lambda_i|<1$
• unstable if $\max_{i\in[n]}|\lambda_i|> 1$
• call $\max_{i\in[n]}|\lambda_i|=1$ "marginally (un)stable"

$\mathbb C$

Linearization via Taylor Series:

$s_{t+1} = F(s_t)$

Stability via linear approximation of nonlinear $F$

The Jacobian $J$ of $G:\mathbb R^{n}\to\mathbb R^{m}$ is defined as $$J(x) = \begin{bmatrix}\frac{\partial G_1}{\partial x_1} & \dots & \frac{\partial G_1}{\partial x_n} \\ \vdots & \ddots & \vdots \\ \frac{\partial G_m}{\partial x_1} &\dots & \frac{\partial G_m}{\partial x_n}\end{bmatrix}$$

$F(s_{eq}) + J(s_{eq})  (s_t - s_{eq})$ + higher order terms

$s_{eq} + J(s_{eq})  (s_t - s_{eq})$ + higher order terms

$s_{t+1}-s_{eq} \approx J(s_{eq})(s_t-s_{eq})$

Consider the dynamics of gradient descent on a twice differentiable function $g:\mathbb R^d\to\mathbb R^d$

$\theta_{t+1} = \theta_t - \alpha\nabla g(\theta_t)$

Jacobian $J(\theta) = I - \alpha \nabla^2 g(\theta)$

Definition: A Lyapunov function $V:\mathcal S\to \mathbb R$ for $F,s_{eq}$ is continuous and

• (positive definite) $V(s_{eq})=0$ and $V(s_{eq})>0$ for all $s\in\mathcal S - \{s_{eq}\}$
• (decreasing) $V(F(s)) - V(s) \leq 0$ for all $s\in\mathcal S$
• Optionally,
  • (strict) $V(F(s)) - V(s) < 0$ for all $s\in\mathcal S-\{s_{eq}\}$
  • (global) $\|s-s_{eq}\|_2\to \infty \implies V(s)\to\infty$

Theorem (3.3): Suppose $F$ is locally Lipschitz, $s_{eq}$ is a fixed point, and let $\{\lambda_i\}_{i=1}^n\subset \mathbb C$ be the eigenvalues of the Jacobian $J(s_{eq})$. Then $s_{eq}$ is

• asymptotically stable if $\max_{i\in[n]}|\lambda_i|<1$
• unstable if $\max_{i\in[n]}|\lambda_i|> 1$

Next time: actions, disturbances, measurement

References: Bof, Carli, Schenato, "Lyapunov Theory for Discrete Time Systems"; Callier & Desoer, "Linear Systems Theory"