Sub-Riemannian landmark matching and ResNets

Slides at slides.com/erikjansson

Link to preprint

Joint work with Klas Modin

ResNets?

Fully-connected equal layers

with a fixed final node

Skip connection \(\implies \) Explicit Euler

\ldots

ResNets?

Infinite number of layers \(\implies\)

Time-continuous optimal control problem

Skip connection \(\implies \) Explicit Euler

\ldots

Diffeomorphic shape matching

f_0 \in C^\infty(M)
f_1 \in C^\infty(M)

Landmark matching

x_1,\dots, x_n \in M
c_1,\dots,c_n \in M

Landmark matching

\dot y_i = v(t,y_i), ~ y_i(0)=x_i, v(t,\cdot) \in \mathcal{X}(M)

Evolve points along a time-dependent vector field!

Which vector field?

Landmark matching

Penalize missing targets:

\sum_{i=1}^n d_M^2(y_i(1),c_i)

Penalize weird vector fields:

\int_0^1 \int_M Lv(t) \cdot v(t)\mathrm{d}x \mathrm{d}t
\,\, E(v) = \int_0^1 \int_M Lv(t) \cdot v(t)\mathrm{d}x \mathrm{d}t+ \sum_{i=1}^n d_M^2(y_i(1),c_i) \,\,

Key question: How does \(v\) evolve in time? 

Landmark matching

 Trajectory of \(m_t\) does not care about matching!

Cannonball does not care about target

Landmark matching

Use the Lagrangian \({\int_0^T \int_M Lv(t) \cdot v(t)\mathrm{d}x\mathrm{d}t}_{}\) 

Calculus of variations gives:

\,\dot m = \nabla^T_v m - \nabla_m v + m \operatorname{div}v\,\\ m = Lv
\dot m = \operatorname{ad}_v^T m

Modifying landmark matching

x_1,\dots, x_n \in M
c_1,\dots,c_n \in M

Modifying landmark matching

x_1,\dots, x_n \in M
c_1,\dots,c_n \in {\color{red} N}
h

Modifying landmark matching

In usual LM, all vector fields are available

Idea: Constrain set of vector fields

Sub-Riemannian landmark matching!

Modifying landmark matching

v \in \mathcal{S} \iff \exists u \in \mathcal{U} \text{ s.t. } v(x) = F(u)(x)

\(\mathcal{S}\)

\(v = F(u)(x)\)

\(\mathfrak{X}(M)\)

\(F(u)\)

\(\mathcal{U}\)

\(u\)

Modifying landmark matching

v = F(u)

Lagrangian becomes \({\int_0^T \int_M LF(u(t)) \cdot F(u(t))\mathrm{d}x\mathrm{d}t}_{}\) 

Points evolve by \(\dot y_i = F(u(t))(y_i)\)

Modifying landmark matching

A(u)\dot u = (D_u^T m \circ L^{-1}) \cdot \operatorname{ad}^T_v m
A(u) = D_u^T m \circ L^{-1} \circ D_u m

How to find dynamics of \(u\)?

Idea: Apply chain rule to \(\dot m = \operatorname{ad}_v^T m\)

A(u)\dot u = (D_u^T m \circ L^{-1}) \cdot \operatorname{ad}^T_v m
\dot m = \operatorname{ad}^T_v m +L\mathcal{M}(u)

Modifying landmark matching

\(\mathcal{S}\)

\(v = F(u)(x)\)

\(\mathfrak{X}(M)\)

\(F(u)\)

\(\mathcal{U}\)

\(u\)

Control-affine systems

\mathcal U= \mathbb{R} ^m
F(u){=}X^0 {+} {\sum_{i=1}^m} u_i X^i
[A(u)]_{ij}{=}\int_{M} LX^i \cdot X^j

Simplification of geometry!

  • Set \(M=\mathbb{R}^2/(2\pi \mathbb{Z})^2 \)
  • Let \(X^i\) be eigenfunctions of toroidal Laplacian
m=16, \\ \cos(2y)\partial y, \sin(3x)\partial x,\cos(y)\partial x,...

Computational example

Control-affine systems

 

  1. Initial guessof inital control \(u^{[0]}_0\)
  2. Evolve \(u^{[0]}_0\) and landmarks
  3. Evaluate energy functional \(E\).
  4. Update control by \(u^{[1]}_0 \leftarrow u^{[0]}_0-\epsilon \nabla_{u^{[1]}_0}E\)
A(u)\dot u = (D_u^T m \circ L^{-1}) \cdot \operatorname{ad}^T_v m
\dot y_i = F(u)(y_i), ~y_i(0)=x_i

Computational example

"Adjust initial conditions until target is hit"

Control-affine systems

Goal: Move landmarks from one fish to the other

Control-affine systems

Control-affine systems

Goal: Move landmarks on torus to \([0,1]\)

[{{\color{maroon}0},\color{blue}1}]

Projection

Control-affine systems

Badly behaved transformation!

Increase regularization strength?

Control-affine systems

Control-affine systems

Why study sR-LM?

When matching landmarks, points are moved by a vector field parametrized by some control variable.

A neural networks moves points along a vector field determined by weights and biases.

  • Landmarks
  • Image
  • Shooting
  • Warping new landmarks
  • Control parameters
  • Input data, e.g. images
  • "Meta-image"
  • Training network
  • Testing
  • Weights and biases

 View networks with shape analysis glasses!

Why study sR-LM?

Why study sR-LM?

Available vector fields:

Iterated Lie brackets

If distribution is integrable, we

move only along blue line

To have reachability, we must destroy integrability

\(\implies\) Nonlinearity!

Move from

to

  • We can parametrize landmark matching and retain geometric framework 

  • There is a connection to neural networks

  • Future work: leverage connection for concrete AI applications