Shape spaces of images and curves

Faculty of Science, University of Copenhagen

Stefan Sommer

Department of Computer Science, University of Copenhagen

\( \phi \)

Statistical Shape models

Session 1: (L 9-9:45) Shape analysis and actions of the  diffeomorphism group

Session 2: (E 10-10:45) Landmark analysis in Theano Geometry

Session 3: (L 11-11:45) Linear representations and random orbit model

Session 4: (E 12:30-13:15) Landmarks statistics in Theano Geometry

Session 5: (L 13:30-14:15) Shape spaces of images, curves and surfaces

Session 6: (E 14:30-15:15) Analysis of continuous shapes

 

Shapes

Objects on which the diffeomorphism group acts

 

\(d=2,3\)

  • landmarks in \(\mathbb{R}^d\): \(\mathbf{q}=(q_1,\ldots,q_n)\)
  • curves: \(\gamma:\mathbb{S}^1\to\mathbb{R}^d\)
  • surfaces: \(\gamma:\mathbb{S}^2\to\mathbb{R}^d\)
  • images: \(I:\Omega\to\mathbb{R}\), \(\Omega\subset\mathbb{R}^d\)
  • tensor fields: \(\Omega\to\mathcal{T}^k_l\), \(\Omega\subset\mathbb{R}^d\)
  • ...

One framework - multiple shape types

Image Matching

matching problem, image registration:

\[E_{s_0,s_1}(\phi)=\mathrm{min}_{\phi_t}R(\phi_t)+\frac1\lambda S(\phi_T.s_0,s_1)\]

 

Dissimilarity measure:

     \(S(I^0,I^1)=\int_\Omega|I^0(x)-I^1(x)|^2dx\)

     or mutual information or similar

 

\(I^0\) moving image: \(\phi_t.I^0\)            \(I^1\) fixed image

 

landmarks can be matched precisely, images generally cannot: lack of transitivity of \(G\)-action

Image Matching

matching problem, image registration:

\[E_{s_0,s_1}(\phi)=\mathrm{min}_{\phi_t}R(\phi_t)+\frac1\lambda S(\phi_T.s_0,s_1)\]

 

Optimal flows \(\phi_t\) for \(E\) with

\(R(\phi_t):=\int_0^T\|\partial_t \phi_t\|_{\phi_t}^2dt\), \(v_t=\partial_t\phi_t\circ\phi_t^{-1}\)

are geodesics on \(G=\mathrm{Diff}(\Omega)\)

 

Geodesic equation:

 

2v(t)-K\left(2 \lambda \: | D \phi^v (t,1)^{-1} | \:\nabla \phi^v(0,t).q_s \left[\phi^v(0,t).q_s - \phi^v(1,t).q_t \right] \right)

Euler-Poincaré reduction

General form of geodesic equation:

\[\frac{d}{dt} \frac{\delta l}{\delta v}+\mathrm{ad}^*_v \frac{\delta l}{\delta v}=0\]

 

Valid for all shape space (only shape action varies)

 

Uses right invariance of metric on \(G\): 

\(\|\partial_t\phi_t\|_{\phi_t}=\|v_t\|\), \(v_t=\partial_t\phi_t\circ\phi_t^{-1}\)

 

The (infinite dimensional) variable \(\phi_t\) is reduced

Lagragian to Eulerian coordinate change in fluid systems

\[\frac{\partial m}{\partial t} + v \cdot \nabla m + \nabla v^T \cdot m + m \nabla \cdot v = 0\]

Computation

Path straigthening: Gradient descent on \(v_t\)

\[v_t^{k+1}=v_t^k-\epsilon\nabla_{v_t}E(v_t)|_{v_t=v_t^k}\]

swipe back and forth between t=0 and t=T

 

Shooting: use that geodesic path is determined by \(v_0\)

\[v_0^{k+1}=v_t^k-\epsilon\nabla_{v_0}E(v_0)|_{v_0=v_0^k}\]

use adjoint equations for transporting

\(\nabla_{\phi_T} S(\phi_T.s^0,s^1)\) to \(\nabla_{v_0}S(\phi_T.s^0,s^1)\)

(backpropagation)

 

Both schemes are numerically challenging

Curves and Surfaces

curves: \(\gamma:\mathbb{S}^1\to\mathbb{R}^d\),           surfaces: \(\gamma:\mathbb{S}^2\to\mathbb{R}^d\)

Similarity \(S(\gamma^0,\gamma^1)=\int_{\mathbb{S}^1}\|\gamma^0(\theta)-\gamma^1(\theta)\|^2d\theta\) is dependent on parametrizations of \(\gamma^0\) and \(\gamma^1\)

 

Current metric provides invariant metric on curves and surfaces:

    RKHS on \(\Omega\) with kernel \(G\)

    curve \(\gamma\) determines a covector in \(V^*\):

            \(v\mapsto\int_\Omega v(x)^T\gamma(x)dx\)

    currents norm is \(\|\gamma\|\) using the RKHS norm on \(V^*\)

 

discretized: \(l\approx \sum_{i=1}^n\delta_{l(t_i)}^{\dot{l}(t_i)}\) gives 

 

Deformetrica http://www.deformetrica.org/

\|l\|^2\approx\|\sum_{i=1}^n\delta_{l(t_i)}^{\dot{l}(t_i)}\|^2=\sum_{i,j=1}^n\dot{l}(t_i)^TG(l(t_i),l(t_j))\dot{l}(t_j)

Inner metrics on curves

outer metrics: measure deformations of \(\Omega\)

inner metrics: measure deformations of \(\gamma\) itself

  • L2: \(g(v,w)=\int_{\mathbb{S}^1}v(\theta)^Tw(\theta)ds\)
  • Sobolev-type: \(g(v,w)=\int_{\mathbb{S}^1}(L_\gamma v(\theta))^Tw(\theta)ds\)

e.g. \(L_\gamma=I_d-D_s\) or higher order

Elastic: \(g^{a,b}(h,k)=\int_{\mathbb{S}^1}a^2|(D_s h)^{\perp}|\:|(D_s k)^{\perp}|+b^2|(D_s h)^{\top}|\:|(D_s k)^{\top}|ds\)

 

Parametrization invariance:

    \(L_{\gamma\circ\phi}h\circ\phi=(L_\gamma h)\circ \phi\) implies metric on \(\mathrm{Emb}(\mathbb{S}^1,\mathbb{R}^2)/\mathrm{Diff}(\mathbb{S}^1)\)

 

h2metrics: https://github.com/h2metrics/h2metrics

Other shape models

  • Kendall's shape space
         points modulo rotation, translation, scale
  • Point distribution models (PDMs): \(s\in\mathbb{R}^{dn}\)
         linear
  • Elastic shape models
         inner metric
  • Level set based models
  • Hausdorff distances of sets
  • ... and many more

Curve and surface matching

Exercise session:

Shape spaces of images and curves

By Stefan Sommer

Shape spaces of images and curves

  • 405

More from Stefan Sommer