Sub-Riemannian geometry and probabilistic geometric statistics

Stefan Sommer, University of Copenhagen

Faculty of Science, University of Copenhagen

Young researchers between geometry and stochastic analysis, 2021

stochastic shapes and probabilistic geometric statistics

w/ Alexis Arnaudon, Darryl Holm, Sarang Joshi, Tom Fletcher, Frank v.d. Meulen, Moritz Schauer, Benjamin Eltzner, Line Kuhnel, Mathias H. Jensen, Pernille E.H. Hansen

Novo nordisk foundation

Villum foundation

Carlsberg foundation

Statistical shape analysis

Geometric statistics

Generalization of Euclidean statistical notions and techniques to spaces without vector space structure

  • i.i.d. samples \(y_1,\ldots,y_N\in M\)
  • Fréchet mean
    \(\bar{x}=\mathrm{argmin}_{x\in M}\sum_{i=1}^Nd(x,y_i)^2\)

The Fréchet mean is not an expected value.

There is no equivalence between different characterizations of means

- in contrast to Euclidean statistics

Geometric data: Directional statistics

Plane directions:      \(\mathbb{S}^1\)

Geographical data:  \(\mathbb{S}^2\)

3D directions:           \(\mathrm{SO}(3), \mathbb{S}^2\)

Angles:                       \(\mathbb{T}^N\)

Geometric data: Diffusion MRI

Positive, symmetric 3 tensors:   SPD(3)

Geometric data: Functional signals

 continuous curves: \(\mathrm{Emb}(\mathbb S^1,\mathbb R^2)/\mathrm{Diff}(\mathbb S^1) \)

Mieritz et al.,JCEM'15

Shape matching

E_{s_0,s_1}(\phi)=R(\phi)+\frac1\lambda S(\phi.s_0,s_1)

\( \phi\in\mathrm{Diff}(\Omega) \) diffeomorphism of domain \(\Omega\)
action: \(\phi.s=\phi\circ s\)         (shapes)
             \(\phi.s=s\circ\phi^{-1}\)     (images)

\( \phi \)

Riemannian view

R(\phi_t)=\int_0^T\|\partial_t \phi_t\|_{\phi_t}^2dt

\( \phi_t:[0,T]\to\mathrm{Diff}(\Omega) \) path of diffeomorphisms (parameter t)

\mathrm{Diff}(\Omega)
\mathrm{Id}_{\mathrm{Diff}(\Omega)}
\varphi_t

LDDMM: Grenander, Miller, Trouve, Younes, Christensen, Joshi, et al.

Evolution with noise

\partial_t \phi_t = F(\phi_t)\ \to\ d\phi_t=F(\phi_t)dt\color{blue}{+\sigma(\phi_t) dW_t}
\mathrm{Diff}(\Omega)
\mathrm{Id}_{\mathrm{Diff}(\Omega)}
\phi_t

Trouve,Vialard,QAM'12;Vialard,SPA'13;Marsland/Shardlow,SIIMS'17
Arnaudon,Holm,Sommer,IPMI'17; FoCM'18; JMIV'19

Least-squares \(\leftrightarrow\) probabilistic

Deterministic:

  • \(\phi_t\) geodesic evolution
  • exact matching:
    \(\quad\mathrm{argmin}_{\phi_1.s_0=s_1} E(\phi_t)\)
  • inexact matching:
    \(\quad\mathrm{argmin}_{\phi_t} E(\phi_t)\)
  • distance:
    \(\quad d(s_0,s1)^2=E(\phi_t)\)
  • Riemannian least-
    squares

Stochastic:

  • \(\phi_t\) stochastic process
  • bridge:
    \(\quad \phi_t|\phi_T.s_0=s_1\)
  • bridge + noise in observation:
    \(\quad \phi_t|\phi_T.s_0+\epsilon=s_1\)
  • (log) transition density
    \(\quad -\log p_T(s_1; s_0)\)
  • ML/MAP

Inferring geometry of landmarks spaces

  • Landmarks \(\mathbf{q}=(q_1,\ldots,q_n),\,q_i\in\Omega\subseteq\mathbb R^d\)
  • LDDMM cometric \( \left<\xi,\eta\right>_{\mathbf q}=\xi^T K(\mathbf q,\mathbf q)\eta\)
  • kernel \(K(\mathbf q_1,\mathbf q_2)=\alpha e^{-1\frac12 x^T\Sigma^{-1}x}\mathrm{Id}_d\)
  • Riemannian Brownian motion
    $$dq_t^i=g^{kl}\Gamma(\mathbf q_t)^i_{kl}dt+\sqrt{g^*(\mathbf q_t)}^idW_t$$
  • Brownian motion transition density
    $$p_{T,\theta}(\mathbf v)=\frac1{\sqrt{|K(\mathbf v,\mathbf v)|(2\pi T)^{Nd}}}e^{-\frac{\|(\mathbf q_0-\mathbf v)^TK(\mathbf q_0,\mathbf q_0)^{-1}(\mathbf q_0-\mathbf v)\|^2}{2T}}\mathbb E_{\mathbf y_\theta}[\varphi_{\theta}(\mathbf y)]$$

 

 

 

 

 

 

 

 

Sommer,Arnaudon,Kuhnel,Josh,MFCA'17

  • parametric families of probability distributions \(\mu_\theta\)
  • likelihood from density:
    \(\quad\mathcal{L}(\theta; y_1,\ldots,y_N)=\prod_{i=1}^Np_\theta(y_i)\)
  • ML/MAP estimates:
    \(\quad\bar{\theta}=\mathrm{argmax}_\theta\mathcal{L}(\theta; y_1,\ldots,y_N)\)
  • Diffusion mean:
    \(\quad x_t\in M\) Brownian motion
    \(\quad\theta=x_0\)
  • assume \(y\sim x_T\):
    \(\quad\bar{x}_{\mathrm{ML}}=\mathrm{argmax}_\theta\mathcal{L}(\theta)\)

 

Generalization of Euclidean statistical notions and techniques.

  • i.i.d. samples \(y_1,\ldots,y_N\in M\)
  • Fréchet mean:
    \(\bar{x}=\mathrm{argmin}_{x\in M}\sum_{i=1}^Nd(x,y_i)^2\)

The Fréchet mean is not an expected value.

There is no equivalence between different characterizations of means

- in contrast to Euclidean statistics

Geometric statistics

Sommer,IPMI'15; Sommer,Svane,JGM'15;
Hansen,Eltzner,Huckemann,Sommer,GSI'21,'21

Diffusion mean on \(\mathbb S^2\)

  • \(x_t\in M\) Brownian motion
  • \(\theta=x_0\), \(y\sim x_T\)
  • \(\bar{x}_{\mathrm{ML}}=\mathrm{argmax}_\theta\mathcal{L}(\theta)\)
dx_t= -\frac12g(x_t)^{kl}\Gamma(x_t)_{kl}dt + \sqrt{g(x_t)^*}dW_t

Brownian motion starting point

Approximating \(p_T(v)\): Stochastic matching

bridge sampling

dx_t = b(t,x_t)dt +\sigma(t,x_t)dW_t

Delyon/Hu 2006:

\(\sigma\) invertible:

  • guided bridge proposal$$dy_t = b(t,y_t)dt - \frac{y_t-v}{T-t}dt + \sigma(t,y_t)dW_t$$
  • \(y_T=v\) a.s.
  • \(x_t|x_T=v\) absolute continuous wrt. \(y_t\)
  • \(\mathbb E_{x_t|x_T=v}[f(x_t)]\propto \mathbb E_{y_t}[f(y_t)\varphi(y_t)]\)

\(v\)

\(x_0\)

\(x_t\)

Højgaard,Sommer,in preparation

Højgaard,Sommer,2021,arXiv:2105.13190

Simulation of Conditioned Semimartingales on Riemannian Manifolds

From means to covariance
\(SO(3)\) and \(\mathbb S^2\)

  • \(A\) quadratic form on \(so(3)\)
  • \(x_t\in SO(3)\) Brownian motion
  • \(\theta=(x_0,A)\)
  • \((\bar{x}_{\mathrm{ML}},\bar{A})=\mathrm{argmax}_\theta\mathcal{L}(\theta)\)

\(\pi\)

Sommer,Joshi,Højgaard, in preparation

Beyond the mean: PCA

Non-Euclidean generalizations of PCA:

  • Principal Geodesic Analysis (PGA, Fletcher et al., ’04)
  • Geodesic PCA (GPCA, Huckeman et al., ’10)
  • Horizontal Component Analysis (HCA, Sommer, ’13)
  • Principal Nested Spheres ((C)PNS, Jung et al., ’12)
  • Barycentric Subspaces (BS, Pennec, ’15)

Covariance and principal components

Infinitesimal Probabilistic principal components (PPCA)

  • map of Euclidean Brownian motion \(W_t\) to \(M\) via the frame bundle \(FM\): \(\qquad\qquad du_t=H_i(u_t)\circ dW_t\)
  • \(\theta=u_0\), \(u_0=(x,\sigma)\in FM\)
    \(x\in M\) mean, \(\Sigma=\sigma\sigma^T\) (infinitesimal) covariance
  • \((\bar{x}_{\mathrm{ML}},\bar{\sigma})=\mathrm{argmax}_\theta\mathcal{L}(\theta)\)

Frame bundles

  • \(\pi:FM\to M\) is the bundle of linear frames, i.e.
    \(u=(u_1,\ldots,u_d)\in FM\) is an ordered basis for \(T_xM\), \(x=\pi(u)\)
  • \(FM\) is a \(\mathrm{GL}(n)\) principal bundle: \(u:\mathbb R^d\to T_x M\) linear, invertible
  • \(OM\) the subbundle of orthonormal frames (orientations)
  • horizontal lift \(h_u:T_xM\to H_uFM\) and fields: \(H_i:FM\to TFM\)

          \(h_u=(\pi_*|_{H_uFM})^{-1}\)
          \(H_i(u)=h_u(ue_i)\)

Transporting along all paths

Stochastic development:

    \(dU_t=\sum_{i=1}^d H_i\circ_{\mathcal S} dW_t^i\)

\(W_t\) Euclidean Brownian motion

\(X_t=\pi(U_t)\) Riemannian Brownian motion

\(X_t\) supports stochastic parallel transport

Fix \(T>0\): \(U_T\) probability distribution in \(FM\)

... as opposed to geodesics only

Need measure on path space  \(W ([0, T ], M )\)

Covariance transport

du_t=H_i(u_t)\circ dW_t^i
u_t\in FM

Most probable paths

  • in \(\mathbb R^d\), straight lines are most probable for stationary (isotropic) diffusion processes
  • Onsager-Machlup functional, \(\gamma_t\in M\):\[L_M(\gamma(t),\dot{\gamma}(t))=-\frac{1}{2}\|\dot{\gamma}(t)\|^2_g+\frac{1}{12}S(\gamma(t))\]
  • most probable paths maximize \(L_M(\gamma)\):\[\int_0^1 L_M(\gamma(t),\dot{\gamma}(t))dt = -E(\gamma)+ \frac{1}{12}\int_0^1S(\gamma(t))dt\]

 

sub-Riemannian geodesics

Curvature and covariance

Sommer,Svane,JGM'15; Sommer,Entropy,'16

sub-Riemannian structure

Hörmander condition

Smooth density

Small time asymptotics

 

MPPs revisited

Grong, Sommer, in preparation, 2021

Grong, 2021

code: http://bitbucket.com/stefansommer/theanogeometry

           http://bitbucket.com/stefansommer/jaxgeometry

slides: https://slides.com/stefansommer

References:

  • Hansen, Eltzner, Huckemann, Sommer: Diffusion Means in Geometric Spaces, 2021, arXiv:2105.12061.
  • Hansen, Eltzner, Sommer: Diffusion Means and Heat Kernel on Manifolds, 2021, arXiv:2103.00588.
  • Højgaard Jensen, Sommer: Simulation of Conditioned Diffusions on Riemannian Manifolds, 2021, arXiv:2105.13190.
  • Højgaard, Joshi, Sommer: Brownian Bridge Simulation and Metric Estimation on Lie Groups, arXiv:2106.03431, 2021.
  • Arnaudon, v.d. Meulen, Schauer, Sommer: Diffusion bridges for stochastic Hamiltonian systems and shape evolutions, 2021, arXiv:2002.00885.
  • Sommer, Bronstein: Horizontal Flows and Manifold Stochastics in Geometric Deep Learning, TPAMI, 2020, doi: 10.1109/TPAMI.2020.2994507
  • Arnaudon, Holm, Sommer: A Geometric Framework for Stochastic Shape Analysis, Foundations of Computational Mathematics, 2019, arXiv:1703.09971.
  • Højgaard Jensen, Mallasto, Sommer: Simulation of Conditioned Diffusions on the Flat Torus, GSI 2019., arXiv:1906.09813.
  • Sommer, Svane: Modelling Anisotropic Covariance using Stochastic Development and Sub-Riemannian Frame Bundle Geometry, JoGM, 2017, arXiv:1512.08544.
  • Sommer: Anisotropically Weighted and Nonholonomically Constrained Evolutions, Entropy, 2017, arXiv:1609.00395 .
  • Sommer, Svane: Modelling Anisotropic Covariance using Stochastic Development and Sub-Riemannian Frame Bundle Geometry, JoGM, 2017, arXiv:1512.08544.
  • Sommer: Anisotropically Weighted and Nonholonomically Constrained Evolutions, Entropy, 2017, arXiv:1609.00395 .
  • Arnaudon, Holm, Sommer: A Stochastic Large Deformation Model for Computational Anatomy, IPMI 2017, arXiv:1612.05323.
  • Sommer: Anisotropic Distributions on Manifolds: Template Estimation and Most Probable Paths, IPMI 2015, doi: 10.1007/978-3-319-19992-4_15.

Sub-Riemannian geometry and geometric statistics

  • smooth, global support
  • Composition:
    \(k_2\ast_{W_{T/2}} (k_1\ast_{W_{T/2}} f)(u)=\)
        \(\mathrm{E}[k_2(-W_{T/2})k_1(-(W_T-W_{T/2}))f(U_T^u)]\)
  • Tensors: \(k^n_m\), \(f:OM\to\mathbb R^m\)
         \(y^n=\mathrm{E}[k^n(-W_t)f(U_T^u)]\)
  • Equivariance: \(a\in O(d)\)
        \(k\ast_{W_T} (a.f)(u)=a.(k\ast_{W_T} f)(u)\)
  • Non-linearities \(\phi_i\):
        \(\phi_n(k_n\ast_{W_{T/n}} \phi_{n-1}(\cdots \phi_1(k_1\ast_{W_{T/n}} f))(u)\)
  • precomputed density \(\rho\)

Geometry and deep learning:

Intrinsic convolutions

Sommer,Bronstein,TPAMI'20