Horizontal Flows, Manifold Stochastics and
Geometric Deep Learning
Faculty of Science, University of Copenhagen
Stefan Sommer and Alex Bronstein
Department of Computer Science, University of Copenhagen
Technion – Israel Institute of Technology
Curvature, convolutions and translation invariance
Geometric deep learning
deep learning with manifold target or manifold domain (Bronstein'17)
Objectives:
- relate curvature to the obstruction of translation invariance when parallel transporting kernels
- define a convolution operator that naturally incorporates curvature
- define efficient manifold-valued convolution operator, generalizing Fréchet mean based operators
- exemplify applications of stochastics and bundle theory in geometric deep learning
Sommer, Bronstein: Horizontal Flows and Manifold Stochastics in Geometric Deep Learning. TPAMI, 2020, doi: 10.1109/TPAMI.2020.2994507
Convolutions on manifolds
- Euclidean: \(k\ast f(\mathbf x)=\int_{\mathbb R^d}k(-\mathbf v)f(\mathbf x+\mathbf v)d\mathbf v\)
- convolution in (pseudo-)coordinates, patch operator (Monti'16,Masci'15,Boscaini'16)
- is it possible to translate kernels? Can we remove the reliance on charts?
Holonomy obstructs translation invariance
- directional functions: \(f:M\times TM\to\mathbb R\) (Poulenard'18)
\[k\ast f(x,v)=\int_{T_xM}k_w( v)f(\overline{\mathrm{Exp}}_x(v))dv\] - gauge equivariant convolution: (Taco'19)
\[k\ast f(x)=\int_{\mathbb R^d} k(\mathbf v)\rho_{x\leftarrow\mathrm{Exp}_x(u_x\mathbf v)}f(\mathrm{Exp}_x(u_x\mathbf v))d\mathbf v \] - Holonomy: parallel transport path dependent
- rotation invariant kernels, orientation from e.g. curvature
Curvature and convolutions
Directional functions: \(f:OM\to\mathbb R\)
\[k\ast f(u)=\int_{\R^d}k(-\mathbf v)f(P_{\gamma(x,u\mathbf v)}(u))d\mathbf v\]
\(k_1,k_2\) kernels with \(\mathrm{supp}(k_i)\subseteq B_r(0)\), and \(f\in C^3(OM,\mathbb R)\)
Riemannian curvature: \(R(v,u)=-\mathcal{C}([h_u(v),h_u(w)])\)
Theorem:
Non-commutativity:
\(k_2\ast (k_1\ast f)-k_1\ast (k_2\ast f) =\)
\(\int_{\mathbb R^d} \int_{\mathbb R^d} k_2(-\mathbf v_2)k_1(-\mathbf v_1) [h_u(u\mathbf v_2),h_u(u\mathbf v_1)]f d\mathbf v_1 d\mathbf v_2 + o(r^{d+1})\)
+ non-associtativity
Frame bundles
- \(\pi:FM\to M\) is the bundle of linear frames, i.e.
\(u=(u_1,\ldots,u_d)\in FM\) is an ordered basis for \(T_xM\), \(x=\pi(u)\) - \(FM\) is a \(\mathrm{GL}(n)\) principal bundle: \(u:\mathbb R^d\to T_x M\) linear, invertible
- \(OM\) the subbundle of orthonormal frames (orientations)
- horizontal lift \(h_u:T_xM\to H_uFM\) and fields: \(H_i:FM\to TFM\)
\(h_u=(\pi_*|_{H_uFM})^{-1}\)
\(H_i(u)=h_u(ue_i)\)
\[k\ast f(u)=\int_{\R^d}k(-\mathbf v)f(P_{\gamma(x,u\mathbf v)}(u))d\mathbf v\]
Non-commutativity:
\(k_2\ast (k_1\ast f)-k_1\ast (k_2\ast f) =\)
\(\int_{\mathbb R^d} \int_{\mathbb R^d} k_2(-\mathbf v_2)k_1(-\mathbf v_1) [h_u(u\mathbf v_2),h_u(u\mathbf v_1)]f d\mathbf v_1 d\mathbf v_2 + o(r^{d+1})\)
Transporting along all paths
Stochastic development:
\(dU_t=\sum_{i=1}^d H_i\circ_{\mathcal S} dW_t^i\)
\(W_t\) Euclidean Brownian motion
\(X_t=\pi(U_t)\) Riemannian Brownian motion
\(X_t\) supports stochastic parallel transport
Fix \(T>0\): \(U_T\) probability distribution in \(FM\)
... as opposed to geodesics only
Need measure on path space \(W ([0, T ], M )\)
New convolution operator
\(k\ast_{W_T} f(u)=\int k(-W_T)f(U_T^u)\mathbb P(dW_t)=\mathrm{E}[k(-W_T)f(U_T^u)]\)
- orientation function:
\(f:OM\to\mathbb R\) - kernel: \(k:\mathbb R^d\to\mathbb R\)
- \(W_t\) Euclidean Brown. motion
- \(U_t\) \(OM\)-development of \(W_t\)
- link between \(W_t\) and \(U_t\):
stochastic development
Properties and Algorithm
- smooth, global support
- Composition:
\(k_2\ast_{W_{T/2}} (k_1\ast_{W_{T/2}} f)(u)=\)
\(\mathrm{E}[k_2(-W_{T/2})k_1(-(W_T-W_{T/2}))f(U_T^u)]\) - Tensors: \(k^n_m\), \(f:OM\to\mathbb R^m\)
\(y^n=\mathrm{E}[k^n(-W_t)f(U_T^u)]\) - Equivariance: \(a\in O(d)\)
\(k\ast_{W_T} (a.f)(u)=a.(k\ast_{W_T} f)(u)\) - Non-linearities \(\phi_i\):
\(\phi_n(k_n\ast_{W_{T/n}} \phi_{n-1}(\cdots \phi_1(k_1\ast_{W_{T/n}} f))(u)\) - precomputed density \(\rho\)
Sampling means
Manifold target:
Euclidean convolution:
\[k\ast f(\mathbf x)=\argmin_{\mathbf y\in\mathbb R}\mathrm E[k(\mathbf x-\mathbf z)\|\mathbf y-f(\mathbf z)\|^2]\]
conv. from weighted Fréchet mean: (Pennec'06/Chakraborty'19/'20)
\[k\ast f(x)=\argmin_{y\in M}\mathrm E[k(x,z)d(y,f(z))^2)]\]
kernel: \(k:M\times M\to\mathbb R\), \(\mathrm E[k(x,\cdot)]=1\)
Bridge sampling to diagonal
- diffusion mean (Hansen'21) weighted:
\[k\ast f(x)=\argmax_{y\in M}\mathrm E[\log p_{T/k(x,z)}(f(z);y)]\] - \(p_T(x,\cdot)\) density of Brownian motion at \(T>0\)
- \(p_T(x,v)\) and \(k\ast f(x)\) can be approximated by bridge sampling:
bridge sampling: target \(v\)
bridge sampling: target \(\mathrm{diag}(M^2)\)
Algorithm
Stochasticity analogous to deep Gaussian process NNs (Gal'16): \[\mathbf y|\mathbf x,w=\mathcal{N}(\hat{y}(\mathbf x,w),\tau^{-1})\]
Horizontal Flows, Manifold Stochastics
- curvature obstructs translation invariance
- measures on paths spaces integrates this out
- manifold-valued operators can be sampled
- bundle theory and stochastics can inspire deep learning algorithms
code: http://bitbucket.com/stefansommer/jaxgeometry
slides: https://slides.com/stefansommer
References:
- Sommer, Bronstein: Horizontal Flows and Manifold Stochastics in Geometric Deep Learning, TPAMI, 2020, doi: 10.1109/TPAMI.2020.2994507
- Hansen, Eltzner, Huckemann, Sommer: Diffusion Means on Riemannian Manifolds, in preparation, 2020.
- Højgaard Jensen, Sommer: Simulation of Conditioned Diffusions on Riemannian Manifolds, in preparation, 2020.
- Sommer: Anisotropic Distributions on Manifolds: Template Estimation and Most Probable Paths, IPMI 2015, doi: 10.1007/978-3-319-19992-4_15.
- Sommer, Svane: Modelling Anisotropic Covariance using Stochastic Development and Sub-Riemannian Frame Bundle Geometry, JoGM, 2017, arXiv:1512.08544.
- Sommer: Anisotropically Weighted and Nonholonomically Constrained Evolutions, Entropy, 2017, arXiv:1609.00395 .
