Klas Modin PRO
Mathematician at Chalmers University of Technology and the University of Gothenburg
Information Geometry
and Diffeomorphisms
Klas Modin
Collaborators
Sarang Joshi
Martin Bauer
Boris Khesin
Gerard Misiolek
Geometric hydrodynamics
Riemannian geometry
of diffeomorphisms
Information geometry
Riemannian geometry
of statistics
Arnold (1966)
Rao (1945), Amari (1968)
?
(Topic of the talk)
Overview
-
Pre-Riemannian geometry: relation between probability densities and diffeomorphisms
-
Geometry of optimal mass transport (OMT)
-
Wasserstein vs. Fisher-Rao
-
Optimal information transport (OIT)
-
Application: random sampling
- Finite dimensional analogue (of OIT)
The two spaces
Probability densities
Prob(M)={μ∈Ωn(M)∣μ>0,∫Mμ=1}
Diffeomorphisms
Diff(M)={φ∈C∞(M,M)∣smooth φ−1}
M compact (Riemannian) manifold
The centerpiece:
Moser's principal bundle
Diff(M)
\mathrm{Diff}(M)
Diffμ0(M)Diff(M)≃Prob(M)
\frac{\mathrm{Diff}(M)}{\mathrm{Diff}_{\mu_0}(M)}\simeq\mathrm{Prob}(M)
Id
\mathrm{Id}
μ0
\mu_0
μ1
\mu_1
π(φ)
\pi(\varphi)
Hor
\mathrm{Hor}
Two versions:
π(φ)=φ∗μ0 (left action)
π(φ)=φ∗μ0 (right action)
Relevant in optimal mass transport
Relevant in information geometry
Optimal mass transport (OMT)
μ0
\mu_0
μ1
\mu_1
φ∗μ0
\varphi_*\mu_0
φ∗μ0=μ1min∫MdM2(φ(x),x)μ0
\displaystyle\min_{\varphi*\mu_0=\mu_1} \int_{M} d_M^2(\varphi(x),x ) \mu_0
M
M
Monge problem, L2 version
Wasserstein distance
dW2(μ0,μ1)=φ∗μ0=μ1inf∫MdM2(φ(x),x)μ0
\displaystyle d_W^2(\mu_0,\mu_1) = \inf_{\varphi_*\mu_0=\mu_1} \int_{M} d_M^2(\varphi(x),x) \mu_0
Symmetric by change of variables
Riemannian structure of OMT
Diff(M)
\mathrm{Diff}(M)
Prob(M)
\mathrm{Prob}(M)
Id
\mathrm{Id}
μ0
\mu_0
μ1
\mu_1
π(η)=η∗μ0
\pi(\eta)=\eta_*\mu_0
Riemannian metric
Gφ(φ˙,φ˙)=∫M∣φ˙∣2μ0
\displaystyle\mathcal{G}_\varphi(\dot\varphi,\dot\varphi) = \int_{M}\left\vert \dot\varphi \right\vert^2 \mu_0
Induces metric
Gμ(μ˙,μ˙)⇒dW2(μ0,μ1)
\overline{\mathcal{G}}_\mu(\dot\mu,\dot\mu) \Rightarrow d_W^2(\mu_0,\mu_1)
Hor
\mathrm{Hor}
[Benamou & Brenier (2000), Otto (2001)]
Invariance: η∈Diffμ0(M)
Gφ(φ˙,φ˙)=Gφ∘η(φ˙∘η,φ˙∘η)
\displaystyle\mathcal{G}_\varphi(\dot\varphi,\dot\varphi) = \mathcal{G}_{\varphi\circ\eta}(\dot\varphi\circ\eta,\dot\varphi\circ\eta)
Exactly L2-Wasserstein distance
Wasserstein vs. Fisher-Rao
Prob(M)
\mathrm{Prob}(M)
TμProb(M)≃C0∞(M)
T_\mu\mathrm{Prob}(M)\simeq C^\infty_0(M)
Wasserstein
Fisher-Rao
Gμ(μ˙,μ˙)=∫Mμμ˙μμ˙μ
\displaystyle\overline{\mathcal{G}}_\mu(\dot\mu,\dot\mu) = \int_{M} \frac{\dot\mu}{\mu}\frac{\dot\mu}{\mu}\mu
Gρμ0(ρ˙μ0,ρ˙μ0)=∫M∣∇θ∣2ρ
\displaystyle\overline{\mathcal{G}}_{\rho\mu_0}(\dot\rho\mu_0,\dot\rho\mu_0) = \int_{M} |\nabla\theta|^2\rho
ρ˙+div(ρ∇θ)=0
\displaystyle \dot\rho + \mathrm{div}(\rho \nabla\theta) = 0
Dependent on Riemannian structure of M
Independent of Riemannian structure of M⇒Diff(M)-invariance
Degenerate Diff(M)-metric compatible with Fisher-Rao
[Khesin, Lenells, Misiolek, Preston, 2013]
Diff(M)
\mathrm{Diff}(M)
Prob(M)
\mathrm{Prob}(M)
id
\mathrm{id}
μ0
\mu_0
π(η)=φ∗μ0
\pi(\eta)=\varphi^*\mu_0
Hor ?
\mathrm{Hor}\text{ ?}
H˙ degenerate metric
Gid(v,v)=∫Mdiv(v)2μ0
\displaystyle\mathcal{G}_\mathrm{id}(v,v) = \int_{M}\mathrm{div}(v)^2 \mu_0
=Gˉμ0(Tidπv,Tidπv)
\displaystyle=\bar{\mathcal{G}}_\mathrm{\mu_0}(T_{\mathrm{id}}\pi \,v,T_{\mathrm{id}}\pi\, v)
Wanted: non-degenerate descending metric
Optimal information transport
[M., 2015]
Natural idea: Hodge decomposition for horizontal directions
Gid(v,v)=∥divv∥L22+∥ξ∥L22v=∇f+ξ
\displaystyle\mathcal{G}_\mathrm{id}(v,v) = \|\operatorname{div} v\|^2_{L^2} + \|\xi\|^2_{L^2} \qquad v = \nabla f + \xi
Gid(v,v)=∥divv∥L22+∥dv♭∥L22+∥h∥L22v=∇f+(δα)♯+h
\displaystyle\mathcal{G}_\mathrm{id}(v,v) = \|\operatorname{div}v\|^2_{L^2} + \|dv^\flat\|^2_{L^2} + \|h\|^2_{L^2} \quad v = \nabla f + (\delta\alpha)^\sharp + h
Diff(M)
\mathrm{Diff}(M)
Prob(M)
\mathrm{Prob}(M)
id
\mathrm{id}
μ0
\mu_0
π(η)=φ∗μ0
\pi(\eta)=\varphi^*\mu_0
Hor
\mathrm{Hor}
Theorem: geodesics are locally well-posed
μ1
\mu_1
Theorem:
Any φ∈Diffs(M) admits unique factorization φ=η∘Expid(∇f)
solves OIT problem
Horizontal lifting equations
Theorem: solution to optimal information transport is φ(1) where φ(t) fulfills
where μ(t) is Fisher-Rao geodesic between μ0 and μ1
Δf(t)=μ(t)μ˙(t)∘φ(t)
\displaystyle
\Delta f(t) = \frac{\dot\mu(t)}{\mu(t)}\circ\varphi(t)
v(t)=∇f(t)
\displaystyle
v(t) = \nabla f(t)
dtdφ(t)−1=v(t)∘φ(t)−1
\displaystyle
\frac{d}{dt}\varphi(t)^{-1} = v(t)\circ\varphi(t)^{-1}
Leads to numerical time-stepping scheme: Poisson problem at each time step
MATLAB code: github.com/kmodin/oit-random
Application: non-uniform sampling on M
Problem 1: given μ1∈Prob(M) generate N samples from μ1
Most cases: use Monte-Carlo based methods
Special case here:
- M low dimensional
- μ very non-uniform
- N very large
transport map approach
might be useful
[Bauer, Joshi, M., 2017]
Transport problem
Problem 1': given μ1∈Prob(M) find φ∈Diff(M) such that
Method:
- N samples x1,…,xN from uniform distribution μ0
- Compute yi=φ(xi)
Diffeomorphism φ not unique!
φ∗μ0=μ1
\varphi_*\mu_0 = \mu_1
Optimal transport problem
Problem 1'': given μ1∈Prob(M) find φ∈Diff(M) minimizing
under constraint φ∗μ0=μ1
Studied case: (Moselhy and Marzouk 2012, Reich 2013, ...)
- dist = L2-Wasserstein distance
- ⇒ optimal mass transport problem
- ⇒ solve Monge-Ampere equation (heavily non-linear PDE)
E(φ)=dist(id,φ)2
E(\varphi) = \mathrm{dist}(\mathrm{id},\varphi)^2
Our notion:
- use optimal information transport
Simple 2D example
Warp computation time (256*256 gridsize, 100 time-steps): ~1s
Sample computation time (10^7 samples): < 1s
OIT in finite dim: manifold of inverse covariance matrices
P(n)={W∈Rn×n∣W=W⊤,W>0}
P(n) = \{ W\in \mathbb{R}^{n\times n}\mid W=W^\top, W>0 \}
ρ(x;W−1)=(2π)n∣W∣exp(−21x⊤Wx)
\displaystyle \rho(x;W^{-1}) = \sqrt{\frac{|W|}{(2\pi)^n}}\exp(-\frac{1}{2}x^\top W x)
[M., 2017]
Fisher-Rao metric on P(n)
TWP(n)={U∈Rn×n∣U=U⊤}
T_W P(n) = \{ U\in \mathbb{R}^{n\times n}\mid U=U^\top \}
U
U
W
W
gW(U,U)=21tr(W−1UW−1U)
g_W(U,U) = \frac{1}{2}\mathrm{tr}(W^{-1}UW^{-1}U)
Geodesics on P(n)
W¨−W˙W−1W˙=0
\ddot W - \dot W W^{-1}\dot W = 0
Explicit distance function
d(W0,W1)2=21tr(log(W1W0−1)log(W1W0−1))
\displaystyle d(W_0,W_1)^2 = \frac{1}{2}\mathrm{tr}\big(\log(W_1W_0^{-1})\log(W_1W_0^{-1}) \big)
Geodesic equation
W0
W_0
W1
W_1
Homogeneous space structure
I
I
fiber
π
\pi
fiber
I
I
W1
W_1
P(n)
P(n)
A
A
Q
Q
GL(n)
\mathrm{GL}(n)
O(n)\GL(n)={[A]∣A∈GL(n),[A]=O(n)⋅A}
\mathrm{O}(n)\backslash \mathrm{GL}(n) = \{ [A] \mid A\in\mathrm{GL}(n), [A]=\mathrm{O}(n)\cdot A \}
O(n)\GL(n)≃P(n)byπ:A↦A⊤A
\mathrm{O}(n)\backslash \mathrm{GL}(n) \simeq P(n) \quad\text{by}\quad \pi\colon A\mapsto A^\top A
Principal bundle
Fisher-Rao invariance
U
U
W
W
gW(U,U)=21tr(W−1UW−1U)
g_W(U,U) = \frac{1}{2}\mathrm{tr}(W^{-1}UW^{-1}U)
GL(n)×P(n)∋(A,W)↦A⊤WA∈P(n)
\mathrm{GL}(n)\times P(n) \ni (A,W) \mapsto A^\top W A \in P(n)
Right action of GL(n) on P(n)
gA⊤WA(A⊤UA,A⊤UA)=gW(U,U)
g_{A^\top W A}(A^\top U A,A^\top U A) = g_W(U,U)
Compatible metric on GL(n)
gˉA(V,V)=21tr(ℓ(VA−1)⊤ℓ(VA−1)+σ(VA−1)σ(VA−1))
\displaystyle \bar g_A(V,V) = \frac{1}{2}\mathrm{tr}\big(\ell(VA^{-1})^\top\ell(VA^{-1})+\sigma(VA^{-1})\sigma(VA^{-1}) \big)
V
V
A
A
horizontal slice
I
I
R
R
fiber
π
\pi
fiber
I
I
W1
W_1
K
K
P(n)
P(n)
A
A
Q
Q
Horizontal distribution
HorA={V∈TAGL(n)∣ℓ(VA−1)=0}
\mathrm{Hor}_A = \{ V\in T_A\mathrm{GL}(n) \mid \ell(VA^{-1}) = 0 \}
K={R∈GL(n)∣ℓ(R)=0,Rii>0}⇒TRK=HorR
K = \{ R\in \mathrm{GL}(n)\mid \ell(R)=0, R_{ii}>0 \} \Rightarrow T_R K = \mathrm{Hor}_R
horizontal slice
I
I
R
R
fiber
π
\pi
fiber
I
I
W1
W_1
K
K
P(n)
P(n)
A
A
Q
Q
Gives QR and Cholesky factorizations of matrices
THANKS!
References:
- K. Modin
Generalized Hunter–Saxton equations, optimal information transport, and factorization of diffeomorphisms, 2015 - M. Bauer, S. Joshi, K. Modin
Diffeomorphic random sampling using optimal information transport, 2017 - K. Modin
Geometry of Matrix Decompositions Seen Through Optimal Transport and Information Geometry, 2017
Slides available at: slides.com/kmodin
Information Geometry and Diffeomorphisms Klas Modin
Information Geometry and Diffeomorphisms
By Klas Modin
Information Geometry and Diffeomorphisms
Presentation given 2019-10 in Toulouse.
- 2,192
