Théo Dumont
PhD student in optimal transport & geometry @ Université Gustave Eiffel
A geometric view of optimal transport
μ0
\mu_0
μ1
\mu_1
Théo Dumont
Dens(Rn)
\operatorname{Dens}(\mathbb R^n)
slides available at https://slides.com/theodumont/geometric-ot
OT(μ0,μ1)=φinf∫Rn∥x−φ(x)∥2dμ0over φ:Rn→Rn such that φ∗μ0=μ1.
\displaystyle \text{OT}(\mu_0,\mu_1)= \inf_{\varphi} \int_{\mathbb R^n} \|x-\varphi(x)\|^2\,\mathrm d\mu_0\qquad\text{over }\varphi:\mathbb R^n\to\mathbb R^n\text{ such that } \varphi_*\mu_0=\mu_1.
μ0,μ1∈P(Rn)
\mu_0,\mu_1\in\mathcal P(\mathbb R^n)
μ0
\mu_0
μ1
\mu_1
OTK(μ0,μ1)=πinf∫Rn×Rn∥x−y∥2dπover π∈P(Rn×Rn) such that P∗iπ=μi.
\displaystyle \text{OT}_K(\mu_0,\mu_1)= \inf_{\pi} \int_{\mathbb R^n\times\mathbb R^n} \|x-y\|^2\,\mathrm d\pi\quad\text{over }\pi\in\mathcal P(\mathbb R^n\times \mathbb R^n)\text{ such that } P^i_*\pi=\mu_i.
Definition. (OT Kantorovitch problem)
δx
\delta_x
δy1
\delta_{y_1}
δy2
\delta_{y_2}
not feasible by a map!
π is induced by a transport map φ
π is a transport plan
relaxation
μ0
\mu_0
μ1
\mu_1
π
\pi
μ0
\mu_0
μ1
\mu_1
π
\pi
∣detdφ−1∣μ0∘φ−1
[Monge, 1781], [Kantorovitch, 1942]
Optimal Transport
Definition. (OT Monge problem)
φ
\varphi
Can we say that the solution of (K) is a map?
?
If μ0 has a density, then there is a unique solution to (K), and it is of the form φ=∇f with f:Rn→R convex.
Theorem. (Brenier)
relaxation
π is induced by a transport map φ
π is a transport plan
Monge (maps)
Kantorovitch (plans)
μ0
\mu_0
μ1
\mu_1
π
\pi
μ0
\mu_0
μ1
\mu_1
π
\pi
[Brenier, 1987]
Optimal Transport
Dens(Rn):={μ(x)dx∣μ∈C∞(Rn),∫μ=1}
\operatorname{Dens}(\mathbb R^n)\coloneqq\{\mu(x)\mathrm dx\mid\mu\in C^{\infty}(\mathbb R^n),\int\mu=1\}
μ0,μ1∈Dens(Rn)
\mu_0,\mu_1\in\operatorname{Dens}(\mathbb R^n)
μ0
\mu_0
μ1
\mu_1
Smooth densities:
can we recover classical results of OT theory with a geometric picture?
?
:=C(μ0,μ1)
\coloneqq\mathcal C(\mu_0,\mu_1)
OT(μ0,μ1)=φmin∫Rn∥x−φ(x)∥2dμ0over φ∈Diff(Rn) such that φ∗μ0=μ1
\displaystyle \text{OT}(\mu_0,\mu_1)= \min_{\varphi} \int_{\mathbb R^n} \|x-\varphi(x)\|^2\,\mathrm d\mu_0\qquad\text{over }\varphi\in\operatorname{Diff}(\mathbb R^n)\text{ such that } \varphi_*\mu_0=\mu_1
Definition. (Smooth OT problem)
Smooth OT
Diff(Rn):={φ:Rn→Rn∣φ,φ−1∈C∞(Rn)}
\operatorname{Diff}(\mathbb R^n)\coloneqq\{\varphi:\mathbb R^n\to\mathbb R^n\mid \varphi,\varphi^{-1}\in C^\infty(\mathbb R^n)\}
Smooth maps:
φ
\varphi
C(μ0,μ1)
\mathcal C(\mu_0,\mu_1)
not right-invariant! only by action of Diffμ0(Rn)
!
Diff(Rn)
\operatorname{Diff}(\mathbb R^n)
φ
\varphi
id
\operatorname{id}
Theorem. (Geodesic equation on Diff(Rn))
{φ¨t=0φt=(1−t)φ0+tφ1
\begin{cases}
\ddot\varphi_t=0\\
\varphi_t=(1-t)\varphi_0+t\varphi_1
\end{cases}
or if we write φ˙=v∘φ,
inviscid Burgers v˙+∇vv=0
d2(φ0,φ1)=∫01gφt(φ˙t,φ˙t)dt
\displaystyle d^2(\varphi_0,\varphi_1)=\int_0^1g_{\varphi_t}(\dot\varphi_t,\dot\varphi_t)\,\mathrm dt
=∫01∫Rd∥φ1−φ0∥2dμ0dt
\displaystyle =\int_0^1\int_{\mathbb R^d}\|\varphi_1-\varphi_0\|^2\,\mathrm d\mu_0\,\mathrm dt
=∫Rd∥φ1−φ0∥2dμ0
\displaystyle =\int_{\mathbb R^d}\|\varphi_1-\varphi_0\|^2\,\mathrm d\mu_0
horizontal?
finding the shortest geodesic from id to C(μ0,μ1)
OT(μ0,μ1)=φ∈C(μ0,μ1)mind2(id,φ)
A first link with OT
TφDiff(Rn)={v∘φ∣v∈X(Rn)}
T_\varphi\operatorname{Diff}(\mathbb R^n)=\{v\circ\varphi\mid v\in \mathfrak X(\mathbb R^n)\}
φ˙
\dot\varphi
- Tangent space at φ:
Gφ(φ˙,φ˙):=∫Rd∥φ˙∥2dμ0=∫Rd∥v∥2dμ
\displaystyle G_\varphi(\dot\varphi,\dot\varphi)\coloneqq\int_{\mathbb R^d}\|\dot\varphi\|^2\,\mathrm d\mu_0 =\int_{\mathbb R^d}\|v\|^2\,\mathrm d\mu
- Metric at φ:
φ∗μ0
\varphi_*\mu_0
[Otto, 2001], [Kriegl & Michor, 1997], [Modin, 2015]
Diffeomorphism group
id
\operatorname{id}
φ
\varphi
Diffμ0(Rn)
\operatorname{Diff}_{\mu_0}(\mathbb R^n)
C(μ0,μ1)
\mathcal C(\mu_0,\mu_1)
Diff(Rn)
\operatorname{Diff}(\mathbb R^n)
π:Diff(Rn)⟶Dens(Rn)
φ⟼φ∗μ0
μ0
\mu_0
μ1
\mu_1
π:φ↦φ∗μ0
\pi:\varphi\mapsto\varphi_*\mu_0
Dens(Rn)
\operatorname{Dens}(\mathbb R^n)
Fiber over μ1:
{φ∣φ∗μ0=μ1}=C(μ0,μ1)
Fiber over μ0:
{φ∣φ∗μ0=μ0}=Diffμ0(Rn)
μ0∈Dens(Rn)
\mu_0\in\operatorname{Dens}(\mathbb R^n)
dπ(φ):TφDiff(Rn)⟶TμDens(Rn)
v∘φ⟼−div(μv)
φ∗μ0
\varphi_*\mu_0
The submersion
id
\operatorname{id}
φ
\varphi
μ0
\mu_0
μ1
\mu_1
Diffμ0(Rn)
\operatorname{Diff}_{\mu_0}(\mathbb R^n)
π:φ↦φ∗μ0
\pi:\varphi\mapsto\varphi_*\mu_0
π:φ↦φ∗μ0 is a smooth submersion.
C(μ0,μ1)
\mathcal C(\mu_0,\mu_1)
∇p
\nabla p
Diff(Rn)
\operatorname{Diff}(\mathbb R^n)
Dens(Rn)
\operatorname{Dens}(\mathbb R^n)
vertical distribution:
Verφ:=kerdπ(φ)={v∘φ∣div(μv)=0}
\operatorname{Ver}_\varphi\coloneqq\ker d\pi(\varphi)\\\hspace{2.8cm}=\{v\circ\varphi\mid \operatorname{div}(\mu v)=0\}
φ∗μ0
\varphi_*\mu_0
horizontal distribution:
Horφ:=(Verφ)⊥G={∇p∘φ∣p∈C∞(Rn)}
\operatorname{Hor}_\varphi\coloneqq (\operatorname{Ver}_\varphi)^{\perp_G}\\\hspace{3.2cm}=\{\nabla p\circ\varphi\mid p\in C^\infty(\mathbb R^n)\}
Any u∈X(Rn) can be written as
u=v+∇p
with div(μ0v)=0 and p∈C∞(Rn).
TφDiff(Rn)=Verφ⊕⊥Horφ
T_ {\varphi}\operatorname{Diff}(\mathbb R^n)=\operatorname{Ver}_{\varphi}\overset{\perp}\oplus\operatorname{Hor}_{\varphi}
Theorem. (Helmholtz/hodge decomposition)
right-invariance under action of fiber Diffμ0(Rn)
⟹
\implies
π induces a metric on Diff(Rn)/Diffμ0(Rn)≃Dens(Rn)
independent
of G
depends
on G!
The submersion
(smooth submersion)
id
\operatorname{id}
φ
\varphi
μ0
\mu_0
μ1
\mu_1
π:φ↦φ∗μ0
\pi:\varphi\mapsto\varphi_*\mu_0
C(μ0,μ1)
\mathcal C(\mu_0,\mu_1)
μ˙=−div(μ∇p)
\dot\mu=-\operatorname{div}(\mu\nabla p)
Diff(Rn)
\operatorname{Diff}(\mathbb R^n)
Dens(Rn)
\operatorname{Dens}(\mathbb R^n)
∇p
\nabla p
μ˙
\dot\mu
π is a Riemannian submersion
gμ(μ˙,μ˙):=dπ(φ).φ˙=μ˙infGφ(φ˙,φ˙)
\displaystyle g_\mu(\dot\mu,\dot\mu)\coloneqq \inf_{d\pi(\varphi).\dot\varphi=\dot\mu} G_\varphi(\dot\varphi,\dot\varphi)
Pythagoras ⟹φ˙ has to be horizontal!
gμ(μ˙,μ˙)=∫Rn∥∇p∥2dμ
\displaystyle g_\mu(\dot\mu,\dot\mu)=\int_{\mathbb R^n}\|\nabla p\|^2\,\mathrm d\mu
where μ˙ and ∇p are linked by μ˙=−div(μ∇p)
Theorem. (Induced metric on Dens(Rn))
=∫Rn∥∇Δμ−1μ˙∥2dμ
\displaystyle=\int_{\mathbb R^n}\|\nabla \Delta_{\mu}^{-1}\dot\mu\|^2\,\mathrm d\mu
where Δμp=div(μ∇p).
The submersion
(Riemannian submersion)
id
\operatorname{id}
φ1
\varphi_1
μ0
\mu_0
μ1
\mu_1
π
\pi
C(μ0,μ1)
\mathcal C(\mu_0,\mu_1)
∇p
\nabla p
μ˙=−div(μ∇p)
\dot\mu=-\operatorname{div}(\mu\nabla p)
Diff(Rn)
\operatorname{Diff}(\mathbb R^n)
Dens(Rn)
\operatorname{Dens}(\mathbb R^n)
what do the geodesics look like in Dens(Rn)?
?
μ˙
\dot\mu
geodesics in Dens(Rn)
horizontal geodesics in Diff(Rn)
⟺
φt=id+t∇p
\varphi_t=\operatorname{id}+t\nabla p
{μ˙+div(μ∇p)=0p˙+21∥∇p∥2=0
\begin{cases}
\dot\mu+\operatorname{div}(\mu\nabla p)=0\\
\dot p+\frac12 \|\nabla p\|^2=0
\end{cases}
(Hamilton-Jacobi)
(continuity equation)
Theorem. (Geodesic equation on Dens(Rn))
The induced geodesic distance on Dens(Rn) is the OT distance!
(see previous computation d2(id,φ)=OT(μ0,μ1))
at time t=1: φ1=id+∇p=∇f
what's the final map?
?
Brenier's theorem
The submersion
(Riemannian submersion)
id
\operatorname{id}
φ
\varphi
μ0
\mu_0
μ1
\mu_1
π
\pi
C(μ0,μ1)
\mathcal C(\mu_0,\mu_1)
vt
v_t
Diff(Rn)
\operatorname{Diff}(\mathbb R^n)
Dens(Rn)
\operatorname{Dens}(\mathbb R^n)
μ˙t
\dot\mu_t
OT(μ0,μ1)=vt,μtinf∫01∥vt∥L2(μt)2dt
\text{OT}(\mu_0,\mu_1)=\inf_{v_t,\,\mu_t}\int_0^1 \|v_t\|^2_{L^2(\mu_t)}\,\mathrm dt
⟹ this is just finding the curve of minimal energy between μ0 and μ1 in Dens(Rn),
i.e. finding a (horizontal) geodesic!
where μ˙t+div(μtvt)=0, and where μt has the right endpoints.
[Benamou & Brenier, 2000]
Usefulness:
- computational: scale to bigger number of points
- theoretical: propose extensions of the OT framework (unbalanced OT, ...)
Applications
Benamou-Brenier
Theorem. (Benamou-Brenier, dynamic formulation of OT)
id
\operatorname{id}
∇f
\nabla f
Diff(Rn)
\operatorname{Diff}(\mathbb R^n)
Any φ∈Diff(Rn) can be written as
φ=∇f∘ϕ
with f∈C∞(Rn) and ϕ∈Diffμ0(Rn).
ϕ
\phi
φ
\varphi
Diffμ0(Rn)
\operatorname{Diff}_{\mu_0}(\mathbb R^n)
[Brenier, 1987]
Applications
Polar factorization
Theorem. (Polar factorization)
μ
\mu
π
\pi
−∇δμδF
-\nabla \frac{\delta F}{\delta\mu}
Diff(Rn)
\operatorname{Diff}(\mathbb R^n)
Dens(Rn)
\operatorname{Dens}(\mathbb R^n)
μ˙=div(μ∇δμδF(μ)).
\dot\mu=\operatorname{div}\Big(\mu\nabla\frac{\delta F}{\delta\mu}(\mu)\Big).
The gradient flow of μ w.r.t. a functional F is
‘‘x˙=−gradgF(x)"
``\dot x=-\operatorname{grad}^g F(x)"
Fréchet derivative
F(μ)=∫Rnμ(logμ−1)+∫RnVμ
F(\mu)=\int_{\mathbb R^n}\mu(\log\mu-1)+\int_{\mathbb R^n}V\mu
Example:
entropy
potential
δμδF(μ)=logμ+V
\frac{\delta F}{\delta \mu}(\mu)=\log\mu+V
μ˙=Δμ+div(μ∇V)
\dot\mu=\Delta\mu+\operatorname{div}(\mu\nabla V)
+ heat flow
Fokker-Planck equation.
μ˙=−div(μ∇p)
\dot\mu=-\operatorname{div}(\mu\nabla p)
- converges to stationary distribution μ∞=∫e−V1e−V
- + λ-convexity of F along geodesics implies exponential convergence in terms of KL divergence
μ˙
\dot\mu
Applications
Gradient flows
Definition. (Gradient flow)
OT = finding the shortest geodesic from id to constraint set C(μ0,μ1)
horizontal
We recover:
- polar factorization
- Hodge decomposition
id
\operatorname{id}
φ1
\varphi_1
μ0
\mu_0
μ1
\mu_1
π:φ↦φ∗μ0
\pi:\varphi\mapsto\varphi_*\mu_0
C(μ0,μ1)
\mathcal C(\mu_0,\mu_1)
∇p
\nabla p
μ˙=−div(μ∇p)
\dot\mu=-\operatorname{div}(\mu\nabla p)
Diff(Rn)
\operatorname{Diff}(\mathbb R^n)
Dens(Rn)
\operatorname{Dens}(\mathbb R^n)
μ˙
\dot\mu
cf. Benamou-Brenier formulation
Riemannian submersion (Diff(Rn),L2(μ0))⟶π(Dens(Rn),OT)
| Geodesic equation on... | is... |
|---|---|
| inviscid Burgers | |
| incompressible Euler | |
| Hamilton-Jacobi + contin. eqn. |
Diff(Rn)
\operatorname{Diff}(\mathbb R^n)
Diffμ0(Rn)
\operatorname{Diff}_{\mu_0}(\mathbb R^n)
Dens(Rn)
\operatorname{Dens}(\mathbb R^n)
| Gradient flow of... | is... |
|---|---|
| entropy | heat flow |
| entropy + potential | Fokker-Planck |
| loss functional L | training inf. wide NN |
[Chizat & Bach, 2018]
(local)
Recap
| OT | Inform. theory | Unbalanced OT | LDDMM | Metamorphoses | |
|---|---|---|---|---|---|
| top space | |||||
| metric | |||||
| right-invariant? | |||||
| bottom space | anything with an action |
anything with an action | |||
| action | ... | ... | |||
| metric | Wasserstein | Fisher-Rao | Wasserstein-Fisher-Rao | induced metric | induced metric |
Diff
\operatorname{Diff}
Diff
\operatorname{Diff}
Diff⋉C∞
\operatorname{Diff}\ltimes C^{\infty}
Diff⋉C∞
\operatorname{Diff}\ltimes C^{\infty}
Diff
\operatorname{Diff}
Dens
\operatorname{Dens}
Dens
\operatorname{Dens}
Dens+
\operatorname{Dens}^+
L2(μ0)
L^2(\mu_0)
H1(μ0)
H^1(\mu_0)
L2(μ0)
L^2(\mu_0)
HK(dx)
\mathcal H_K(\mathrm dx)
HK(dx)
\mathcal H_K(\mathrm dx)
[Bauer, Bruveris & Michor, 2016], [Gallouët & Vialard, 2018], [Younes, 2010], [Trouvé & Younes, 2005], [Modin, 2015]
finite-dimensional equivalents when restricting to Gaussian measures:
submersion GL(n)→PSD(n),
induces Bures-Wasserstein and Fisher-Rao
φ↦φ∗μ
\varphi\mapsto \varphi_*\mu
φ↦φ∗μ
\varphi\mapsto \varphi^*\mu
(φ,λ)↦φ∗(λ2μ)
(\varphi,\lambda)\mapsto \varphi^*(\lambda^2\mu)
Some other submersions
L. Ambrosio, N. Gigli, and G. Savaré. Gradient flows: in metric spaces and in the space of probability measures, 2005.
J.-D. Benamou and Y. Brenier. A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem, 2000.
M. Bauer, M. Bruveris, and P.W. Michor. Uniqueness of the Fisher–Rao metric on the space of smooth densities, 2016.
Y. Brenier. Décomposition polaire et réarrangement monotone des champs de vecteurs, 1987.
L. Chizat and F. Bach. On the global convergence of gradient descent for over-parameterized models using optimal transport, 2018.
W. Gangbo, H.K. Kim, and T. Pacini. Differential forms on Wasserstein space and infinite-dimensional Hamiltonian systems, 2010.
T. Gallouët and F.-X. Vialard. The Camassa–Holm equation as an incompressible Euler equation: A geometric point of view, 2018.
A. Kriegl and P.W. Michor. The convenient setting of global analysis, 1997.
K. Modin. Geometry of matrix decompositions seen through optimal transport and information geometry, 2016.
F. Otto. The geometry of dissipative evolution equations: the porous medium equation, 2001.
A. Trouvé and L. Younes. Metamorphoses through lie group action, 2005.
L. Younes. Shapes and diffeomorphisms, 2010.
slides available at https://slides.com/theodumont/geometric-ot
•
•
•
•
•
•
•
•
•
•
•
•
References
A geometric view of optimal transport μ 0 \mu_0 μ 1 \mu_1 Théo Dumont Dens ( R n ) \operatorname{Dens}(\mathbb R^n) slides available at https://slides.com/theodumont/geometric-ot
A geometric view of optimal transport
By Théo Dumont
A geometric view of optimal transport
Talk about the infinite-dimensional Riemannian geometry of Optimal Transport for the shape analysis seminar (https://shape-analysis.github.io/) at the MAP5 lab.
- 197
