Klas Modin PRO
Mathematician at Chalmers University of Technology and the University of Gothenburg
Optimal transport, information, and matrix decompositions
Klas Modin
Riemannian principal bundles
E
E
E/H≃B
E/H\simeq B
Hor
\mathrm{Hor}
E
E
↩H
\hookleftarrow H
↓
\downarrow
B
B
π
\pi
Invariant Riemannian metric on E
⇒ π Riemannian submersion
Riemannian principal bundles
E
E
E/H≃B
E/H\simeq B
Hor
\mathrm{Hor}
E
E
↩H
\hookleftarrow H
↓
\downarrow
B
B
π
\pi
Invariant Riemannian metric on E
⇒ π Riemannian submersion
Riemannian principal bundles
G
G
G/H≃B
G/H\simeq B
e
e
Hor
\mathrm{Hor}
G
G
↩H
\hookleftarrow H
↓
\downarrow
B
B
π
\pi
left co-sets [g]=g⋅H
Semi-invariant Riemannian metric on G
⇒ π Riemannian submersion
Riemannian principal bundles
G
G
G/Gb0≃B
G/G_{b_0}\simeq B
e
e
Hor
\mathrm{Hor}
G
G
↩Gb0
\hookleftarrow G_{b_0}
↓
\downarrow
B
B
π(g)=g⋅b0
\pi(g) = g\cdot b_0
left co-sets [g]=g⋅Gb0
Semi-invariant Riemannian metric on G
⇒ π Riemannian submersion
b0
b_0
Gb0
G_{b_0}
Riemannian principal bundles
G
G
Gb0\G≃B
G_{b_0}\backslash G\simeq B
e
e
Hor
\mathrm{Hor}
G
G
Gb0↪
G_{b_0}\hookrightarrow
↓
\downarrow
B
B
π(g)=b0⋅g
\pi(g) = b_0\cdot g
right co-sets [g]= Gb0⋅g
Semi-invariant Riemannian metric on G
⇒ π Riemannian submersion
b0
b_0
Gb0
G_{b_0}
Riemannian principal bundles
G
G
G/Gb0≃B
G/G_{b_0}\simeq B
e
e
horizontal flow
\text{horizontal flow}
G
G
↩Gb0
\hookleftarrow G_{b_0}
↓
\downarrow
B
B
π(g)=g⋅b0
\pi(g) = g\cdot b_0
left co-sets [g]=g⋅Gb0
Semi-invariant Riemannian metric on G
⇒ π Riemannian submersion
b0
b_0
Riemannian principal bundles
G
G
G/Gb0≃B
G/G_{b_0}\simeq B
e
e
vertical flow
\text{vertical flow}
G
G
↩Gb0
\hookleftarrow G_{b_0}
↓
\downarrow
B
B
π(g)=g⋅b0
\pi(g) = g\cdot b_0
left co-sets [g]=g⋅Gb0
Semi-invariant Riemannian metric on G
⇒ π Riemannian submersion
b0
b_0
Riemannian principal bundles
G
G
G/Gb0≃B
G/G_{b_0}\simeq B
e
e
g∈G
g\in G
G
G
↩Gb0
\hookleftarrow G_{b_0}
↓
\downarrow
B
B
π(g)=g⋅b0
\pi(g) = g\cdot b_0
left co-sets [g]=g⋅Gb0
Semi-invariant Riemannian metric on G
⇒ π Riemannian submersion
b0
b_0
b1=π(g)=g⋅b0
b_1 = \pi(g) = g\cdot b_0
Riemannian principal bundles
G
G
G/Gb0≃B
G/G_{b_0}\simeq B
e
e
γ(1)∈G
\gamma(1)\in G
G
G
↩Gb0
\hookleftarrow G_{b_0}
↓
\downarrow
B
B
π(g)=g⋅b0
\pi(g) = g\cdot b_0
left co-sets [g]=g⋅Gb0
Semi-invariant Riemannian metric on G
⇒ π Riemannian submersion
b0
b_0
b1=π(g)=g⋅b0
b_1 = \pi(g) = g\cdot b_0
γ′(t)∈Hor
\gamma'(t)\in \mathrm{Hor}
Riemannian principal bundles
G
G
G/Gb0≃B
G/G_{b_0}\simeq B
e
e
g=γ(1)(γ(1)−1g)
g = \gamma(1)(\gamma(1)^{-1}g)
G
G
↩Gb0
\hookleftarrow G_{b_0}
↓
\downarrow
B
B
π(g)=g⋅b0
\pi(g) = g\cdot b_0
left co-sets [g]=g⋅Gb0
Semi-invariant Riemannian metric on G
⇒ π Riemannian submersion
b0
b_0
b1=π(g)=g⋅b0
b_1 = \pi(g) = g\cdot b_0
γ′(t)∈Hor
\gamma'(t)\in \mathrm{Hor}
klasklas∈Gb0
\overbrace{\phantom{klasklas}}^{\in G_{b_0}}
Riemannian principal bundles
G
G
G/Gb0≃B
G/G_{b_0}\simeq B
e
e
g=γ(1)h
g = \gamma(1)h
G
G
↩Gb0
\hookleftarrow G_{b_0}
↓
\downarrow
B
B
π(g)=g⋅b0
\pi(g) = g\cdot b_0
left co-sets [g]=g⋅Gb0
Semi-invariant Riemannian metric on G
⇒ π Riemannian submersion
b0
b_0
b1=π(g)=g⋅b0
b_1 = \pi(g) = g\cdot b_0
γ′(t)∈Hor
\gamma'(t)\in \mathrm{Hor}
Riemannian principal bundles
G
G
G/Gb0≃B
G/G_{b_0}\simeq B
e
e
g=kh
g = k h
G
G
↩Gb0
\hookleftarrow G_{b_0}
↓
\downarrow
B
B
π(g)=g⋅b0
\pi(g) = g\cdot b_0
left co-sets [g]=g⋅Gb0
Semi-invariant Riemannian metric on G
⇒ π Riemannian submersion
b0
b_0
b1=π(g)=g⋅b0
b_1 = \pi(g) = g\cdot b_0
K
K
polar cone
Riemannian principal bundles
G
G
Gb0\G≃B
G_{b_0}\backslash G\simeq B
e
e
g=hk
g = h k
G
G
Gb0↪
G_{b_0}\hookrightarrow
↓
\downarrow
B
B
π(g)=b0⋅g
\pi(g) = b_0\cdot g
right co-sets [g]=g⋅Gb0
Semi-invariant Riemannian metric on G
⇒ π Riemannian submersion
b0
b_0
b1=π(g)=b0⋅g
b_1 = \pi(g) = b_0\cdot g
K
K
polar cone
Examples
-
Optimal mass transport
(infinite- and finite-dimensional)
⇒ polar decomposition
Vertical flows: Euler equations, free rigid body, gradient OT flow
Horizontal flows: heat flow (entropy gradient flow)
-
Information geometry
⇒QR, Cholesky, singular value, and spectral decompositions
Vertical flows: gradient flow for QR, isospectral flows
Horizontal flows: Brocket flow (entropy gradient flow)
Optimal mass transport (OMT)
μ0
\mu_0
μ1
\mu_1
φ∗μ0
\varphi_*\mu_0
φ∗μ0=μ1min∫MdM2(φ(x),x)μ0
\displaystyle\min_{\varphi*\mu_0=\mu_1} \int_{M} d_M^2(\varphi(x),x ) \mu_0
M
M
Monge problem, L2 version
Optimal mass transport (OMT)
μ0
\mu_0
μ1
\mu_1
φ∗μ0
\varphi_*\mu_0
φ∗μ0=μ1min∫Rn∣φ(x)−x∣2μ0
\displaystyle\min_{\varphi*\mu_0=\mu_1} \int_{\mathbb{R}^n} \lvert \varphi(x)-x \rvert^2 \mu_0
Rn
\mathbb{R}^n
Monge problem, L2 version
J(φ)klasklkllkklklas
\underbrace{\phantom{klasklkllkklklas}}_{J(\varphi)}
Riemannian structure of OMT
Diff(M)
\mathrm{Diff}(M)
Dens(M)≃Diff(M)/Diffμ0(M)
\mathrm{Dens}(M)\simeq \mathrm{Diff}(M)/\mathrm{Diff}_{\mu_0}(M)
Id
\mathrm{Id}
μ0
\mu_0
μ1
\mu_1
π(φ)=φ∗μ0
\pi(\varphi)=\varphi_*\mu_0
Riemannian metric
Gφ(φ˙,φ˙)=∫M∣φ˙∣2μ0
\displaystyle\mathcal{G}_\varphi(\dot\varphi,\dot\varphi) = \int_{M}\left\vert \dot\varphi \right\vert^2 \mu_0
Induced metric
Gμ(μ˙,μ˙)=∫M∣∇θ∣2μ
\overline{\mathcal{G}}_\mu(\dot\mu,\dot\mu) = \int_M \lvert \nabla \theta\rvert^2 \mu
[Benamou & Brenier (2000), Otto (2001)]
Invariance: η∈Diffμ0(M)
Gφ(φ˙,φ˙)=Gφ∘η(φ˙∘η,φ˙∘η)
\displaystyle\mathcal{G}_\varphi(\dot\varphi,\dot\varphi) = \mathcal{G}_{\varphi\circ\eta}(\dot\varphi\circ\eta,\dot\varphi\circ\eta)
ρ˙+div(ρ∇θ)=0,ρ=μ/dx
\dot\rho + \operatorname{div}(\rho \nabla\theta) = 0, \; \rho = \mu/dx
Geodesics on Diff(Rn)
Geodesic equation:
φ¨=0⇒φ(t)=Id+tv0
\ddot\varphi = 0 \Rightarrow \varphi(t) = \mathrm{Id} + t\,v_0
Id
\mathrm{Id}
μ0
\mu_0
K
K
μ1
\mu_1
HorId=∇C∞(Rn)
\mathrm{Hor}_{\mathrm{Id}} = \nabla C^\infty(\mathbb{R}^n)
Easy to prove:
Polar cone K is isomorphic to strictly convex smooth functions via ϕ↦∇ϕ
Hard to prove:
Polar cone K a section of principal bundle
Geodesics on Diff(Rn)
Geodesic equation:
φ¨=0⇒φ(t)=Id+tv0
\ddot\varphi = 0 \Rightarrow \varphi(t) = \mathrm{Id} + t\,v_0
Id
\mathrm{Id}
μ0
\mu_0
K
K
μ1
\mu_1
φ=∇ϕ∘η,η∈Diffμ0(Rn)
\varphi = \nabla\phi\circ\eta, \; \eta\in \operatorname{Diff}_{\mu_0}(\mathbb{R}^n)
Easy to prove:
Polar cone K is isomorphic to strictly convex smooth functions via ϕ↦∇ϕ
Hard to prove:
Polar cone K a section of principal bundle
Brenier's decomposition of transport maps
Geodesic distance on Diff(Rn)
Geodesic curve:
φ(t)=(1−t)φ0+tφ1
\varphi(t) = (1-t)\varphi_0 + t \varphi_1
Id
\mathrm{Id}
μ0
\mu_0
K
K
μ1
\mu_1
dist(φ0,φ1)2=∫01Gφ(t)(φ˙(t),φ˙(t))dt
\mathrm{dist}(\varphi_0,\varphi_1)^2 = \int_0^1 \mathcal{G}_{\varphi(t)}(\dot\varphi(t),\dot\varphi(t)) dt
=∫Rn∣φ1(x)−φ0(t)∣2
= \int_{\mathbb{R}^n} \lvert \varphi_1(x)-\varphi_0(t) \rvert^2
=∫01∫Rn∣φ˙(t)∣2μ0dt
= \int_0^1\int_{\mathbb{R}^n} \lvert \dot\varphi(t)\rvert^2\mu_0 dt
In particular:
J(φ)=dist(Id,φ)2
J(\varphi) = \mathrm{dist}(\mathrm{Id},\varphi)^2
Monge-Ampere equation on Rn
Geodesic curve:
φ(t)=∇(∣x∣2/2+tf)
\varphi(t) = \nabla( |x|^2/2 + t f )
Id
\mathrm{Id}
μ0
\mu_0
K
K
μ1
\mu_1
(∇ϕ)∗μ0=μ1
(\nabla\phi)_*\mu_0 = \mu_1
In particular:
J(φ)=dist(Id,φ)2
J(\varphi) = \mathrm{dist}(\mathrm{Id},\varphi)^2
∇ϕklaklklsklkasl
\underbrace{\phantom{klaklklsklkasl}}_{\nabla\phi}
⇒det(∇2ϕ)=ρ1∘∇ϕρ0
\displaystyle \Rightarrow \operatorname{det}(\nabla^2\phi) = \frac{\rho_0}{\rho_1\circ \nabla\phi}
Monge-Ampere equation on Rn
Geodesic curve:
φ(t)=∇(∣x∣2/2+tf)
\varphi(t) = \nabla( |x|^2/2 + t f )
Id
\mathrm{Id}
μ0
\mu_0
K
K
μ1
\mu_1
(∇ϕ)∗μ0=μ1
(\nabla\phi)_*\mu_0 = \mu_1
In particular:
J(φ)=dist(Id,φ)2
J(\varphi) = \mathrm{dist}(\mathrm{Id},\varphi)^2
∇ϕklaklklsklkasl
\underbrace{\phantom{klaklklsklkasl}}_{\nabla\phi}
⇒det(∇2ϕ)=ρ1∘∇ϕρ0
\displaystyle \Rightarrow \operatorname{det}(\nabla^2\phi) = \frac{\rho_0}{\rho_1\circ \nabla\phi}
Linear optimal mass transport
Trivial observation: φ0(x)=A0x, φ1(x)=A1x linear diffeomorphisms ⇒ geodesic consists of linear diffeomorphisms
Consequence: GL(n) is totally geodesic subgroup of Diff(Rn)
Corresponding subspace of densities (statistical submanifold): multivariate Gaussians with zero mean
ρ(x)=(2π)ndet(Σ)1exp(−21x⊤Σ−1x)
\displaystyle \rho(x) = \frac{1}{\sqrt{(2\pi)^n\mathrm{det}(\Sigma)}}\mathrm{exp}(-\frac{1}{2}x^\top \Sigma^{-1}x)
Σ∈P(n)
\Sigma \in P(n)
Bundle structure
GL(n)
GL(n)
P(n)≃GL(n)/O(n,Σ0)
P(n)\simeq GL(n)/O(n,\Sigma_0)
I
I
Σ0
\Sigma_0
Σ1
\Sigma_1
π(A)=AΣ0A⊤
\pi(A)=A\Sigma_0 A^\top
GA(A˙,A˙)=tr(Σ0A˙⊤A˙)
\mathcal G_A(\dot A,\dot A) = \mathrm{tr}(\Sigma_0 \dot A^\top \dot A)
GˉΣ(Σ˙,Σ˙)=tr(ΣSS)
\bar{\mathcal G}_\Sigma(\dot \Sigma,\dot \Sigma) = \mathrm{tr}(\Sigma SS)
Σ˙=SΣ+ΣS
\dot\Sigma = S\Sigma + \Sigma S
K
K
Bundle structure
GL(n)
GL(n)
P(n)≃GL(n)/O(n,Σ0)
P(n)\simeq GL(n)/O(n,\Sigma_0)
I
I
Σ0
\Sigma_0
Σ1
\Sigma_1
π(A)=AΣ0A⊤
\pi(A)=A\Sigma_0 A^\top
GA(A˙,A˙)=tr(Σ0A˙⊤A˙)
\mathcal G_A(\dot A,\dot A) = \mathrm{tr}(\Sigma_0 \dot A^\top \dot A)
GˉΣ(Σ˙,Σ˙)=tr(ΣSS)
\bar{\mathcal G}_\Sigma(\dot \Sigma,\dot \Sigma) = \mathrm{tr}(\Sigma SS)
Σ˙=SΣ+ΣS
\dot\Sigma = S\Sigma + \Sigma S
K=P(n)
K = P(n)
Bundle structure
GL(n)
GL(n)
P(n)≃GL(n)/O(n,Σ0)
P(n)\simeq GL(n)/O(n,\Sigma_0)
I
I
Σ0
\Sigma_0
Σ1
\Sigma_1
π(P)=PΣ0P=Σ1
\pi(P)=P\Sigma_0 P = \Sigma_1
GA(A˙,A˙)=tr(Σ0A˙⊤A˙)
\mathcal G_A(\dot A,\dot A) = \mathrm{tr}(\Sigma_0 \dot A^\top \dot A)
GˉΣ(Σ˙,Σ˙)=tr(ΣSS)
\bar{\mathcal G}_\Sigma(\dot \Sigma,\dot \Sigma) = \mathrm{tr}(\Sigma SS)
Σ˙=SΣ+ΣS
\dot\Sigma = S\Sigma + \Sigma S
K=P(n)
K = P(n)
Monge-Ampere equation:
P
P
Bundle structure
GL(n)
GL(n)
P(n)≃GL(n)/O(n,Σ0)
P(n)\simeq GL(n)/O(n,\Sigma_0)
I
I
Σ0
\Sigma_0
Σ1
\Sigma_1
A=PQ
A = PQ
GA(A˙,A˙)=tr(Σ0A˙⊤A˙)
\mathcal G_A(\dot A,\dot A) = \mathrm{tr}(\Sigma_0 \dot A^\top \dot A)
GˉΣ(Σ˙,Σ˙)=tr(ΣSS)
\bar{\mathcal G}_\Sigma(\dot \Sigma,\dot \Sigma) = \mathrm{tr}(\Sigma SS)
Σ˙=SΣ+ΣS
\dot\Sigma = S\Sigma + \Sigma S
K=P(n)
K = P(n)
Factorization theorem:
A
A
P
P
Bundle structure
GL(n)
GL(n)
P(n)≃GL(n)/O(n,Σ0)
P(n)\simeq GL(n)/O(n,\Sigma_0)
I
I
Σ0
\Sigma_0
Σ1
\Sigma_1
B˙=−Pr∇GJ(B),B(0)=A
\dot B = -\mathrm{Pr}\nabla_{\mathcal G}J(B), \; B(0) = A
GA(A˙,A˙)=tr(Σ0A˙⊤A˙)
\mathcal G_A(\dot A,\dot A) = \mathrm{tr}(\Sigma_0 \dot A^\top \dot A)
GˉΣ(Σ˙,Σ˙)=tr(ΣSS)
\bar{\mathcal G}_\Sigma(\dot \Sigma,\dot \Sigma) = \mathrm{tr}(\Sigma SS)
Σ˙=SΣ+ΣS
\dot\Sigma = S\Sigma + \Sigma S
K=P(n)
K = P(n)
Vertical gradient flow:
A
A
Bundle structure
GL(n)
GL(n)
P(n)≃GL(n)/O(n,Σ0)
P(n)\simeq GL(n)/O(n,\Sigma_0)
I
I
Σ0
\Sigma_0
Σ1
\Sigma_1
B˙=ΩB,B(0)=A
\dot B = \Omega B, \; B(0) = A
GA(A˙,A˙)=tr(Σ0A˙⊤A˙)
\mathcal G_A(\dot A,\dot A) = \mathrm{tr}(\Sigma_0 \dot A^\top \dot A)
GˉΣ(Σ˙,Σ˙)=tr(ΣSS)
\bar{\mathcal G}_\Sigma(\dot \Sigma,\dot \Sigma) = \mathrm{tr}(\Sigma SS)
Σ˙=SΣ+ΣS
\dot\Sigma = S\Sigma + \Sigma S
K=P(n)
K = P(n)
Vertical gradient flow:
A
A
Σ1Ω+ΩΣ1=2Σ1(B−1−B−⊤)
\Sigma_1 \Omega + \Omega\Sigma_1 = 2\Sigma_1 (B^{-1}-B^{-\top})
Bundle structure
GL(n)
GL(n)
P(n)
P(n)
I
I
Σ0
\Sigma_0
Σ1
\Sigma_1
B˙=ΩB,B(0)=A
\dot B = \Omega B, \; B(0) = A
GA(A˙,A˙)=tr(Σ0A˙⊤A˙)
\mathcal G_A(\dot A,\dot A) = \mathrm{tr}(\Sigma_0 \dot A^\top \dot A)
GˉΣ(Σ˙,Σ˙)=tr(ΣSS)
\bar{\mathcal G}_\Sigma(\dot \Sigma,\dot \Sigma) = \mathrm{tr}(\Sigma SS)
Σ˙=SΣ+ΣS
\dot\Sigma = S\Sigma + \Sigma S
K=P(n)
K = P(n)
Vertical gradient flow:
A
A
Σ1Ω+ΩΣ1=2Σ1(B−1−B−⊤)
\Sigma_1 \Omega + \Omega\Sigma_1 = 2\Sigma_1 (B^{-1}-B^{-\top})
Bundle structure
GL(n)
GL(n)
P(n)
P(n)
I
I
Σ0
\Sigma_0
Σ1
\Sigma_1
Σ˙=Pr∇GˉHΣ1(Σ)
\dot \Sigma = \mathrm{Pr}\nabla_{\bar{\mathcal G}}H_{\Sigma_1}(\Sigma)
GA(A˙,A˙)=tr(Σ0A˙⊤A˙)
\mathcal G_A(\dot A,\dot A) = \mathrm{tr}(\Sigma_0 \dot A^\top \dot A)
GˉΣ(Σ˙,Σ˙)=tr(ΣSS)
\bar{\mathcal G}_\Sigma(\dot \Sigma,\dot \Sigma) = \mathrm{tr}(\Sigma SS)
Σ˙=SΣ+ΣS
\dot\Sigma = S\Sigma + \Sigma S
K=P(n)
K = P(n)
Horizontal gradient (heat) flow:
Σ(t)
\Sigma(t)
HΣ1(Σ)=−21tr(Σ1−1Σ)+21logdet(Σ1−1Σ)
\displaystyle H_{\Sigma_1}(\Sigma) = -\frac{1}{2}\mathrm{tr}(\Sigma_1^{-1}\Sigma) + \frac{1}{2}\log\det(\Sigma_1^{-1}\Sigma)
Relative entropy
(Kullback-Leibler)
Bundle structure
GL(n)
GL(n)
P(n)
P(n)
I
I
Σ0
\Sigma_0
Σ1
\Sigma_1
Σ˙=2I−Σ1−1Σ−ΣΣ1−1
\dot \Sigma = 2I - \Sigma_1^{-1}\Sigma - \Sigma\Sigma_1^{-1}
GA(A˙,A˙)=tr(Σ0A˙⊤A˙)
\mathcal G_A(\dot A,\dot A) = \mathrm{tr}(\Sigma_0 \dot A^\top \dot A)
GˉΣ(Σ˙,Σ˙)=tr(ΣSS)
\bar{\mathcal G}_\Sigma(\dot \Sigma,\dot \Sigma) = \mathrm{tr}(\Sigma SS)
Σ˙=SΣ+ΣS
\dot\Sigma = S\Sigma + \Sigma S
K=P(n)
K = P(n)
Horizontal gradient (heat) flow:
Σ(t)
\Sigma(t)
HΣ1(Σ)=−21tr(Σ1−1Σ)+21logdet(Σ1−1Σ)
\displaystyle H_{\Sigma_1}(\Sigma) = -\frac{1}{2}\mathrm{tr}(\Sigma_1^{-1}\Sigma) + \frac{1}{2}\log\det(\Sigma_1^{-1}\Sigma)
Relative entropy
(Kullback-Leibler)
Bundle structure
GL(n)
GL(n)
P(n)
P(n)
I
I
Σ0
\Sigma_0
Σ1
\Sigma_1
P˙=P−1Σ0−1−Σ1−1P+V
\dot P = P^{-1}\Sigma_0^{-1} - \Sigma_1^{-1}P + V
GA(A˙,A˙)=tr(Σ0A˙⊤A˙)
\mathcal G_A(\dot A,\dot A) = \mathrm{tr}(\Sigma_0 \dot A^\top \dot A)
GˉΣ(Σ˙,Σ˙)=tr(ΣSS)
\bar{\mathcal G}_\Sigma(\dot \Sigma,\dot \Sigma) = \mathrm{tr}(\Sigma SS)
Σ˙=SΣ+ΣS
\dot\Sigma = S\Sigma + \Sigma S
K=P(n)
K = P(n)
Horizontal gradient (heat) flow:
P(t)
P(t)
F(P)=HΣ1(PΣ0P)
\displaystyle F(P) = H_{\Sigma_1}(P\Sigma_0 P)
Lifted gradient flow on K for
Bundle structure
GL(n)
GL(n)
P(n)
P(n)
I
I
Σ0
\Sigma_0
Σ1
\Sigma_1
P˙=P−1Σ0−1−Σ1−1P+V
\dot P = P^{-1}\Sigma_0^{-1} - \Sigma_1^{-1}P + V
GA(A˙,A˙)=tr(Σ0A˙⊤A˙)
\mathcal G_A(\dot A,\dot A) = \mathrm{tr}(\Sigma_0 \dot A^\top \dot A)
GˉΣ(Σ˙,Σ˙)=tr(ΣSS)
\bar{\mathcal G}_\Sigma(\dot \Sigma,\dot \Sigma) = \mathrm{tr}(\Sigma SS)
Σ˙=SΣ+ΣS
\dot\Sigma = S\Sigma + \Sigma S
K=P(n)
K = P(n)
Horizontal gradient (heat) flow:
P(t)
P(t)
F(P)=HΣ1(PΣ0P)
\displaystyle F(P) = H_{\Sigma_1}(P\Sigma_0 P)
Lifted gradient flow on K for
Hessian of F(P) strictly positive on K ⇒ unique limit!
Bundle structure
GL(n)
GL(n)
P(n)
P(n)
I
I
Σ0
\Sigma_0
Σ1
\Sigma_1
P˙=P−1Σ0−1−Σ1−1P+V
\dot P = P^{-1}\Sigma_0^{-1} - \Sigma_1^{-1}P + V
GA(A˙,A˙)=tr(Σ0A˙⊤A˙)
\mathcal G_A(\dot A,\dot A) = \mathrm{tr}(\Sigma_0 \dot A^\top \dot A)
GˉΣ(Σ˙,Σ˙)=tr(ΣSS)
\bar{\mathcal G}_\Sigma(\dot \Sigma,\dot \Sigma) = \mathrm{tr}(\Sigma SS)
Σ˙=SΣ+ΣS
\dot\Sigma = S\Sigma + \Sigma S
K=P(n)
K = P(n)
Horizontal gradient (heat) flow:
P(t)
P(t)
F(P)=HΣ1(PΣ0P)
\displaystyle F(P) = H_{\Sigma_1}(P\Sigma_0 P)
Lifted gradient flow on K for
Hessian of F(P) strictly positive on K ⇒ unique limit!
Bundle structure
GL(n)
GL(n)
P(n)
P(n)
I
I
Σ0
\Sigma_0
Σ1
\Sigma_1
P˙=P−1Σ0−1−Σ1−1P+V
\dot P = P^{-1}\Sigma_0^{-1} - \Sigma_1^{-1}P + V
GA(A˙,A˙)=tr(Σ0A˙⊤A˙)
\mathcal G_A(\dot A,\dot A) = \mathrm{tr}(\Sigma_0 \dot A^\top \dot A)
GˉΣ(Σ˙,Σ˙)=tr(ΣSS)
\bar{\mathcal G}_\Sigma(\dot \Sigma,\dot \Sigma) = \mathrm{tr}(\Sigma SS)
Σ˙=SΣ+ΣS
\dot\Sigma = S\Sigma + \Sigma S
K=P(n)
K = P(n)
Horizontal gradient (heat) flow:
P(t)
P(t)
F(P)=HΣ1(PΣ0P)
\displaystyle F(P) = H_{\Sigma_1}(P\Sigma_0 P)
Lifted gradient flow on K for
Hessian of F(P) strictly positive on K ⇒ unique limit!
Bundle structure
GL(n)
GL(n)
P(n)
P(n)
I
I
Σ0
\Sigma_0
Σ1
\Sigma_1
P˙=P−1Σ0−1−Σ1−1P+V
\dot P = P^{-1}\Sigma_0^{-1} - \Sigma_1^{-1}P + V
GA(A˙,A˙)=tr(Σ0A˙⊤A˙)
\mathcal G_A(\dot A,\dot A) = \mathrm{tr}(\Sigma_0 \dot A^\top \dot A)
GˉΣ(Σ˙,Σ˙)=tr(ΣSS)
\bar{\mathcal G}_\Sigma(\dot \Sigma,\dot \Sigma) = \mathrm{tr}(\Sigma SS)
Σ˙=SΣ+ΣS
\dot\Sigma = S\Sigma + \Sigma S
K=P(n)
K = P(n)
Horizontal gradient (heat) flow:
P(t)
P(t)
F(P)=HΣ1(PΣ0P)
\displaystyle F(P) = H_{\Sigma_1}(P\Sigma_0 P)
Lifted gradient flow on K for
Hessian of F(P) strictly positive on K ⇒ unique limit!
Wasserstein-Otto vs. Fisher-Rao
Dens(M)
\mathrm{Dens}(M)
TμDens(M)≃C0∞(M)
T_\mu\mathrm{Dens}(M)\simeq C^\infty_0(M)
Wasserstein
Fisher-Rao
Gμ(μ˙,μ˙)=∫Mμμ˙μμ˙μ
\displaystyle\overline{\mathcal{G}}_\mu(\dot\mu,\dot\mu) = \int_{M} \frac{\dot\mu}{\mu}\frac{\dot\mu}{\mu}\mu
Gρdx(ρ˙dx,ρ˙dx)=∫M∣∇θ∣2ρ
\displaystyle\overline{\mathcal{G}}_{\rho dx}(\dot\rho dx,\dot\rho dx) = \int_{M} |\nabla\theta|^2\rho
ρ˙+div(ρ∇θ)=0
\displaystyle \dot\rho + \mathrm{div}(\rho \nabla\theta) = 0
Dependent on Riemannian structure of M
Independent of Riemannian structure of M⇒Diff(M)-invariance
ρ(x)=(2π)ndet(Σ)1exp(−21x⊤Σ−1x)
\displaystyle \rho(x) = \frac{1}{\sqrt{(2\pi)^n\mathrm{det}(\Sigma)}}\mathrm{exp}(-\frac{1}{2}x^\top \Sigma^{-1}x)
ρ(x)=(2π)ndet(W)exp(−21x⊤Wx)
\displaystyle \rho(x) = \sqrt{\frac{\mathrm{det}(W)}{(2\pi)^n}}\mathrm{exp}(-\frac{1}{2}x^\top W x)
Bundle structure
GL(n)
GL(n)
P(n)≃O(n,W0−1)\GL(n)
P(n)\simeq O(n,W_0^{-1})\backslash GL(n)
I
I
W0
W_0
W1
W_1
π(A)=A⊤W0A
\pi(A)=A^\top W_0 A
GˉW(W˙,W˙)=tr(W−1W˙W−1W˙)
\bar{\mathcal G}_W(\dot W,\dot W) = \mathrm{tr}(W^{-1}\dot W W^{-1}\dot W)
K
K
Requirement for compatible metric on GL(n) :
- Descending, meaning
left-invariant w.r.t. O(n,W0−1 ) - Right-invariant w.r.t. GL(n)
Bundle structure
GL(n)
GL(n)
P(n)≃O(n,W0−1)\GL(n)
P(n)\simeq O(n,W_0^{-1})\backslash GL(n)
I
I
W0
W_0
W1
W_1
π(A)=A⊤W0A
\pi(A)=A^\top W_0 A
GI(ξ,ξ)=tr(ℓ(ξ)⊤ℓ(ξ)+σ(ξ)2)
\mathcal G_I(\xi,\xi) = \mathrm{tr}( \ell(\xi)^\top\ell(\xi) + \sigma(\xi)^2)
GˉW(W˙,W˙)=tr(W−1W˙W−1W˙)
\bar{\mathcal G}_W(\dot W,\dot W) = \mathrm{tr}(W^{-1}\dot W W^{-1}\dot W)
K
K
Requirement for compatible metric on GL(n) :
- Descending, meaning
left-invariant w.r.t. O(n,W0−1 ) - Right-invariant w.r.t. GL(n)
Bundle structure
GL(n)
GL(n)
P(n)≃O(n,W0−1)\GL(n)
P(n)\simeq O(n,W_0^{-1})\backslash GL(n)
I
I
W0
W_0
W1
W_1
π(A)=A⊤W0A
\pi(A)=A^\top W_0 A
GI(ξ,ξ)=tr(ℓ(ξ)⊤ℓ(ξ)+σ(ξ)2)
\mathcal G_I(\xi,\xi) = \mathrm{tr}( \ell(\xi)^\top\ell(\xi) + \sigma(\xi)^2)
GˉW(W˙,W˙)=tr(W−1W˙W−1W˙)
\bar{\mathcal G}_W(\dot W,\dot W) = \mathrm{tr}(W^{-1}\dot W W^{-1}\dot W)
K
K
Requirement for compatible metric on GL(n) :
- Descending, meaning
left-invariant w.r.t. O(n,W0−1 ) - Right-invariant w.r.t. GL(n)
HorI= upper triangular matrices
Bundle structure
GL(n)
GL(n)
P(n)≃O(n,W0−1)\GL(n)
P(n)\simeq O(n,W_0^{-1})\backslash GL(n)
I
I
W0
W_0
W1
W_1
π(A)=A⊤W0A
\pi(A)=A^\top W_0 A
GI(ξ,ξ)=tr(ℓ(ξ)⊤ℓ(ξ)+σ(ξ)2)
\mathcal G_I(\xi,\xi) = \mathrm{tr}( \ell(\xi)^\top\ell(\xi) + \sigma(\xi)^2)
GˉW(W˙,W˙)=tr(W−1W˙W−1W˙)
\bar{\mathcal G}_W(\dot W,\dot W) = \mathrm{tr}(W^{-1}\dot W W^{-1}\dot W)
K
K
Requirement for compatible metric on GL(n) :
- Descending, meaning
left-invariant w.r.t. O(n,W0−1 ) - Right-invariant w.r.t. GL(n)
HorI= upper triangular matrices
Lie subalgebra ⇒ Hor is integrable
Bundle structure
GL(n)
GL(n)
P(n)≃O(n,W0−1)\GL(n)
P(n)\simeq O(n,W_0^{-1})\backslash GL(n)
I
I
W0
W_0
W1
W_1
π(A)=A⊤W0A
\pi(A)=A^\top W_0 A
GI(ξ,ξ)=tr(ℓ(ξ)⊤ℓ(ξ)+σ(ξ)2)
\mathcal G_I(\xi,\xi) = \mathrm{tr}( \ell(\xi)^\top\ell(\xi) + \sigma(\xi)^2)
GˉW(W˙,W˙)=tr(W−1W˙W−1W˙)
\bar{\mathcal G}_W(\dot W,\dot W) = \mathrm{tr}(W^{-1}\dot W W^{-1}\dot W)
K
K
Requirement for compatible metric on GL(n) :
- Descending, meaning
left-invariant w.r.t. O(n,W0−1 ) - Right-invariant w.r.t. GL(n)
HorI= upper triangular matrices
Lie subalgebra ⇒ Hor is integrable
K= subgroup of upper triangular matrices with positive entries on diagonal
Bundle structure
GL(n)
GL(n)
P(n)≃O(n,W0−1)\GL(n)
P(n)\simeq O(n,W_0^{-1})\backslash GL(n)
I
I
W0
W_0
W1
W_1
π(A)=A⊤W0A
\pi(A)=A^\top W_0 A
GI(ξ,ξ)=tr(ℓ(ξ)⊤ℓ(ξ)+σ(ξ)2)
\mathcal G_I(\xi,\xi) = \mathrm{tr}( \ell(\xi)^\top\ell(\xi) + \sigma(\xi)^2)
GˉW(W˙,W˙)=tr(W−1W˙W−1W˙)
\bar{\mathcal G}_W(\dot W,\dot W) = \mathrm{tr}(W^{-1}\dot W W^{-1}\dot W)
K
K
Requirement for compatible metric on GL(n) :
- Descending, meaning
left-invariant w.r.t. O(n,W0−1 ) - Right-invariant w.r.t. GL(n)
HorI= upper triangular matrices
Lie subalgebra ⇒ Hor is integrable
K= subgroup of upper triangular matrices with positive entries on diagonal
A
A
R
R
Q
Q
Bundle structure
GL(n)
GL(n)
P(n)≃O(n,W0−1)\GL(n)
P(n)\simeq O(n,W_0^{-1})\backslash GL(n)
I
I
W0
W_0
W1
W_1
π(A)=A⊤W0A
\pi(A)=A^\top W_0 A
GI(ξ,ξ)=tr(ℓ(ξ)⊤ℓ(ξ)+σ(ξ)2)
\mathcal G_I(\xi,\xi) = \mathrm{tr}( \ell(\xi)^\top\ell(\xi) + \sigma(\xi)^2)
GˉW(W˙,W˙)=tr(W−1W˙W−1W˙)
\bar{\mathcal G}_W(\dot W,\dot W) = \mathrm{tr}(W^{-1}\dot W W^{-1}\dot W)
K
K
Requirement for compatible metric on GL(n) :
- Descending, meaning
left-invariant w.r.t. O(n,W0−1 ) - Right-invariant w.r.t. GL(n)
HorI= upper triangular matrices
Lie subalgebra ⇒ Hor is integrable
K= subgroup of upper triangular matrices with positive entries on diagonal
A
A
R
R
Q
Q
Bundle structure
GL(n)
GL(n)
P(n)≃O(n,W0−1)\GL(n)
P(n)\simeq O(n,W_0^{-1})\backslash GL(n)
I
I
W0
W_0
W1
W_1
π(A)=A⊤W0A
\pi(A)=A^\top W_0 A
GI(ξ,ξ)=tr(ℓ(ξ)⊤ℓ(ξ)+σ(ξ)2)
\mathcal G_I(\xi,\xi) = \mathrm{tr}( \ell(\xi)^\top\ell(\xi) + \sigma(\xi)^2)
GˉW(W˙,W˙)=tr(W−1W˙W−1W˙)
\bar{\mathcal G}_W(\dot W,\dot W) = \mathrm{tr}(W^{-1}\dot W W^{-1}\dot W)
K
K
Requirement for compatible metric on GL(n) :
- Descending, meaning
left-invariant w.r.t. O(n,W0−1 ) - Right-invariant w.r.t. GL(n)
HorI= upper triangular matrices
Lie subalgebra ⇒ Hor is integrable
K= subgroup of upper triangular matrices with positive entries on diagonal
A=
A =
R
R
Q
Q
Bundle structure
GL(n)
GL(n)
P(n)≃O(n,W0−1)\GL(n)
P(n)\simeq O(n,W_0^{-1})\backslash GL(n)
I
I
W0
W_0
W1
W_1
π(A)=A⊤W0A
\pi(A)=A^\top W_0 A
GI(ξ,ξ)=tr(ℓ(ξ)⊤ℓ(ξ)+σ(ξ)2)
\mathcal G_I(\xi,\xi) = \mathrm{tr}( \ell(\xi)^\top\ell(\xi) + \sigma(\xi)^2)
GˉW(W˙,W˙)=tr(W−1W˙W−1W˙)
\bar{\mathcal G}_W(\dot W,\dot W) = \mathrm{tr}(W^{-1}\dot W W^{-1}\dot W)
K
K
Requirement for compatible metric on GL(n) :
- Descending, meaning
left-invariant w.r.t. O(n,W0−1 ) - Right-invariant w.r.t. GL(n)
HorI= upper triangular matrices
Lie subalgebra ⇒ Hor is integrable
K= subgroup of upper triangular matrices with positive entries on diagonal
W1=π(R)=R⊤W0R
W_1 = \pi(R) = R^\top W_0 R
R
R
Entropy gradient flow (Fisher-Rao)
GL(n)
GL(n)
P(n)
P(n)
I
I
W0
W_0
W1
W_1
W(t)
W(t)
HW1(Σ)=−21tr(W1W−1)+21logdet(W1W−1)
\displaystyle H_{W_1}(\Sigma) = -\frac{1}{2}\mathrm{tr}(W_1 W^{-1}) + \frac{1}{2}\log\det(W_1 W^{-1})
Relative entropy
W˙=W1−W
\dot W = W_1 - W
W˙=∇GˉHW0(W)
\dot W = \nabla_{\bar{\mathcal G}}H_{W_0}(W)
Further reduction
GL(n)
GL(n)
I
I
I
I
W1
W_1
P(n)≃O(n)\GL(n)
P(n)\simeq O(n)\backslash GL(n)
P(n)/O(n,)≃polyn+
P(n)/O(n,)\simeq \mathrm{poly}^+_n
π(A)=A⊤A
\pi(A) = A^\top A
ϖ(P)=det(λI−W)
\varpi(P) = \det(\lambda I - W)
Right action by (Q,W)↦Q⊤WQ, clearly not free!
Spectral decomposition
P(n)
P(n)
I
I
W
W
P(n)/O(n,)≃polyn+
P(n)/O(n,)\simeq \mathrm{poly}^+_n
D(n)
D(n)
Geodesic equation on D(n):
D(n) is totally geodesic submanifold!
γ¨i−γiγ˙i2=0
\displaystyle \ddot\gamma_i - \frac{\dot\gamma_i^2}{\gamma_i} = 0
Notice: D(n) intersects W1-orbit n! times
( D(n) an n!-covering of polyn+ )
Λ
\Lambda
Spectral decomposition
P(n)
P(n)
I
I
W=
W =
P(n)/O(n,)≃polyn+
P(n)/O(n,)\simeq \mathrm{poly}^+_n
D(n)
D(n)
Geodesic equation on D(n):
D(n) is totally geodesic submanifold!
γ¨i−γiγ˙i2=0
\displaystyle \ddot\gamma_i - \frac{\dot\gamma_i^2}{\gamma_i} = 0
Notice: D(n) intersects W1-orbit n! times
( D(n) an n!-covering of polyn+ )
QΛQ⊤
Q\Lambda Q^\top
Brockett flow
P(n)
P(n)
N
N
W1
W_1
D(n)
D(n)
HN(W) relative entropy functional
Functional F(Q)=HN(Q⊤W1Q) on O(n)
Orb(W1)
\mathrm{Orb}(W_1)
THANKS!
Reference:
- K. Modin
Geometry of Matrix Decompositions Seen Through Optimal Transport and Information Geometry, 2017
Slides available at: slides.com/kmodin
Optimal transport, information, and matrix decompositions Klas Modin slides.com/kmodin/matrix-decompositions
Geometry of matrix decompositions seen through optimal transport and information geometry
By Klas Modin
Geometry of matrix decompositions seen through optimal transport and information geometry
Online-presentation given 2020-12 in the Hamiltonian Seminar Series, University of Toronto.
- 1,816
