Polar Decomposition as an Entropy Gradient Flow
Klas Modin
Outline
- Polar decompositions and optimal transport
- Wasserstein geometry
- Optimal transport in linear category
- Lifted entropy gradient flow
- Convexity and convergence
- Numerical example
- Outlook: infinite-dimensional flow
Polar decomposition
A∈GL(n)⇒A=PQ
A\in\mathrm{GL}(n) \quad\Rightarrow\quad A=PQ
Q=
Q=
P=
P=
Proof techniques
- Elementary linear algebra
- Toy example of Brenier's factorization of maps
- Limit of entropy gradient flow on
P(n)={P∈Rn∣P⊤=P,P>0}
\mathrm{P}(n) = \{ P \in \mathbb{R}^n \mid P^\top = P, P>0 \}
Optimal transport
μ0
\mu_0
μ1
\mu_1
η∗μ0
\eta_*\mu_0
(L2)
(L^2)
minη∗μ0=μ1∫Rn∣η(x)−x∣2dμ0
\min_{\eta_*\mu_0=\mu_1} \int_{\mathbb{R}^n} \left|\eta(x)-x \right|^2 d\mu_0
Rn
\mathbb{R^n}
Monge
Wasserstein distance
dW2(μ0,μ1)=infη∗μ0=μ1∫Rn∣η(x)−x∣2dμ0
d_W^2(\mu_0,\mu_1) = \inf_{\eta_*\mu_0=\mu_1} \int_{\mathbb{R}^n} \left|\eta(x)-x \right|^2 d\mu_0
Symmetric by change of variables
Riemannian structure
Diff(Rn)
\mathrm{Diff}(\mathbb{R}^n)
Dens(Rn)
\mathrm{Dens}(\mathbb{R}^n)
Id
\mathrm{Id}
μ0
\mu_0
μ1
\mu_1
π(η)=η∗μ0
\pi(\eta)=\eta_*\mu_0
Moser 1965:
Principal bundle
Diff(Rn)/Diffμ0(Rn)
\mathrm{Diff}(\mathbb{R}^n)/\mathrm{Diff}_{\mu_0}(\mathbb{R}^n)
Otto 2001:
Gη(η˙,η˙)=∫Rn∣η˙∣2dμ0
\mathcal{G}_\eta(\dot\eta,\dot\eta) = \int_{\mathbb{R}^n}\left\vert \dot\eta \right\vert^2 d \mu_0
Induces metric
Gμ(μ˙,μ˙)⇒dW2(μ0,μ1)
\overline{\mathcal{G}}_\mu(\dot\mu,\dot\mu) \Rightarrow d_W^2(\mu_0,\mu_1)
Hor
\mathrm{Hor}
polar cone
densities
id
\mathrm{id}
∇ϕ
\nabla\phi
fiber
π
\pi
fiber
μ0
\mu_0
μ1
\mu_1
K={∇ϕ∣∇2ϕ>0}
K = \{ \nabla\phi \mid \nabla^2\phi > 0 \}
Dens(Rn)
\mathrm{Dens}(\mathbb{R}^n)
Central Lemma: The mapping
is an isomorphism (section of principal bundle)
π∣K:K→Dens(Rn)
\pi|_{K}\colon K \to \mathrm{Dens}(\mathbb{R}^n)
η
\eta
=∇ϕ∘ψ
= \nabla\phi \circ \psi
ψ
\psi
Linear/Gaussian category
μΣ=det(Σ)(2π)n1exp(−21x⊤Σ−1x)dx
\mu_\Sigma = \frac{1}{\sqrt{\det(\Sigma)(2\pi)^n}}\exp(-\frac{1}{2}x^\top \Sigma^{-1}x) dx
Multivariate zero-mean Gaussian densities
GL(n)≃{x↦Ax∣A∈GL(n)}⊂Diff(Rn)
\mathrm{GL}(n) \simeq \{ x\mapsto Ax \mid A \in \mathrm{GL}(n) \} \subset \mathrm{Diff}(\mathbb{R}^n)
Linear transformations are totally geodesic
Nn={μΣ∣Σ∈P(n)}⊂Dens(Rn)
\mathcal{N}_n =\{ \mu_\Sigma \mid \Sigma\in \mathrm{P}(n) \} \subset \mathrm{Dens}(\mathbb{R}^n)
Action map:
η↦η∗μ⇔A↦AΣA⊤
\eta \mapsto \eta_*\mu \quad\Leftrightarrow\quad A \mapsto A\Sigma A^\top
polar cone
covar. matrices
I
I
P
P
fiber
π
\pi
fiber
Σ0
\Sigma_0
Σ1
\Sigma_1
K=P(n)
K = \mathrm{P}(n)
P(n)
\mathrm{P}(n)
Central Lemma: The mapping
is an isomorphism (section of principal bundle)
K∋P↦PΣ0P⊤∈P(n)
K \ni P \mapsto P\Sigma_0 P^\top \in \mathrm{P}(n)
A
A
=PQ
= P Q
Q
Q
polar cone
covar. matrices
I
I
P
P
Σ0
\Sigma_0
Σ1
\Sigma_1
A
A
Lifted gradient flow
K=P(n)
K = \mathrm{P}(n)
P(n)
\mathrm{P}(n)
- Convex functional on with
- Lifted functional on
- Consider gradient flow on polar cone
Fˉ
\bar F
P(n)
\mathrm{P}(n)
∇GˉFˉ(Σ1)=0
\nabla_{\bar{\mathcal{G}}}\bar{F}(\Sigma_1) = 0
GL(n)
\mathrm{GL}(n)
F(A)=Fˉ(π(A))=Fˉ(AΣ0A⊤)
F(A) = \bar F(\pi(A)) = \bar F(A\Sigma_0 A^\top)
P˙=−Π∇GF(P)
\dot P = -\Pi \nabla_{\mathcal G} F(P)
Lifted relative entropy
Hˉ(Σ)=2n−21tr(Σ1−1Σ)+21log(det(Σ1−1Σ))
\bar H(\Sigma) = \frac{n}{2}- \frac{1}{2}\mathrm{tr}(\Sigma_1^{-1}\Sigma) + \frac{1}{2}\log(\det(\Sigma_1^{-1}\Sigma))
H(A)=2n−21tr(Σ1−1AΣ0A⊤)+log(det(A))
H(A) = \frac{n}{2}- \frac{1}{2}\mathrm{tr}(\Sigma_1^{-1}A\Sigma_0A^\top) + \log(\det(A))
Final gradient flow
(Σ0=I)
(\Sigma_0 = I)
P˙=P−1−21(Σ1−1P+PΣ1−1)
\dot P = P^{-1} - \frac{1}{2}(\Sigma_1^{-1}P + P \Sigma_1^{-1})
Convergence
Lemma:
−Hess(H∣K)P≥αGPα>0
-\mathrm{Hess}(H|_{K})_P \geq \alpha \mathcal{G}_P \quad \alpha > 0
Corollary:
d2(P(t),P∞)≤e−2αtd2(P(0),P∞)
d^2(P(t),P_\infty) \leq \, \mathrm{e}^{-2\alpha t} d^2(P(0),P_\infty)
Example
P=[3−1−12]
P = \begin{bmatrix}
3 & -1 \\
-1 & 2
\end{bmatrix}
Σ1=π(P)=PP⊤
\Sigma_1 = \pi(P) = PP^\top
Example
−H(P(t))
-H(P(t))
d2(P(t),P∞)
d^2(P(t),P_{\infty})
Outlook: inf-dim version
H(∇ϕ)=−∫Rndet(∇2ϕ∘(∇ϕ)−1)ρ0∘(∇ϕ)−1log(ρ1det(∇2ϕ∘(∇ϕ)−1)ρ0∘(∇ϕ)−1)dx
H(\nabla\phi) = -\int_{\mathbb{R}^n}\frac{\rho_0\circ (\nabla\phi)^{-1}}{\det(\nabla^2\phi\circ(\nabla\phi)^{-1})}\log\left( \frac{\rho_0\circ(\nabla\phi)^{-1}}{\rho_1 \det(\nabla^2\phi\circ (\nabla\phi)^{-1})} \right)dx
ϕ˙=−∇⋅ρ0∇⋅(∇2ϕ)−1−∇⋅(∇2ϕ)−1⋅∇ρ0+∇⋅ρ0ρ1∘(∇ϕ)−1)∇ρ1∘(∇ϕ)−1)
\dot\phi = -\nabla\cdot\rho_0\nabla\cdot (\nabla^2\phi)^{-1} - \nabla\cdot (\nabla^2\phi)^{-1}\cdot\nabla\rho_0 + \nabla\cdot\rho_0 \frac{\nabla\rho_1\circ(\nabla\phi)^{-1})}{\rho_1\circ (\nabla\phi)^{-1})}
Observation and conjecture
Observation: if log-concave then
ρ1
\rho_1
−Hess(H)∇ϕ≥αG∇ϕ
-\mathrm{Hess}(H)_{\nabla\phi} \geq \alpha \mathcal{G}_{\nabla\phi}
Conjecture: if log-concave then convergence
ρ1
\rho_1
THANKS!
Polar Decomposition as an Entropy Gradient Flow Klas Modin