Embed

Diffeomorphic image registration taking topological differences into account

Metamorphosis on brain MRI containing Glioblastoma

Anton François

Supervised by

Joan Glaunès & Pietro Gori

Introduction

Anton François - 23/05/2023

Let $V$ be the Reproducing Kernel Hilbert Space (RKHS) of vector fields whose kernel is $K$ . We denote $L: V \rightarrow V^*$ the Riesz operator such that $\|v\|^2_V = ( Lv,v)$ .

Let $K$ be the Gaussian reproducing kernel with $K(x,y) = \exp\left( -\frac{|x -y|^2}{2\sigma^2}\right) \cdot \mathrm{Id}_{\mathbb{R}^d}$

V is an admissible RKHS of vector fields.

Deformation as a flow of vector field

$\partial_t \varphi_t = v_t \circ \varphi_t$

$v_t \in V, \forall t\in [0,1]$

Anton François - 23/05/2023

$G$ is a group of diffeomorphisms

The deformation defined from the ordinary differential equation:

$\partial_t \varphi_t = v_t \circ \varphi_t; \quad \varphi_0 = \mathrm{Id}$

with $v_t \in V,\forall t\in [0,1]$ , is a diffeomorphism. We note $\varphi^v \in G$ such a diffeomorphism.

$G$ := Space of Deformations

Deformation as a flow of vector field

Anton François - 23/05/2023

$\partial_t \varphi_t = v_t \circ \varphi_t; \quad \varphi_0 = \mathrm{Id}$

Image transport (advection): $\partial_t I_t = v_t \cdot \nabla I_t$

Deformed image

$I_t = I_0 \circ (\varphi^v)^{-1}$

The deformation acts on images.

Geodesic := Shortest path for the exact matching:

$\mathrm{inf}_v \int_0^1 \|v_t\|_V^2 dt$

Anton François - 23/05/2023

LDDMM can reach only images with same topology

Metamorphosis deforms images and adds intensity.

Making the registration of images B and C possible.

$\partial_t I_t = v_t \cdot \nabla I_t$

$+ \mu z_t$

Topology and appearances variations

Anton François - 23/05/2023

Metamorphosis

A Metamorphosis on $\mathcal I$ is a pair of curves $(\varphi_t, \psi_t)$ , respectively on $G$ and $\mathcal I$ , with $\varphi_0 = \mathrm{Id}$ .

$G$ := Space of Deformations

$\mathcal I$ := Space of images

$\psi_t$ is the intensity changes evolution part: $I_t = \psi_t \circ (\varphi_t)^{-1}$

Anton François - 23/05/2023

Metamorphosis

$\left\{\begin{array}{rl}v_t &= -\frac{\rho}{\mu} K_\sigma \star (z_t \nabla I_t)\\ \dot z_t &= -\quad \nabla \cdot (z_t v_t) \\ \dot I_t &= -\langle v_t , \nabla I_t\rangle + \mu z_t\end{array}\right.$

Advection equation with source

Continuity equation

By computing the Euler-Lagrange equation of the exact matching cost :

and doing the variation with respect to $I$ and $v$ we obtain this set of geodesic equations:

[Trouvé & Younes, 2005]

$E_\mathrm M (I,v) = \frac12\int_0^1 \|v_t\|_V^2 + \rho \|z_t\|_{L_2}^2 dt \quad s.t.\ I_0 = S, I_1 = T$

Anton François - 23/05/2023

Metamorphosis Optimisation

via Geodesic shooting

$\left\{\begin{array}{rl}v_t &= -\frac{\rho}{\mu} K_\sigma \star (z_t \nabla I_t)\\ \partial_t z_t &= -\quad \nabla \cdot (z_t v_t) \\ \partial_t I_t &= -\langle v_t , \nabla I_t\rangle + \mu z_t\end{array}\right.$

We minimize the inexact matching cost :
$H_\mathrm M (I,v) = \frac1 2 \left\| I_1 - T \right\|_2^2 + \frac \lambda 2\int_0^1 \|v_t\|_V^2 + \rho \|z_t\|_{L_2}^2 dt$

Geodesic Integration

$z_0$

$I_1$

Step 1:

Step 2:

We iterate over two steps:

Anton François - 23/05/2023

We minimize this cost :
$H_\mathrm M (I,v) = \frac1 2 \left\| I_1 - T \right\|_2^2 + \frac \lambda 2\int_0^1 \|v_t\|_V^2 + \rho \|z_t\|_{L_2}^2 dt$

$\Leftrightarrow H_\mathrm M (z_0) = \frac1 2 \left\| I_1 - T \right\|_2^2 + \frac \lambda 2\left( \|z_0\nabla I_0\|_V^2 + \rho \|z_0\|_{L_2}^2\right)$

$\nabla H_M$ and adjoint equations computed

with auto differentiation.

$v_0 = - \frac \rho \mu K\star (z_0 \nabla I_0)$

$\|v_0\|_V+ \rho \|z_0\|_{L^2} = \|v_t\|_V +\rho \|z_t\|_{L^2} ,\forall t\in [0,1]$

Metamorphosis Optimisation

via Geodesic shooting

Anton François - 23/05/2023

Lagrangian Formulation

$x'(t) = v(t,x)$

Lagrangian scheme

Not suitable for images

Eulerian scheme

$I_{t+\delta_t} = I_t - \delta_t(v_t \times\nabla I_t)$

Eulerian Formulation

$\partial_t I(t,x) = - v(t,x) \cdot \nabla I(t,x)$

Integration of the geodesics equations

Anton François - 23/05/2023

Semi-Lagrangian scheme principle

$\partial_t I_t = v_t \cdot \nabla I_t + \mu z_t$

Semi-Lagrangian scheme

$I_{t+\delta_t} = \mathrm{Interp}(I_t,\varphi^{v_t}) + \delta_t \mu z_t$

Anton François - 23/05/2023

Schemes stability

Stable image

Stable residual

Semi-Lagrangian scheme on residual

$z_{t+\delta_t} = \mathrm{Interp}(z_t,\varphi^{v_t}) - \delta_t z_t \mathrm{div}(v_t)$

$\partial_t z_t = - \mathrm{div} (z_t v_t)$

- Full semi-Lagrangian -

Anton François - 23/05/2023

Problem solved ?

$\rho$ and $\mu$ control ratio between intensity changes and deformation

small intensity changes

big intensity changes

Anton François - 23/05/2023

Weighted Metamorphosis

$\left\{\begin{array}{rl}v_t &= -\frac{\rho}{\mu} K_\sigma \star (z_t \nabla I_t)\\ \dot z_t &= \quad - \nabla \cdot (z_t v_t) \\ \dot I_t &= - v_t \cdot \nabla I_t + \mu M_t z_t\end{array}\right.$

Let $(M_t)_{t\in [0,1]}$ be a continuous temporal mask. By computing the Euler-Lagrange equation of the exact matching cost :

$E_{\mathrm{WM}}(I,v) = \frac 12\int_0^1\|v_t\|_V^2 + \rho\langle z_t, M_t z_t \ \ \rangle_{L^2} dt \quad s.t.\ I_0 = S, I_1 = T$
and doing the variation with respect to $I$ and $v$ we obtain this set of geodesic equations:

$\langle z_t, M_t z_t \rangle_{L^2}$

$M_t$

Anton François - 23/05/2023

Weighted Metamorphosis

with a growing mask.

1. We set $M_1$ as the topological difference segmentation.

2. We initialise $M_0$ as a small ball at its center.

3. We register $M_0$ to $M_1$ using LDDMM.

Simple and realistic prior modelling:

Anton François - 23/05/2023

$M_t$ : Adding intensities

Oriented Metamorphosis

$P_t$ : Indicates where the vector field should be followed

$w_t$ : vector field to follow

We want to force the registration to follow the LDDMM field.

Anton François - 23/05/2023

Constrained Metamorphosis

Let $(M_t)_{t\in [0,1]}$ and $(P_t)_{t\in [0,1]}$ be two continuous temporal masks and $w_t \in V$ , an admissible vector field.

By computing the Euler-Lagrange equation of the exact matching cost:
$E_{\mathrm{CM}}(I,v) = \int_0^1 \|v_t\|_V^2 + \rho \langle z_t, M_t z_t \rangle_{L^2} +\gamma\| P_t(v_t - w_t) \|_V^2 dt$

Oriented & Weighted

$s.t., I_0 = S, I_1 = T$

$\| P_t(v_t - w_t) \|_V^2$

Anton François - 23/05/2023

Constrained Metamorphosis

Let $(M_t)_{t\in [0,1]}$ and $(P_t)_{t\in [0,1]}$ be two continuous temporal masks and $w_t \in V$ , an admissible vector field.

By computing the Euler-Lagrange equation of the exact matching cost:
$E_{\mathrm{WM}}(I,v) = \int_0^1 \|v_t\|_V^2 + \rho \langle z_t, M_t z_t \rangle_{L^2} +\gamma\| P_t(v_t - w_t) \|_V^2 dt$

Oriented & Weighted

$s.t., I_0 = S, I_1 = T$

$\| P_t(v_t - w_t) \|_V^2$

Oriented Norm:

We seak to estimate $v_t$ similar to $w_t$ where the mask $P_t$ is positive.

Anton François - 23/05/2023

Constrained Metamorphosis

Let $(M_t)_{t\in [0,1]}$ and $(P_t)_{t\in [0,1]}$ be two continuous temporal masks and $w_t \in V$ , an admissible vector field.

By computing the Euler-Lagrange equation of the exact matching cost:
$E_{\mathrm{CM}}(I,v) = \int_0^1 \|v_t\|_V^2 + \rho \langle z_t, M_t z_t \rangle_{L^2} +\gamma\| P_t(v_t - w_t) \|_V^2 dt$
and doing the variation with respect to $I$ and $v$ we obtain this set of geodesic equations:

Oriented & Weighted

$s.t., I_0 = S, I_1 = T$

Theorem:

$\left\{\begin{array}{rl} v_t &= - \frac{\rho}{\mu (1 + \gamma P_t)} K \star (z_t\nabla I_t) + \frac{\gamma P_t }{1 + \gamma P_t} w_t\\ \dot z_t &= -\quad \nabla \cdot (z_t v_t) \\ \dot I_t &= - v_t , \cdot\nabla I_t + \mu M_t z_t\end{array}\right.$

$\frac{\rho}{\mu(1+\gamma P_t)}$

$\frac{\gamma P_t}{1+\gamma P_t}$

Anton François - 23/05/2023

$I_t$

$I_t$ & $\varphi^{v_t}$

$z_t$

Constrained Metamorphosis

Results on ToyExamples

Anton François - 23/05/2023

A Diffeomorphic Atlas represents shape variations, it is a Template along with registrations from the template to each data points (images)

Given a sequence of images $I^1,\dots,I^n$ and a notion of distance $d$ corresponding to the length of a geodesic path in shape space. We compute the template $T$ by minimising:

$\mathcal M(T) = \frac{1}{2n} \sum_{k=1}^{n} d(T,I^k)^2$

Speculations about atlases.

Can we build a Metamorphic atlas ?

How to estimate $T$ for a glioma data base?

A combination of a Statistical Atlas and a diffeomorphic shape space.

Anton François - 23/05/2023

Can we build a CM atlas ?

Not as it is: Because it is prior dependent, one can not define a single metric for a whole data set

Technically speaking, yes but there are interpretability issues.

Diffeomorphic image registration taking topological differences into account Metamorphosis on brain MRI containing Glioblastoma Anton François Supervised by Joan Glaunès & Pietro Gori

Diffeomorphic image registration taking topological differences into account

Introduction

With modern methods

With modern methods

Metamorphosis

Metamorphosis

Constrained

Metamorphosis

Weighted Metamorphosis

Weighted Metamorphosis

Weighted Metamorphosis

Constrained Metamorphosis

Constrained Metamorphosis

Constrained Metamorphosis

Constrained Metamorphosis

Constrained Metamorphosis

TDA Glioblastoma

Segmentation

Conclusion & Perspective

Published Articles: