Aim 3: Posterior Sampling and Uncertainty

June 6, 2023

Diffusion schema

At every iteration \(\{t_n = 1 > t_{n-1} > \dots > t_1 > t_0 =0 \}\):

1. \(\{\hat{\mu}_{t_i}\}\) is the unconditional diffusion process

2. \(\{\hat{y}_{t_i}\}\) are the diffused noisy observations such that

\[\hat{y}_{t_i} = e^{-A\beta(t_i)z}y,~z \sim \mathcal{N}(0, \mathbb{I})\]

\[y = Be^{-Au} + v,~v \sim \mathcal{N}(0, \Sigma)\]

\[u \in \mathbb{R}^n,~y \in \mathbb{R}^m,B \in \mathbb{R}^{m \times m},~A \in \mathbb{R}^{m \times n}\]

Then, to sample from \(p(y \mid x)\):

\[\hat{\mu}'_{t_i} = \texttt{CONSISTENCY}(\hat{\mu}_{t_i}, \hat{y}_{t_i}, \lambda)\]

\[\hat{\mu}_{t_{i-1}} = \texttt{REVERSE}(\hat{\mu}'_{t_i}, s(\hat{\mu}_{t_i}, t_i))\]

\(\texttt{CONSISTENCY}(\hat{\mu}_{t_i}, \hat{y}_{t_i}, \lambda)\)

\[\hat{\mu}'_{t_i} = \text{arg~min}_{\mu \in \mathbb{R}^n} \left\{(1-\lambda) \|\mu - \hat{\mu}_{t_i}\|^2 + \lambda \min_{v \in \mathbb{R}^n} \|v - \mu\|^2\right\}\]

\[~\text{s.t.}~Be^{-Av} = \hat{y}_{t_i}\]

\(\texttt{CONSISTENCY}(\hat{\mu}_{t_i}, \hat{y}_{t_i}, \lambda)\)

\[v^* = \text{arg~min}_{v \in \mathbb{R}^n} \frac{1}{2}\|v - \mu\|^2~\quad\text{s.t.}~Be^{-Av} = \hat{y}_{t_i}\]

\[\downarrow\]

\[v^*= \mu - A^{\dagger}(A\mu - g)\]

\[A^{\dagger} = A^T(AA^T)^{-1},~g = -\ln(B^{-1}\hat{y}_{t_i})\]

and

\[\min_{v \in \mathbb{R}^{n}} \|v - \mu\|^2 = \|v^* - \mu\|^2 = \|A^{\dagger}(A\mu -g)\|^2\]

\(\texttt{CONSISTENCY}(\hat{\mu}_{t_i}, \hat{y}_{t_i}, \lambda)\)

\[\hat{\mu}'_{t_i} = \text{arg~min}_{\mu \in \mathbb{R}^n} (1 - \lambda) \|\mu - \hat{\mu}_{t_i}\|^2 + \lambda \|A^{\dagger}(A\mu - g)\|^2\]

\[\downarrow\]

\[[(1-\lambda)\mathbb{I} + \lambda\tilde{A}^T\tilde{A}]\hat{\mu}'_{t_i} = (1-\lambda)\hat{\mu}_{t_i} + \lambda\tilde{A}^T\tilde{g}\]

where

\[\tilde{A} = A^{\dagger}A,~\tilde{g} = A^{\dagger}g\]