Aim 3: Posterior Sampling and Uncertainty
June 6, 2023
Diffusion schema
At every iteration \(\{t_n = 1 > t_{n-1} > \dots > t_1 > t_0 =0 \}\):
1. \(\{\hat{\mu}_{t_i}\}\) is the unconditional diffusion process
2. \(\{\hat{y}_{t_i}\}\) are the diffused noisy observations such that
\[\hat{y}_{t_i} = e^{-A\beta(t_i)z}y,~z \sim \mathcal{N}(0, \mathbb{I})\]
\[y = Be^{-Au} + v,~v \sim \mathcal{N}(0, \Sigma)\]
\[u \in \mathbb{R}^n,~y \in \mathbb{R}^m,B \in \mathbb{R}^{m \times m},~A \in \mathbb{R}^{m \times n}\]
Then, to sample from \(p(y \mid x)\):
\[\hat{\mu}'_{t_i} = \texttt{CONSISTENCY}(\hat{\mu}_{t_i}, \hat{y}_{t_i}, \lambda)\]
\[\hat{\mu}_{t_{i-1}} = \texttt{REVERSE}(\hat{\mu}'_{t_i}, s(\hat{\mu}_{t_i}, t_i))\]
\(\texttt{CONSISTENCY}(\hat{\mu}_{t_i}, \hat{y}_{t_i}, \lambda)\)
\[\hat{\mu}'_{t_i} = \text{arg~min}_{\mu \in \mathbb{R}^n} \left\{(1-\lambda) \|\mu - \hat{\mu}_{t_i}\|^2 + \lambda \min_{v \in \mathbb{R}^n} \|v - \mu\|^2\right\}\]
\[~\text{s.t.}~Be^{-Av} = \hat{y}_{t_i}\]
\(\texttt{CONSISTENCY}(\hat{\mu}_{t_i}, \hat{y}_{t_i}, \lambda)\)
\[v^* = \text{arg~min}_{v \in \mathbb{R}^n} \frac{1}{2}\|v - \mu\|^2~\quad\text{s.t.}~Be^{-Av} = \hat{y}_{t_i}\]
\[\downarrow\]
\[v^*= \mu - A^{\dagger}(A\mu - g)\]
\[A^{\dagger} = A^T(AA^T)^{-1},~g = -\ln(B^{-1}\hat{y}_{t_i})\]
and
\[\min_{v \in \mathbb{R}^{n}} \|v - \mu\|^2 = \|v^* - \mu\|^2 = \|A^{\dagger}(A\mu -g)\|^2\]
\[\hat{\mu}'_{t_i} = \text{arg~min}_{\mu \in \mathbb{R}^n} \left\{(1-\lambda) \|\mu - \hat{\mu}_{t_i}\|^2 + \lambda \min_{v \in \mathbb{R}^n} \|v - \mu\|^2\right\}\]
\[~\text{s.t.}~Be^{-Av} = \hat{y}_{t_i}\]
\(\texttt{CONSISTENCY}(\hat{\mu}_{t_i}, \hat{y}_{t_i}, \lambda)\)
\[\hat{\mu}'_{t_i} = \text{arg~min}_{\mu \in \mathbb{R}^n} (1 - \lambda) \|\mu - \hat{\mu}_{t_i}\|^2 + \lambda \|A^{\dagger}(A\mu - g)\|^2\]
\[\downarrow\]
\[[(1-\lambda)\mathbb{I} + \lambda\tilde{A}^T\tilde{A}]\hat{\mu}'_{t_i} = (1-\lambda)\hat{\mu}_{t_i} + \lambda\tilde{A}^T\tilde{g}\]
where
\[\tilde{A} = A^{\dagger}A,~\tilde{g} = A^{\dagger}g\]