Github Repository
\text{Github Repository}
Publication
\text{Publication}
Unsupervised Image Denoising with Parametric Noise Models
\text{Unsupervised Image } \newline
\text{Denoising with Parametric} \newline \text{ Noise Models}
Manan Lalit, Mangal Prakash, Pavel TomancakAlex Krull, Florian Jug
\text{\textbf{Manan Lalit}, Mangal Prakash, Pavel Tomancak}
\newline
\text{Alex Krull, Florian Jug}
Recap
\text{Recap}
Noisy ImageStack (Input)
\text{Noisy Image}
\newline
\text{Stack (\textit{Input})}
Clean GT Stack (Target)
\text{Clean GT }
\newline
\text{Stack (\textit{Target})}
Weigert et al, Content Aware Image restoration: Pushing the limits of Fluoroscence Microscopy, Nature Methods 15, pages 1090–1097 (2018)
Noise2Void
\text{Noise2Void}
Publication
\text{Publication}
Alex Krull
\text{Alex Krull}
Tim-Oliver Buchholz
\text{Tim-Oliver Buchholz}
Florian Jug
\text{Florian Jug}
Krull and Buchholz et al, Noise2Void - Learning Denoising From Single Noisy Images, CVPR 2019
Krull et al, Probabilistic Noise2Void: Unsupervised Content-Aware Denoising, Frontiers in Computer Science, 2020
Tomaˊsˇ Vicˇar
\text{Tomáš Vičar}
Supervised Image Denoising (e.g. CARE)
\text{Supervised Image Denoising (e.g. CARE)}
Publication
\text{Publication}
Alex Krull
\text{Alex Krull}
Florian Jug
\text{Florian Jug}
Pavel Tomancak
\text{Pavel Tomancak}
Mangal Prakash
\text{Mangal Prakash}
| 18958 |
|---|
| 20758 |
| 20701 |
| 22378 |
| 18473 |
| 20397 |
| 19523 |
| 18295 |
| ... |
| 21845 |
| 20295.52 |
|---|
100
8
7
6
5
4
3
2
1
1
# pixels
# pixels
1
Extracting Noise Model from Calibration Samples
\text{Extracting Noise Model from Calibration Samples}
100
1
2
3
99
98
| 2930 |
|---|
| 2721 |
| 3369 |
| 3061 |
| 2369 |
| 3245 |
| 2987 |
| 2809 |
| ... |
| 3210 |
| 3004.32 |
|---|
Noisy observations(x)
\text{Noisy observations} \left( x \right)
1
2
3
4
5
6
7
8
100
P(x∣s=3004.32)=?
P(x|s=3004.32)=?
P(x∣s∈[2978,3006))=?
P \left(x|s \in [2978, 3006) \right)=?
P(xbin∣s∈[2978,3006))=?
P \left(x_{\text{bin}}|s \in [2978, 3006) \right)=?
Counts
\text{Counts}
Investigating the Noisy Calibration Pixels
\text{Investigating the Noisy Calibration Pixels}
P(x∣s)
P(x|s)
Noisy observation(x)bins
\text{Noisy observation} \left( x \right) \text{bins}
Noisy observation(x)bins
\text{Noisy observation} \left( x \right) \text{bins}
Clean signal(s)bins
\text{Clean signal} \left( s \right) \text{bins}
Binning the Noise Model
\text{Binning the Noise Model}
Histogram-based Noise Model has some drawbacks ...
\text{Histogram-based Noise Model has some drawbacks ...}
100
99
98
3
2
1
Noisy observation(x)bins
\text{Noisy observation} \left( x \right) \text{bins}
Clean signal(s)bins
\text{Clean signal} \left( s \right) \text{bins}
Histogram-based Noise Model has some drawbacks ...
\text{Histogram-based Noise Model has some drawbacks ...}
p(xi∣si)=∑k=1Kαkf(xi;μk,σk2)
p(x_i|s_i) = \sum_{k=1}^{K} \alpha_{k} f\big( x_i; \mu_{k},\sigma^{2}_{k} \big)
p(xi∣si)=∑k=1Kαk(si)f(xi;μk(si),σk2(si))
p(x_i|s_i) = \sum_{k=1}^{K} \alpha_{k}(s_{i}) f\big( x_i; \mu_{k} (s_{i}),\sigma^{2}_{k} (s_{i}) \big)
θ^=a^,b^,…=arg maxθ∑i,jlogp(xij∣si)
\hat{\theta} = \hat{a}, \hat{b}, \ldots = \text{arg max}_{\theta} \sum_{i,j} \log p \big( x_{i}^{j}| s_{i})
xi:noisy observation
x_{i} : \text{noisy observation}
si:ground truth signal
s_{i} : \text{ground truth signal}
K:number of gaussians
K: \text{number of gaussians}
αk:weight of gaussian k
\alpha_{k} : \text{weight of gaussian $k$}
μk:mean of gaussian k
\mu_{k} : \text{mean of gaussian $k$}
σk2:variance of gaussian k
\sigma_{k}^{2} : \text{variance of gaussian $k$}\\
μk(si)=a+b×si+c×si2+…
\mu_{k} \left( s_{i} \right) = a + b \times s_{i} + c \times s_{i}^{2} + \ldots\\
σk(si)=…
\sigma_{k} \left( s_{i} \right) = \ldots
αk(si)=…
\alpha_{k} \left( s_{i} \right) = \ldots
θ
\theta
}
\}
Could we use a Parametric Noise Model?
\text{Could we use a Parametric Noise Model?}
Calibration Data
\text{Calibration Data}
Ablation Study Results
\text{Ablation Study Results}
Image to be denoised
\text{Image to be denoised}
Input zoom
\text{Input zoom}
PN2V GMM
\text{PN2V GMM
}
Noisy Observation
\text{Noisy Observation}
Pseudo Ground Truth
\text{Pseudo Ground Truth}
PN2V Prediction
\text{PN2V Prediction}
PN2V
\text{PN2V}
N2V
\text{N2V}
Hist/GMMbased Noise Model
\text{Hist/GMM} \newline
\text{based } \newline
\text{Noise Model}
But what if calibration data is not available ...
\text{But what if calibration data is not available ...
}
Boot. GMM & Boot. Hist.
\text{Boot. GMM \& Boot. Hist.}
Github Repository
\text{Github Repository}
Future Directions
\text{Future Directions
}
Application of PN2V GMM & Boot. GMM to a diversity of biological data
\text{Application of \textbf{PN2V GMM} \& \textbf{Boot. GMM}}
\newline
\text{ to a diversity of biological data}
Joint estimation of noise model and PN2V traininginstead of the current sequential route
\text{\textbf{Joint} estimation of noise model and \textbf{PN2V} training}
\newline
\text{instead of the current \textit{sequential} route}
Acknowledgements
\text{Acknowledgements}
Thank you for listening!Any questions?
\text{Thank you for listening!}
\newline
\text{Any questions?}
Alex Krull
\text{Alex Krull}
Matthias Arzt
\text{Matthias Arzt}
Tim-Oliver Buchholz
\text{Tim-Oliver Buchholz}
Mangal Prakash
\text{Mangal Prakash}
Anna Goncharova
\text{Anna Goncharova}
Nuno Martins
\text{Nuno Martins}
Tobias Pietzsch
\text{Tobias Pietzsch}
Deborah Schmidt
\text{Deborah Schmidt}
Florian Jug
\text{Florian Jug}
Pavel Tomancak
\text{Pavel Tomancak}
Gabriella Turek
\text{Gabriella Turek}
Marina Cuenca
\text{Marina Cuenca}
Giulia Serafini
\text{Giulia Serafini}
Vladimir Ulman
\text{Vladimir Ulman}
Bruno Vellutini
\text{Bruno Vellutini}
Johannes Girstmair
\text{Johannes Girstmair}
Mette Thorsager
\text{Mette Thorsager}
G i t h u b R e p o s i t o r y \text{Github Repository} P u b l i c a t i o n \text{Publication} U n s u p e r v i s e d I m a g e D e n o i s i n g w i t h P a r a m e t r i c N o i s e M o d e l s \text{Unsupervised Image } \newline
\text{Denoising with Parametric} \newline \text{ Noise Models} M a n a n L a l i t , M a n g a l P r a k a s h , P a v e l T o m a n c a k A l e x K r u l l , F l o r i a n J u g \text{\textbf{Manan Lalit}, Mangal Prakash, Pavel Tomancak}
\newline
\text{Alex Krull, Florian Jug}
ISBI 2020
By Manan Lalit
ISBI 2020
Presentation at ISBI conference
- 1,033
