
ML for Scientists: L14
Carol(ina) Cuesta-Lazaro

https://tinyurl.com/moneyplot
https://meet.google.com/sqh-ttyp-rqt
Do we actually need Density Estimation?
Just use binary classifiers!

Binary cross-entropy
Sample from simulator
Mix-up
Likelihood-to-evidence ratio
Likelihood-to-evidence ratio
NLE
No implicit prior
Not amortized
Goodness-of-fit
Scaling with dimensionality of x
NPE
NRE
Amortized
Scales well to high dimensional x
Implicit marginalization
No need variational distribution
No implicit prior
Implicit marginalization
Approximately normalised
Not amortized
Implicit marginalization
Goodness-of-fit?
Robustness?
Fixed prior










How good is your posterior?
Test log likelihood
["Benchmarking simulation-based inference"
Lueckmann et al
arXiv:2101.04653]
Posterior predictive checks

Observed
Re-simulated posterior samples
Classifier 2 Sample Test (C2ST)



Real or Fake?
Benchmarking SBI
["Benchmarking simulation-based inference"
Lueckmann et al
arXiv:2101.04653]

Classifier 2 Sample Test (C2ST)
["A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful" Hermans et al
arXiv:2110.06581]

Much better than overconfident!
Coverage: assessing uncertainties
["A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful" Hermans et al
arXiv:2110.06581]
Credible region (CR)
Not unique
High Posterior Density region (HPD)
Smallest "volume"

True value in CR with
probability

Empirical Coverage Probability (ECP)

["Investigating the Impact of Model Misspecification in Neural Simulation-based Inference" Cannon et al arXiv:2209.01845 ]
Underconfident
Overconfident
Calibrated doesn't mean informative!
Always look at information gain too
["A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful" Hermans et al
arXiv:2110.06581]

["Calibrating Neural Simulation-Based Inference with Differentiable Coverage Probability" Falkiewicz et al
arXiv:2310.13402]
["A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful" Hermans et al
arXiv:2110.06581]
Model mispecification

["Investigating the Impact of Model Misspecification in Neural Simulation-based Inference" Cannon et al arXiv:2209.01845]
More misspecified

Sequential SBI

["Benchmarking simulation-based inference"
Lueckmann et al
arXiv:2101.04653]

[Image credit: https://www.mackelab.org/delfi/]Sequential SBI
Real life scaling: Gravitational lensing
["A Strong Gravitational Lens Is Worth a Thousand Dark Matter Halos: Inference on Small-Scale Structure Using Sequential Methods" Wagner-Carena et al arXiv:2404.14487]


Generative Priors: Plug and Play
EHT posterior samples with different priors
["Event-horizon-scale Imaging of M87* under Different Assumptions via Deep Generative Image Priors" Feng et al]

CIFAR-10
GRMHD
RIAF
CelebA
(Sims)
(Sims)

(LR Natural Images)
(Human Faces)

Prior

Learned Noise Models
Money plot - Nov 18th -> Discuss in class Nov 19th

ML For Scientists: L14
By carol cuesta
ML For Scientists: L14
- 102