https://tinyurl.com/moneyplot
https://meet.google.com/sqh-ttyp-rqt
Just use binary classifiers!
Binary cross-entropy
Sample from simulator
Mix-up
Likelihood-to-evidence ratio
Likelihood-to-evidence ratio
No implicit prior
Not amortized
Goodness-of-fit
Scaling with dimensionality of x
Amortized
Scales well to high dimensional x
Implicit marginalization
No need variational distribution
No implicit prior
Implicit marginalization
Approximately normalised
Not amortized
Implicit marginalization
Goodness-of-fit?
Robustness?
Fixed prior
Test log likelihood
["Benchmarking simulation-based inference"
Lueckmann et al
arXiv:2101.04653]
Posterior predictive checks
Observed
Re-simulated posterior samples
Real or Fake?
["Benchmarking simulation-based inference"
Lueckmann et al
arXiv:2101.04653]
["A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful" Hermans et al
arXiv:2110.06581]
Much better than overconfident!
["A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful" Hermans et al
arXiv:2110.06581]
Credible region (CR)
Not unique
High Posterior Density region (HPD)
Smallest "volume"
True value in CR with
probability
Empirical Coverage Probability (ECP)
["Investigating the Impact of Model Misspecification in Neural Simulation-based Inference" Cannon et al arXiv:2209.01845 ]
Underconfident
Overconfident
Always look at information gain too
["A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful" Hermans et al
arXiv:2110.06581]
["Calibrating Neural Simulation-Based Inference with Differentiable Coverage Probability" Falkiewicz et al
arXiv:2310.13402]
["A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful" Hermans et al
arXiv:2110.06581]
["Investigating the Impact of Model Misspecification in Neural Simulation-based Inference" Cannon et al arXiv:2209.01845]
More misspecified
["Benchmarking simulation-based inference"
Lueckmann et al
arXiv:2101.04653]
[Image credit: https://www.mackelab.org/delfi/]["A Strong Gravitational Lens Is Worth a Thousand Dark Matter Halos: Inference on Small-Scale Structure Using Sequential Methods" Wagner-Carena et al arXiv:2404.14487]
EHT posterior samples with different priors
["Event-horizon-scale Imaging of M87* under Different Assumptions via Deep Generative Image Priors" Feng et al]
CIFAR-10
GRMHD
RIAF
CelebA
(Sims)
(Sims)
(LR Natural Images)
(Human Faces)
Prior
Money plot - Nov 18th -> Discuss in class Nov 19th