Simulation-based inference
= Likelihood-free inference, implicit inference
Why?
- Likelihoods are rarely gaussian, specially true for non-two point functions
- Large parameter spaces -> MCMC hugely expensive
Density estimation
How?
θ
\theta
Simulated Data
Data
Prior
Posterior
xobs
x_\mathrm{obs}
x
x
P(θ∣xobs)
P(\theta|x_\mathrm{obs})
Forwards
Inverse
Inference = Optimisation
L(D)
\mathcal{L}(\mathcal{D})
L(D∣θ)
\mathcal{L}(\mathcal{D}|\theta)
P(θ∣D)
\mathcal{P}(\theta|\mathcal{D})
MCMC or another density estimator
Can also estimate posterior directly!
Density estimator
x=fϕ(z)
x = \green{f_\phi}(z)
L(D)=−∣D∣1∑x∈Dlogp(x)
\mathcal{L}(\mathcal{D}) = - \frac{1}{\vert\mathcal{D}\vert}\sum_{\mathbf{x} \in \mathcal{D}} \log p(\mathbf{x})
Maximize the data likelihood
NeuralNet
Normalising flows
p(x)=pz(z)detdxdz=pz(f−1(x))detdxdf−1
p(\mathbf{x}) = p_z(\mathbf{z}) \left\vert \det \dfrac{d \mathbf{z}}{d \mathbf{x}} \right\vert
= p_z(f^{-1}(\mathbf{x})) \left\vert \det \dfrac{d f^{-1}}{d \mathbf{x}} \right\vert
f must be invertible
J efficient to compute
1-D
n-D
p(x)=pz(f−1(x))detJ(f−1)
p(\mathbf{x}) = p_z(f^{-1}(\mathbf{x})) \left\vert \det J(f^{-1}) \right\vert
x=f(z),z=f−1(x)
x = f(z), \, z = f^{-1}(x)
p(x)=pz(z)detdxdz
p(\mathbf{x}) = p_z(\mathbf{z}) \left\vert \det \dfrac{d \mathbf{z}}{d \mathbf{x}} \right\vert
p(x∣θ)=pz(f−1(x∣θ))detJ(f−1∣θ)
p(\mathbf{x}|\theta) = p_z(f^{-1}(\mathbf{x}|\theta)) \left\vert \det J(f^{-1}|\theta) \right\vert
Sequentially improve
Sample from
p(G∣C,xobs)
p(\mathcal{G}|\mathcal{C}, x_\mathrm{obs})
Refine accuracy on HOD parameters close to the data
Issue -> we estimate
p(G,C∣xobs)
p(\mathcal{G},\mathcal{C}| x_\mathrm{obs})
Targeting the posterior or the likelihood?
Pros posterior
- Fast sampling (no need MCMC)
Middle ground: Likelihood + Variational Inference
Pros likelihood
- combine multiple observations
- use without retraining if the prior is changed
- easier to adapt to sequential models
Implicit Likelihood vs Mean emulators
- No likelihood assumption -> Requires simulations with different seeds
- Assume Gaussian likelihood and estimate the covariance matrix from a different set of simulations
- Learn mean relation -> fixed seed simulations useful
- Learn likelihood or posterior directly
S(X∣θ)
S(X|\theta)
P(θ∣S(X))
P(\theta|S(X))
Hybrid for Abacus Summit
{C,G}
\{\mathcal{C}, \mathcal{G} \}
fϕ
f_\phi
S
S
L(S∣C,D)
\mathcal{L}(S|\mathcal{C}, \mathcal{D})
Gaussian or estimated from fixed cosmology
Density estimator / Variational Inference
p(C,D∣S)
p(\mathcal{C}, \mathcal{D}|S)
fϕ′
f^\prime_\phi
Loss
Loss
Loss (MSE)
Simulation-based inference = Likelihood-free inference, implicit inference