desilike
Current status and prospects
cosmodesi
- An environment, at NERSC, that contains (most of) the packages (publicly available at https://github.com/cosmodesi) required to run the DESI DR1 (BAO DR2) standard analyses
# Source the environment
source /global/common/software/desi/users/adematti/cosmodesi_environment.sh main
# You may want to have it in the jupyter-kernel for plots
${COSMODESIMODULES}/install_jupyter_kernel.sh main # this to be done once
cosmodesi
- An environment, at NERSC, that contains (most of) the packages (publicly available at https://github.com/cosmodesi) required to run the DESI DR1 (BAO DR2) standard analyses
-
Contains (w/ many Python packages, inc. pytorch, tensorflow, jax):
- cosmoprimo: primordial cosmology (class, camb, + MG/EDE variations, fftlog, interpolator, BAO filtering)
- pycorr: correlation function estimation
- pypower: power spectrum (and window function) estimation
- thecov: power spectrum covariance
- pyrecon: standard BAO reconstruction
- mockfactory: tools to be build fast mocks
- desilike: DESI likelihoods
- desipipe: pipelining tools
-
Cobaya, CosmoSIS (standard library), CLASS, Planck likelihoods, etc.
-
scripts: desi-y1-kp, desi-y3-kp, RascalC-scripts
desilike
- Goal #1: facilitate BAO / fNL / Full Shape analyses (running on many mocks, on-the-fly emulation, comparing theory codes, sharing scripts/results)
- Goal #2: automatic bindings for Cobaya, MontePython, CosmoSIS
- Goal #3: integrate non-KP analyses, e.g.
- project 342 (H0 from radiation == matter scale)
- project 353 (turnover scale)
- project 377 (BAO phase shift)
- ...
Also, by design, easy extensions to other observables than P(k)/ξ(s)
desilike - in a nutshell
To specify the likelihood for power spectrum or correlation function multipoles, just choose:
-
template (cosmo: DirectPowerSpectrumTemplate, BAO: BAOPowerSpectrumTemplate, ShapeFit: ShapeFitPowerSpectrumTemplate, WiggleSplitPowerSpectrumTemplate)
-
theory (KaiserTracerPowerSpectrumMultipoles, LPTVelocileptorsTracerPowerSpectrumMultipoles (plus other flavors), FOLPSTracerPowerSpectrumMultipoles, etc.)
-
observable (TracerPowerSpectrumMultipolesObservable, TracerCorrelationFunctionMultipolesObservable, ...)
define a likelihood
from desilike.theories.galaxy_clustering import ShapeFitPowerSpectrumTemplate,\
FOLPSTracerPowerSpectrumMultipoles
from desilike.observables.galaxy_clustering import TracerPowerSpectrumMultipolesObservable
from desilike.likelihoods import ObservablesGaussianLikelihood
template = ShapeFitPowerSpectrumTemplate(z=0.8)
theory = FOLPSTracerPowerSpectrumMultipoles(template=template)
theory.init.params['b1'].update(prior=dict(dist='norm', loc=1.5, scale=0.5)) # update theory parameters
observable = TracerPowerSpectrumMultipolesObservable(klim={0: [0.02, 0.2, 0.005], 2: [0.02, 0.2, 0.005]},
data='path_to_data',
covariance='path_to_covariance',
wmatrix='path_to_window_matrix',
theory=theory)
# Here we could give a list of several observables + joint covariance matrix
likelihood = ObservablesGaussianLikelihood(observables=[observable])
for param in likelihood.all_params.select(basename=['alpha*', 'sn*']):
param.update(derived='.auto') # analytic marginalization for linear parameters
# We can also sum up independent likelihoods (likelihood1 + likelihood2)
# Call likelihood
likelihood(qpar=0.99, qper=1.01) # log-posterior
define a likelihood
from desilike.theories.galaxy_clustering import ShapeFitPowerSpectrumTemplate,\
FOLPSTracerPowerSpectrumMultipoles
from desilike.observables.galaxy_clustering import TracerPowerSpectrumMultipolesObservable
from desilike.likelihoods import ObservablesGaussianLikelihood
template = ShapeFitPowerSpectrumTemplate(z=0.8)
theory = FOLPSTracerPowerSpectrumMultipoles(template=template)
theory.init.params['b1'].update(prior=dict(dist='norm', loc=1.5, scale=0.5)) # update theory parameters
observable = TracerPowerSpectrumMultipolesObservable(klim={0: [0.02, 0.2, 0.005], 2: [0.02, 0.2, 0.005]},
data='path_to_data',
covariance='path_to_covariance',
wmatrix='path_to_window_matrix',
theory=theory)
# Here we could give a list of several observables + joint covariance matrix
likelihood = ObservablesGaussianLikelihood(observables=[observable])
for param in likelihood.all_params.select(basename=['alpha*', 'sn*']):
param.update(derived='.auto') # analytic marginalization for linear parameters
# We can also sum up independent likelihoods (likelihood1 + likelihood2)
# Call likelihood
likelihood(qpar=0.99, qper=1.01) # log-posterior
ObservablesGaussianLikelihood(observables=[power_spectrum, bispectrum, gk, kk, ...],
covariance=big_total_covariance)
param.update(namespace='kk')
namespace scheme:
emulate on-the-fly
from desilike.emulators import Emulator, TaylorEmulatorEngine, EmulatedCalculator
# theory.pt = perturbation theory part, only depends on template parameters (qpar, qper, df, dm)
emulator = Emulator(theory.pt, engine=TaylorEmulatorEngine(method='finite', order=4))
emulator.set_samples()
emulator.fit()
pt = emulator.to_calculator()
# emulator.save(emulator_fn); pt = EmulatedCalculator.load(emulator_fn)
# Replace perturbation theory part by new one!
theory.init.update(pt=pt)
profile / sample
from desilike.samplers import EmceeSampler
sampler = EmceeSampler(likelihood, chains=8, nwalkers=40, seed=42, save_fn='chain_*.npy')
chains = sampler.run(min_iterations=2000, check={'max_eigen_gr': 0.03})
### Re-load chains, concatenate them
from desilike.samples import Chain
chain = Chain.concatenate([Chain.load('chain_{:d}.npy'.format(i)).remove_burnin(0.5)[::10]
for i in range(8)])
# To turn into GetDist samples
chain.to_getdist()
from desilike.profilers import MinuitProfiler
profiler = MinuitProfiler(likelihood, seed=42, save_fn='profiles.npy')
profiler.maximize(niterations=10) # optimization
profiler.profile(params=['qpar']) # 1D profile
profiler.contour(params=['qpar']) # 2D contours
Tip: with MPI! (also: JIT-compilation + gradient if likelihood is JAX-friendly)
Tip: with MPI! (also: many other samplers, inc. HMC, NUTS)
also (in progress) cosmological inference
from desilike.theories import Cosmoprimo
from desilike.likelihoods.cmb import TTTEEEHighlPlanckNPIPECamspecLikelihood
from desilike.likelihoods.supernovae import Union3SNLikelihood
cosmo = Cosmoprimo(fiducial='DESI', engine='capse') # emulator by Marco Bonici
likelihood = TTTEEEHighlPlanckNPIPECamspecLikelihood(cosmo=cosmo)
likelihood += Union3SNLikelihood(cosmo=cosmo) # in JAX
sampler = NUTSSampler(likelihood, seed=42)
samples = sampler.run()
Materials
Tutorials in https://github.com/cosmodesi/cosmodesi-tutorials, https://github.com/cosmodesi/desilike-tutorials
desilike docs, especially getting started
Practical applications: look at them to see how you can benefit from MPI!
application to mock challenge (velocileptors, pybird, folps)
application to mock Y1 cosmological inference
application to real Y1 cosmological inference
application to DESI Y5/Y1 forecasts (Uendert Andrade)
application to blinding validation (Uendert Andrade & Edmond Chaussidon for fNL)
application to tests of fiber collisions (Mathilde Pinon)
In progress
- Unit tests, PIP versioning - Johannes Lange
- Extension to bispectrum - Daniel Forero-Sanchez, myself
- Cosmological inference on the GPU - Marco Bonici, myself
ToDo's
- More JAX-friendliness (à la equinox, bindings with numpyro) - myself
- More tests, docs - Johannes Lange
Why include x lensing?
- We can, so why not do it? It shouldn't be difficult if you have codes in place (~1 week work?)
- Then you can try it - if you're not happy with it, it's fine too!
desilike_c3_April2025
By Arnaud De Mattia
desilike_c3_April2025
- 92