Recent Abacus Simulations for DESI

Lehman Garrison

Research Software Engineer
Scientific Computing Core

Flatiron Institute

October 12, 2023

Local Primordial Non-Gaussianity

  • Local-type primordial non-Gaussianity (PNG) is a modification to the primordial potential of the form
    $$\Phi = \Phi_\mathrm{G} + f_\mathrm{NL}(\Phi_\mathrm{G}^2 - \langle\Phi_\mathrm{G}\rangle^2)$$
  • Local PNG is parameterized by \(f_\mathrm{NL}\)
  • To run an \(f_\mathrm{NL}\) \(N\)-body simulation, the only modification is to the initial conditions
  • Unfortunately, the easiest way to implement this is
    $$\tilde\Phi_\mathrm{G}(\mathbf{k}) \rightarrow \Phi_\mathrm{G}(\mathbf{x}) \rightarrow \Phi_\mathrm{G} + f_\mathrm{NL}\Phi_\mathrm{G}^2 \rightarrow \tilde\Phi(\mathbf{k})$$
  • Then ICs proceed as normal to generate displacements and velocities
  • Main computer cost is an extra backward and forward FFT
  • Main human cost is getting the units right

Implementation in Abacus ICs

  • Implementation of f_NL can be found here: https://github.com/abacusorg/zeldovich-PLT/pull/8
    • There's a failed idea in this PR to skip the extra FFTs via an application of the chain rule
  • Validation was performed on the power spectrum and bispectrum (specifically \(dP/df_\mathrm{NL}\) and \(dB/df_\mathrm{NL}\))
  • Compared against 2LPTPNG (labeled Gadget in the plots), and theory where known (to me)
  • Scale-dependent halo bias also has the right sign

Initial conditions power spectrum comparison

Bispectrum Validation

  • Three bispectrum triangle configurations were tested: equilateral, squeezed, and folded
  • Theory prediction is simple:
    $$B_\Phi(k_1,k_2,k_3) = 2 f_{NL} P_\Phi(k_1)P_\Phi(k_2) + 2\ \rm{cyc.}$$ (from 2206.01619)
  • Abacus, 2LPTPNG, and theory all match for equilateral and folded
  • Abacus and 2LPTPNG match each other for squeezed, but do not match the theory prediction
  • Pylians3 code was used for bispectrum

List of Abacus PNG Simulations

  • 10 simulations
    • 5 f_NL values
    • 2 phases each, ph000 & ph001
  • 2 Gpc/h box (same as "base")
  • \(1.5 \times 10^{10}\, \mathrm{M}_\odot/h\) particle mass (3.5x worse than "base", suitable for LRGs)
  • 4096^3 particles
  • 500 GPU node-hours each on Perlmutter
  • 16 hours on 32 nodes, 1D decomposition
  • Final redshift of 0.3, matching DESI LRG selection
Name f_NL Phase Full Outputs
pngbase_c000 {-100,-50,0,50,100} ph000 Full
pngbase_c000 {-100,-50,0,50,100} ph001 Partial

Checking for scale-dependent halo bias

Abacus DESI-II Simulations

  • Two high mass resolution simulations with outputs between z=5 and z=2
  • Abacus_DESI2_c000_ph300
    • Larger volume
  • Abacus_DESI2_c000_ph310
    • Better mass resolution
    • Still running
  • 2D domain decomposition
  • For reference, AbacusSummit base resolution was \(2.1 \times 10^{9}\, \mathrm{M}_\odot/h\)
  • Minor optimizations for high redshift
    • Proper softening length is smaller and reaches 1/40th of the particle spacing at z=2
    • Input linear power spectrum taken at z=2 (for ph310)
Name DESI2_c000 DESI2_c000
Box 1250 Mpc/h 750 Mpc/h
Particle mass 7.9e8 M_⊙/h 1.1e8 M_⊙/h
N particles 6000^3 6912^3
Cost 850 node-hours Still running
Phase ph300 ph310

Summary

  • Implemented f_NL initial conditions for Abacus

    • Any experts want to do additional validation on the ICs?

  • Ran 10 f_NL simulations

    • Simulations are named "Abacus_pngbase_..."

  • 2 high-redshift simulation for DESI-II studies

    • Named "Abacus_DESI2_..."

    • 1 finished and 1 running

  • Available now on NERSC, in the usual location:
    $CFS/desi/cosmosim/Abacus

  • Requests for more simulations?

  • Special thanks: Martin White, Shadab Alam, Sandy Yuan, Daniel Eisenstein