Build Big Meets Build Smart To Explore The Universe

Flatiron Institute

Institute for Advanced Studies

Carolina Cuesta-Lazaro

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

Observation

Question

Hypothesis

Testable Predictions

Gather data

Alter, Expand, Reject Hypothesis

Develop General Theories

[Figure adapted from ArchonMagnus]

High-dimensional data

Simulators as theory models

The Scientific Method in 2025

The Universe accelerates!

The Universe expands, it should decelerate

What is the ultimate fate of the Universe?

Need a repulsive dark energy component

Measure supernovae redshifts

Matter domination -> the Universe decelerates: rate?

Distance-redshift relation via standard candles

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

Simulations?

"Semantic" lower dimensional representation

Foundation Models for Science

[On the Opportunities and Risks of Foundation Models" Bommasani et al]

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

Simulations in Foundation Models for Science

x^\mathcal{O}
x^\mathcal{S}

Simulated Data

Observed Data

z^\mathcal{O}_p
z^\mathcal{O}_s
z^\mathcal{S}_s
z^\mathcal{S}_p

Alignment Loss

\mathcal{L} = \sum_{\mathcal{D} \in (\mathcal{S}, \mathcal{O})} p(x^\mathcal{D}|z^\mathcal{D}_s, z^\mathcal{D}_p) + \lambda d(z^\mathcal{O}_s,z^\mathcal{S}_s)
E^\mathcal{O}
E^\mathcal{S}

Reconstruction

Alignment

50\%

(OT / Adversarial)

\hat{x}^\mathcal{O}
\hat{x}^\mathcal{S}
D

Shared Decoder

D

Observed Reconstructed

Simulated Reconstructed

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

A Toy Model Example

Idealized Simulations

Observations

+ Scale Dependent Noise

+ Bump

x^\mathcal{O}
x^\mathcal{S}
["Disentangling Foundation Models for Science: Robust Integration of Simulated and Observed Data" Cuesta-Lazaro, Alvarez-Melis (in-prep)]

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

Amplitude

Tilt

Tilt

p(\theta|z^\mathcal{O}_s)
p(\theta|z^\mathcal{O}_p)
p(\theta|z^\mathcal{O}_p,z^\mathcal{O}_s)
p(\theta|z^\mathcal{O}_p)

Robust SBI from Shared

p(x^\mathcal{O}|z^\mathcal{O}_p,z^\mathcal{O}_s)
p(x^\mathcal{O}|z^\mathcal{O}_s)

Visualizing Information Split

["Disentangling Foundation Models for Science: Robust Integration of Simulated and Observed Data" Cuesta-Lazaro, Alvarez-Melis (in-prep)]

Late Universe

Early Universe

Tension

From Tensions to Discoveries:  Anomalies in Cosmology

Early vs Late

Parametric Extensions

[Image Credit: Prof. Wendy Freedman]

 

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

Looking for what we don't know to look for

The missing pieces: Beyond parametric searches

Axion Dark Matter

Dark Matter - Baryon Interactions

Primordial Non-Gaussianity

Early Dark Energy

Dark Radiation

[Credit: Sandbox Studio]

 

[Credit: Sandbox Studio]

 

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

Learning Structured Representations with Flow Matching

["CosmoFlow: Scale-Aware Representation Learning for Cosmology with Flow Matching" Kannan, Qiu, Cuesta-Lazaro, Jeong]

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

Sid Kannan (UCSB)

Autoregressive in Frequency

["CosmoFlow: Scale-Aware Representation Learning for Cosmology with Flow Matching" Kannan, Qiu, Cuesta-Lazaro, Jeong]

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

["Detecting Model Misspecification in Cosmology with Scale-Dependent Normalizing Flows" Akhmetzhanova, Cuesta-Lazaro, Mishra-Sharma]

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

Base

OOD Mock 1

OOD Mock 2

Large Scales

Small Scales

Small Scales

OOD Mock 1

OOD Mock 2

Parameter Inference Bias (Supervised)

OOD Metric (Unsupervised)

Large Scales

Small Scales

Aizhan Akhmetzhanova (Harvard)

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

Observation

Question

Hypothesis

Testable Predictions

Gather data

Alter, Expand, Reject Hypothesis

Develop General Theories

[Figure adapted from ArchonMagnus]

Simulators as theory models

The Scientific Method in 2025

High-dimensional data

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

["An LLM-driven framework for cosmological
model-building and exploration" Mudur, Cuesta-Lazaro, Toomey (in prep)]

Can LLMs turn these anomalies into new hypothesis?

Propose a model for Dark Energy

Implement it in a Cosmology simulation code: CLASS

Test fit to DESI Observations

Iterate to improve fit

Quintessence, DE/DM interactions....

Must pass a set of general tests for "reasonable" models

Ideally, compare evidence to LCDM.

For now, Bayesian Information Criteria (BIC)

1

2

Nayantara Mudur (Harvard)

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

Can LLMs implement new physics models?

Thawing Quintessence

Axion-like Early Dark Energy

Ultra-light scalar field that temporarily acts as dark energy in the early universe 

Implementation Challenge:

Dynamic dark energy model: scalar field transitions from "frozen"  (cosmological constant-like) to evolving as the universe expands.

Oscillatory behaviour

Can take advantage of existing scalar field implementations in CLASS

+ 43,000 lines of C code

+ 10,000 lines of numerical files

CLASS Challenge:

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

1) Code compiles + obtains reasonable observables

2) Implementation agrees with target repository

3) Goodness of fit for DESI + Supernovae

4) H0 tension metrics

Curated

1 page long description of model to be implemented,  CLASS tips + very explicit units

Paper

Directly from a full paper

If fails, get feedback from another LLM

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

Propose a Dark Energy Model

Shortcut: field that produces this?

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

Propose a Dark Energy Model

Asked for physical motivation. It tried :( 

Not true, preferred scale

Reinforcement Learning

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

How to iterate

Update the base model weights  to optimize a scalar reward (s)

DeepSeek R1

Base LLM

(being updated)

What rewards are more advantageous?

Base LLM

(frozen)

Develop basic skills: numerics, theoretical physics, UNIT CONVERSION

Community Effort!

Evolutionary algorithms

Learning in natural language, reflect on traces and results

Examples: EvoPrompt, FunSearch,AlphaEvolve

How to iterate

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

["GEPA: Reflective prompt evolution can outperform reinforcement learning" Agrawal et al]

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

GEPA: Evolutionary

GRPO: RL

+10% improvement over RL with x35 less rollouts

Scientific reasoning with LLMs still in its infancy!

Carolina Cuesta-Lazaro Flatiron/IAS - Build Big or Build Smart

Observation

Question

Hypothesis

Testable Predictions

Gather data

Alter, Expand, Reject Hypothesis

Develop General Theories

[Figure adapted from ArchonMagnus]

The Scientific Method in > 2025