Towards Transparent AI in Gravitational Wave Data Analysis: Methods, Limitations, and New Directions
He Wang (王赫)
hewang@ucas.ac.cn
International Centre for Theoretical Physics Asia-Pacific (ICTP-AP), UCAS
Taiji Laboratory for Gravitational Wave Universe (Beijing/Hangzhou), UCAS
The Rise of Machine Learning
AI is taking over the world... literally everywhere
Why should you consider applying ML to gravitational wave astrophysics?
The "Real" Reasons We Apply ML to GW Astrophysics
Let's be honest about our motivations... 😉
The perfectly valid "scientific" reasons:
- ✓ It sounded like a cool project
- ✓ My supervisor said it was a good thing to work on
- ✓ I will learn some really useful ML skills
- ✓ I'm already good at ML
- ✓ I want to get better at ML
- ✓ I want to get a high-paying job after this PhD/postdoc
- ✓ I want to be spared when the machines take over
Credit: Chris Messenger (MLA meeting,, Jan 2025)
Why is AI/ML Everywhere in GW Research?
The core motivations behind nearly all AI+GW research
ML is FAST
So much data, so little time!
• Bayesian parameter estimation
• Replaces computationally intensive components
ML is ACCURATE*
Consistently outperforms traditional approaches
• Unmodelled burst searches
• Continuous GW searches
ML is FLEXIBLE
Provides deeper insights into complex problems
• Reveals patterns through interpretability
• Enables previously impractical approaches
* When properly trained and validated on appropriate datasets
Credit: Chris Messenger (MLA meeting,, Jan 2025)
Why is AI/ML Everywhere in GW Research?
The core motivations behind nearly all AI+GW research
ML is FAST
So much data, so little time!
• Bayesian parameter estimation
• Replaces computationally intensive components
ML is ACCURATE*
Consistently outperforms traditional approaches
• Unmodelled burst searches
• Continuous GW searches
ML is FLEXIBLE
Provides deeper insights into complex problems
• Reveals patterns through interpretability
• Enables previously impractical approaches
* When properly trained and validated on appropriate datasets
Credit: Chris Messenger (MLA meeting,, Jan 2025)
Key question: If an ML (or any) analysis doesn't do 1 or more of these things, then from a scientific perspective,
what is the point?
Is It Really So Simple?
The reality of ML in scientific research is more nuanced
No: We need to think more critically
- ❓ Are we just trying to predict a function?
- ❓ Are there any astrophysical constraints?
- ❓ Do we need to understand how/why it works?
- ❓ What about errors? Quality flags?
- ❓ What happens if things go wrong?
Twitter: @DeepLearningAI_
Why Even Use AI?
The mathematical inevitability and the path to understanding
Universal Approximation Theorem
The existence theorem that guarantees solutions
- Neural networks with sufficient hidden layers can approximate any continuous function on compact subsets of \(\mathbb{R}^n\)
- Ref: Cybenko, G. (1989), Hornik et al. (1989)
The solution is mathematically guaranteed — our challenge is finding the path to it
Machine learning will win in the long run
AI models still have vast potential compared to the human brain's efficiency. Beating traditional methods is mathematically inevitable given sufficient resources.
The question is not if AI/ML will win, but how
Understanding AI's inner workings is the real challenge, not proving its capabilities.
That's where we can learn something exciting with Foundation Models.
How do we understand AI's inner workings in GW data analysis?
Uncovering the "black box" to reveal
how AI actually processes GW strain data
Core Insight from Computer Vision
- Direct approach from Computer Vision (CV) to GW signal processing: pixel point \(\Rightarrow\) sampling point.
- The CNN framework treats time series data similar to images, where each sampling point represents a feature to learn.
Performance Analysis
- Convolutional neural networks (CNN) can achieve comparable performance to Matched Filtering under Gaussian stationary noise.
- CNNs significantly outperform traditional methods in terms of execution speed (with GPU support).
- Modern architectures show improved robustness against non-Gaussian noise transients (glitches).
Pioneering Research Publications
PRL, 2018, 120(14): 141103.
PRD, 2018, 97(4): 044039.
CNN for GW Detection: Pioneering Approaches
Matched-filtering Convolutional Neural Network (MFCNN)
HW, SC Wu, ZJ CAO, et al. PRD 101, 10 (2020): 104003
CNN for GW Detection: Feature Extraction
Convolutional Neural Network (ConvNet or CNN)
feature extraction
classifier
>> Is it matched-filtering ? >> Wait, It can be matched-filtering!
- Matched-filtering (cross-correlation with templates) can be interpreted as a convolutional layer with predefined kernels.
GW150914
GW150914
-
Transform matched-filtering method from frequency domain to time domain.
-
The square of matched-filtering SNR for a given data \(d(t) = n(t)+h(t)\):
\(S_n(|f|)\) is the one-sided average PSD of \(d(t)\)
where
Deep Learning Framework
Time Domain
(matched-filtering)
(normalizing)
(whitening)
Frequency Domain
CNN for GW Detection: Feature Extraction
import mxnet as mx
from mxnet import nd, gluon
from loguru import logger
def MFCNN(fs, T, C, ctx, template_block, margin, learning_rate=0.003):
logger.success('Loading MFCNN network!')
net = gluon.nn.Sequential()
with net.name_scope():
net.add(MatchedFilteringLayer(mod=fs*T, fs=fs,
template_H1=template_block[:,:1],
template_L1=template_block[:,-1:]))
net.add(CutHybridLayer(margin = margin))
net.add(Conv2D(channels=16, kernel_size=(1, 3), activation='relu'))
net.add(MaxPool2D(pool_size=(1, 4), strides=2))
net.add(Conv2D(channels=32, kernel_size=(1, 3), activation='relu'))
net.add(MaxPool2D(pool_size=(1, 4), strides=2))
net.add(Flatten())
net.add(Dense(32))
net.add(Activation('relu'))
net.add(Dense(2))
# Initialize parameters of all layers
net.initialize(mx.init.Xavier(magnitude=2.24), ctx=ctx, force_reinit=True)
return net
1 sec duration
35 templates used
Explainable AI Approach
- Implements matched filtering operations through custom convolutional layers
- Makes the network more interpretable by embedding domain knowledge
- Connects traditional signal processing with deep learning
- Outperforms standard CNNs in both accuracy and efficiency
Matched-filtering Convolutional Neural Network (MFCNN)
The available codes (2019): https://gist.github.com/iphysresearch/a00009c1eede565090dbd29b18ae982c
CNN for GW Detection: Feature Extraction
HW, SC Wu, ZJ CAO, et al. PRD 101, 10 (2020): 104003
CNN for GW Detection: Feature Extraction
Visualization for the high-dimensional feature maps of learned network in layers for bi-class using t-SNE.
feature extraction
Convolutional Neural Network (ConvNet or CNN)
classifier
Is there GW or non-GW in it?
GW + noise / noise
Chapter 4, PhD thesis (2020):
https://iphysresearch.github.io/PhDthesis_html/C4/#45
CNN for GW Detection: Feature Extraction
signal
noise
signal + noise
glitch_H1 + noise
Jun Tian, HW, et al. In Preparation (2025)
Is there GW or non-GW in it?
feature extraction
Convolutional Neural Network (ConvNet or CNN)
classifier
CNN for GW Detection: Feature Extraction
signal
noise
signal + noise
glitch_H1 + noise
Jun Tian, HW, et al. In Preparation (2025)
Is there GW or non-GW in it?
feature extraction
Convolutional Neural Network (ConvNet or CNN)
classifier
Key insight: At test time, one can easily construct statistics to differentiate between signal, noise, and glitches
CNN for GW Detection: Feature Extraction
Jun Tian, HW, et al. 2505.20357
Proformance: Is there GW or non-GW in the data?
GW / noise + Glitch
GW / noise / Glitch
GW / noise
GW / noise / Glitch
GW / noise
Random
Forest
- Gravitational wave signal search algorithm benchmark (MLGWSC-1)
- Dataset-4: Sampled from O3a real gravitational wave observation data
First Benchmark for GW Detection Algorithms
Benchmark Results
Publications
Key Findings
- On simulated noise data, machine learning algorithms are highly competitive compared to LIGO's most sensitive signal search pipelines
- Most tested machine learning algorithms are overly sensitive to non-Gaussian real noise backgrounds, resulting in high false alarm rates
- Traditional signal search algorithms can identify gravitational wave signals at low false alarm rates with assured confidence
- Tested machine learning algorithms have very limited ability to identify long-duration signals
Note on Benchmark Limitations:
Outperforming PyCBC doesn't conclusively prove that matched filtering is inferior to AI methods. This is both because the dataset represents a specific distribution and because PyCBC settings could be further optimized for this particular benchmark.
arXiv:2501.13846 [gr-qc]
Phys. Rev. D 110, 024024 (2024)
Phys. Rev. D 107, 023021 (2023)
Interpretability Challenges: Comparing Detection Statistics
-
Challenges in Model Interpretability:
- The black-box nature of AI models complicates interpretability, challenging the comparison of AI-generated detection statistics with traditional matched filtering chi-square distributions.
- Convincing the scientific community of the pipeline's validity and the statistical significance of new discoveries remains difficult despite the model's ability to identify potential gravitational wave signals.
AI Model Denoising
Our Model's Detection Statistics
LVK Official Detection Statistics
Signal denoising visualization using our deep learning model (Transformer-based)
Detection statistics from our AI model showing O1 events
HW et al 2024 MLST 5 015046
GW151226
GW151012
Official detection statistics from LVK collaboration
LVK. PRD (2016). arXiv:1602.03839
arXiv:2407.07820 [gr-qc]
Recent AI Discoveries & Validation Hurdles:
- A recent study (arXiv:2407.07820) demonstrates how a ResNet-based (CNN) architecture with careful signal search strategy and post-processing can identify 8 new potential gravitational wave events from LIGO O3 data.
- The absence of these events in traditional PyCBC results raises questions: could adjustments to rate priors and p_astro parameters in signal models help traditional pipelines detect these candidates (if they are real GW events)?
- The ideal approach combines multiple diverse pipelines working in parallel to ensure comprehensive detection (requiring interpretable models) and using evidence-based detection statistics while simultaneously optimizing both real signal population (p_astro) and noise model (likelihood) fits.
Search
PE
Rate
Key Insight:
Interpretability Challenges: Discoveries vs. Validation (part 1/2)
Interpretability Challenges: Discoveries vs. Validation (part 1/2)
Recent AI Discoveries & Validation Hurdles:
- A recent study (arXiv:2407.07820) demonstrates how a ResNet-based (CNN) architecture with careful signal search strategy and post-processing can identify 8 new potential gravitational wave events from LIGO O3 data.
- The absence of these events in traditional PyCBC results raises questions: could adjustments to rate priors and p_astro parameters in signal models help traditional pipelines detect these candidates (if they are real GW events)?
- The ideal approach combines multiple diverse pipelines working in parallel to ensure comprehensive detection (requiring interpretable models) and using evidence-based detection statistics while simultaneously optimizing both real signal population (p_astro) and noise model (likelihood) fits.
Search
PE
Rate
Key Insight:
Credit: DCC-XXXXXXXX
Representation Space Interpolation
Core Insights: Generative models' ability to perform accurate statistical inference can be understood as manifold learning rather than mere density estimation:
- Models learn a continuous latent manifold of data distributions
- Statistical parameters act as coordinates in this space
- Inference occurs through latent space navigation
- Enables robust generalization for complex distributions
Theoretical Understanding of Generative Models
Generative models don't memorize examples, but learn a continuous manifold
where similar concepts lie near each other. Statistical inference becomes
a form of navigation through this learned representation space.
Deep Learning is Not As Impressive As you Think, It's Mere Interpolation.
CVAE
Encodes data into latent space, enabling conditional generation
Flow-based
Transforms simple distributions into complex ones via invertible mappings
AI for Science
The core driving force of AI4Sci largely lies in its “interpolation” generalization capabilities, showcasing its powerful complex modeling abilities.
From 李宏毅
The core driving force of AI4Sci largely lies in its “interpolation” generalization capabilities, showcasing its powerful complex modeling abilities.
AI for Science
Parameter Estimation Challenges with AI Models:
- In parameter estimation, AI models' lack of interpretability requires substantial additional scientific validation to ensure credibility and acceptance of results.
- Parameter distributions from AI models often lack robustness across different noise realizations and are difficult to calibrate against established methods.
- Scientific papers using AI methods must dedicate significant space to validation procedures, comparing against traditional methods and demonstrating reliability across multiple test cases.
arXiv:2404.14286
Phys. Rev. D 109, 123547 (2024)
Interpretability Challenges: Discoveries vs. Validation (part 2/2)
Why Do We Trust Traditional Models?
Key Trust Factors:
- ✓ Interpretable: Parameters have physical meaning
- ✓ Built-in uncertainties: Input uncertainties propagate to outputs
- ✓ Model selection: Balance simplicity with accuracy
- ✓ Scientific insight: Reduces complexity, reveals principles
Traditional Physics Approach
Input
Human-Designed Algorithm
(Based on human insight)
Output
Example: Matched Filtering,
Linear Regression
Black-Box AI Approach
Input
AI Model
(Low interpretability)
Output
Examples: CNN, AlphaGo, DINGO
Key Challenge: How can we maintain the interpretability advantages of traditional models while leveraging the power of AI approaches?
Data/
Experience
Data/
Experience
Bridging Trust & Performance in GW Analysis
Combining the interpretability of physics with the power of AI
Our Mission: To create transparent AI systems that combine physics-based interpretability with deep learning capabilities
Interpretable AI Approach
The best of both worlds
Input
Physics-Informed
AI Algorithm
(High interpretability)
Output
Example: Our Approach
(In Preparation)
AI Model
Physics
Knowledge
Traditional Physics Approach
Input
Human-Designed Algorithm
(Based on human insight)
Output
Example: Matched Filtering, linear regression
Black-Box AI Approach
Input
AI Model
(Low interpretability)
Output
Examples: CNN, AlphaGo, DINGO
Data/
Experience
Data/
Experience
MLGWSC1 Benchmark: Optimization Performance Results
Preliminary Results (February 2025)
Optimization Progress & Algorithm Diversity
Optimization Target: Maximizing Area Under Curve (AUC) in the 10-100Hz frequency range, balancing detection sensitivity and false alarm rates across algorithm generations
Optimization Target: Maximizing Area Under Curve (AUC) in the 10-100Hz frequency range, balancing detection sensitivity and false alarm rates across algorithm generations
The framework (LLMs) can effectively optimize complex algorithms and guide iterative development along specified optimization directions, achieving targeted performance improvements in GW detection
Pipeline Workflow
- Conditions raw detector data (whitening)
- Computes time-frequency metrics
- Identifies peaks above background
- Returns event candidates with timestamps
This pipeline combines adaptive PSD whitening and multi-band spectral coherence computation with a noise floor-aware peak detection and a non-linear timing uncertainty model to enhance gravitational wave signal detection accuracy and robustness. It computes coherent time-frequency metric (with frequency-dependent regularization and entropy-based symmetry enforcement) and validates candidate signals via geometric features and multi-resolution thresholding (including dyadic wavelet analysis).
Integrate asymmetric PSD whitening, extended STFT overlap optimization, chirp-enhanced prominence scaling, multi-channel noise floor refinement, and dynamic timing calibration for improved gravitational wave signal detection.
The pipeline first applies adaptive local parameter control and noise-adaptive statistical regularization\u2014dynamically tuning median filter kernels, whitening gains, and spectral smoothness\u2014to detrend and whiten the dual-channel gravitational wave data, prioritizing robust noise baseline estimation over high-frequency variations. Then, it computes a coherent time-frequency metric (with frequency-dependent regularization and entropy-based symmetry enforcement) and validates candidate signals via geometric features and multi-resolution thresholding (including dyadic wavelet analysis), ultimately outputting candidate trigger GPS times, significance levels, and timing uncertainties.
MLGWSC1 Benchmark: Optimization Performance Results
Preliminary Results (February 2025)
Optimization Progress & Algorithm Diversity
Sensitivity vs False Alarm Rate
Optimization Target: Maximizing Area Under Curve (AUC) in the 10-100Hz frequency range, balancing detection sensitivity and false alarm rates across algorithm generations
Optimization Target: Maximizing Area Under Curve (AUC) in the 10-100Hz frequency range, balancing detection sensitivity and false alarm rates across algorithm generations
The framework (agent-based LLMs) can effectively optimize complex algorithms and guide iterative development along specified optimization directions, achieving targeted performance improvements in GW detection
Pipeline Workflow
- Conditions raw detector data (whitening)
- Computes time-frequency metrics
- Identifies peaks above background
- Returns event candidates with timestamps
Beyond Quick Wins: The Essence of AI in Gravitational Wave Science
Understanding the fundamental principles rather than seeking shortcuts
The true value of AI in gravitational wave science emerges not from quick implementation, but from patient cultivation of deep understanding. This journey requires time, thoughtfulness, and respect for fundamental principles.
The Path to Deeper Understanding
True algorithmic mastery requires patience and depth:
- ⟐ Scientific Insight: Not just predictions, but understanding why different architectures excel at capturing specific physical phenomena
- ⟐ Meaningful Integration: Building bridges between AI and physics requires understanding their fundamental connections, not just surface similarities
- ⟐ Principled Control: Beyond black-box usage—knowing exactly when to trust results and when to question them
- ⟐ Physics-Informed Design: Algorithms should reflect our understanding of gravitational wave physics, not replace or obscure it
Beyond Quick Wins: The Essence of AI in Gravitational Wave Science
Understanding the fundamental principles rather than seeking shortcuts
The true value of AI in gravitational wave science emerges not from quick implementation, but from patient cultivation of deep understanding. This journey requires time, thoughtfulness, and respect for fundamental principles.
The Path to Deeper Understanding
True algorithmic mastery requires patience and depth:
- ⟐ Scientific Insight: Not just predictions, but understanding why different architectures excel at capturing specific physical phenomena
- ⟐ Meaningful Integration: Building bridges between AI and physics requires understanding their fundamental connections, not just surface similarities
- ⟐ Principled Control: Beyond black-box usage—knowing exactly when to trust results and when to question them
- ⟐ Physics-Informed Design: Algorithms should reflect our understanding of gravitational wave physics, not replace or obscure it
for _ in range(num_of_audiences):
print('Thank you for your attention! 🙏')hewang@ucas.ac.cn