2024年7月24日 | 甘肃 · 兰州大学

王赫 (He Wang)

hewang@ucas.ac.cn

International Centre for Theoretical Physics Asia-Pacific (ICTP-AP), UCAS

Taiji Laboratory for Gravitational Wave Universe (Beijing/Hangzhou), UCAS

On behalf of the LIGO-VIRGO-KAGRA collaborations

From Data Processing to Scientific Discovery

https://twitter.com/chipro/status/1768388213008445837?s=46&t=JmDXWgIucgr_FlsBFTvuRQ

DINGO+SEOBNRv4EHM找了3个ebbh

Evidence for eccentricity in the population of binary black holes observed by LIGO-Virgo-KAGRA
https://dcc.ligo.org/LIGO-G2400750

• GW Astronomy
• AI for Science · GW Data Analysis
• GW search · Pipeline
• Parameter estimation · Scientific discovery
• Key Takeaways
• (Space-based GW Detection)

• In 1916, A. Einstein proposed the GR and predicted the existence of GW.

• Gravitational waves (GW) are a strong field effect in the GR.

  • 2015: the first experimental detection of GW from the merger of two black holes was achieved.

  • 2017: the first multi-messenger detection of a BNS signal was achieved, marking the beginning of multi-messenger astronomy.

  • 2017: the Nobel Prize in Physics was awarded for the detection of GW.

  • As of now: more than 90 gravitational wave events have been discovered.

  • O4, which began on May 24th 2023, is currently in progress.

Gravitational waves generated by binary black holes system

GW detector

LIGO-VIRGO-KAGRA network

2017 Nobel Prize in Physics

• Fundamental Physics
  • Existence of gravitational waves
  • To put constraints on the properties of gravitons
• Astrophysics
  • Refine our understanding of stellar evolution
  • and the behavior of matter under extreme conditions.
• Cosmology
  • The measurement of the Hubble constant
  • Dark energy

The first GW event of GW150914

• Detecting gravitational waves require a mix of FIVE key ingredients:
  1. good detector technology
  2. good waveform predictions
  3. good data analysis methodology and technology
  4. coincident observations in several independent detectors
  5. coincident observations in electromagnetic astronomy

—— Bernard F. Schutz

​​DOI: 10.1063/1.1629411 

GWTC-3

• Fundamental Physics
  • Existence of gravitational waves
  • To put constraints on the properties of gravitons
• Astrophysics
  • Refine our understanding of stellar evolution
  • and the behavior of matter under extreme conditions.
• Cosmology
  • The measurement of the Hubble constant
  • Dark energy

• Detecting gravitational waves require a mix of FIVE key ingredients:
  1. good detector technology
  2. good waveform predictions
  3. good data analysis methodology and technology
  4. coincident observations in several independent detectors
  5. coincident observations in electromagnetic astronomy

—— Bernard F. Schutz

​​DOI: 10.1063/1.1629411 

GWTC-3

©Floor Broekgaarden (repo)

Matched filtering techniques (匹配滤波方法)
 

• In Gaussian and stationary noise environments, the optimal linear algorithm for extracting weak signals

• Works by correlating a known signal model \(h(t)\) (template) with the data.
• Starting with data: \(d(t) = h(t) + n(t)\).
• Defining the matched-filtering SNR \(\rho(t)\):
  \(\rho^2(t)\equiv\frac{1}{\langle h|h \rangle}|\langle d|h \rangle(t)|^2 \) , where \(\langle d|h \rangle (t) = 4\int^\infty_0\frac{\tilde{d}(f)\tilde{h}^*(f)}{S_n(f)}e^{2\pi ift}df \) ,
  \(\langle h|h \rangle = 4\int^\infty_0\frac{\tilde{h}(f)\tilde{h}^*(f)}{S_n(f)}df \), \(S_n(f)\) is noise power spectral density (one-sided).

LISA / Taiji project

• Frequentist hypothesis testing and likelihood princple:

  • make some assumptions about signal and noise hypothesis

  • write down the likelihood function for a signal in noise

  • find the parameters that maximise it

  • define a corresponding detection statistic

    \(\rightarrow\) recover the MF
• Bayesian hypothesis testing:
  • start from the same likelihood
  • define a prior over signal parameters
  • marginalise over them to arrive at a Bayes factor
  • Often the dirty secret: just treat this as a Frequentist detection statistic
    \(\rightarrow\) recover the MF (for certain prior choices)

• GW Astronomy
• AI for Science · GW Data Analysis
• GW search · Pipeline
• Parameter estimation · Scientific discovery
• Key Takeaways
• (Space-based GW Detection)

• 2016年，AlphaGo 第一版发表在了 Nature 杂志上

• 2021年，AI预测蛋白质结构登上 Science、Nature 年度技术突破，潜力无穷

• 2022年，DeepMind团队通过游戏训练AI发现矩阵乘法算法问题​

• 《达摩院2022十大科技趋势》将 AI for Science 列为重要趋势

  • “人工智能成为科学家的新生产工具，催生科研新范式”

• 2023年，DeepMind发布AI工具GNoME (Nature)，成功预测220万种晶体结构

• AI for Science：为科学带来了模型与数据双驱动的新的研究范式

  • AI + 数学、AI + 化学、AI + 医药、AI + 量子、AI + 物理、AI + 天文 ...

Pioneering works utilizing CNN

• The most common and direct approach, from Computer Vision (CV) to GW signal processing: pixel point \(\Rightarrow\) sampling point.

• Convolutional neural networks (CNN) can achieve comparable performance to Matched Filtering and surpass them in terms of execution speed (with GPU support) under Gaussian stationary noise.

AI for Science \(\rightarrow\) AI for GW Astronomy






 

• Artificial Intelligence (AI) has great potential to revolutionize gravitational wave astronomy by improving data analysis, modeling, and detector development.
• Representation and supervised learning crucially extract features from GW signals, autonomously identifying informative features and leveraging labeled data for accuracy.

• GW Astronomy
• AI for Science · GW Data Analysis
• GW search · Pipeline
• Parameter estimation · Scientific discovery
• Key Takeaways
• (Space-based GW Detection)

Matched-filtering Convolutional Neural Network (MFCNN)

• GW templates can be utilized as recognizable features for signal detection.
• It is feasible to generalize both matched-filtering and neural networks.
• Linear filters (i.e., matched-filtering) in signal processing can be reformulated as neural layers (i.e., CNNs).

MLGWSC-1

• The majority of AI algorithms used for testing are highly sensitive to non-Gaussian real noise backgrounds, resulting in high false positive rates.

CL.M., W.W., H.W., et al. PRD (2022)

Ensemble learning

• Leverages statistical approaches to utilize more information for making informed decisions by combining multiple models.

Expanding the dimension of the output

• is to call more information to make decisions in improving AI models.

 Introduction to Speed and Efficiency

• Machine Learning (ML) offers unparalleled speed in GW detection.
• The increasing sensitivity to noise and the growing number of GW events demand faster analysis capabilities.
   

The Need for Integration (an AI pipeline!)

• Numerous AI algorithms exist for signal detection (see more: https://wiki.ligo.org/MLA/ML_at_LIGO_and_VIRGO), highlighting the need for a streamlined pipeline.
• However, none are operational in O4 due to infrastructure challenges and the lack of standardized sensitivity measures against non-stationary noise.

Case study:                   Pipeline

• The Aframe online pipeline (arXiv:2403.18661) represents a significant step forward.
• Utilizes GPU acceleration for signal preprocessing, model inference, and real-time false alarm rate calculation.

Aframe

S.S. Chaudhary, et al. arXiv:2308.04545

Challenges and Future Directions

• A comprehensive online pipeline should consider various signal types, including BNS and NSBH, and all detectors, including Virgo and KAGRA.
• Beyond false alarm rates, additional real-time information is crucial for a holistic analysis.

Case study:                   Pipeline
 

• The Aframe online pipeline (arXiv:2403.18661) represents a significant step forward.

• Despite its advancements, it primarily focuses on BBH signals, with decreased performance on longer signals like BNS, a point also highlighted in MLGWSC1(PRD, 2209.11146).

Aframe

Beyond Speed: Generalization and Discovery in GW Detection

• Our primary goal is not speed but the model's ability to generalize and discover new GW signals, including those beyond the reach of matched filtering techniques and General Relativity (GR).
• Leveraging our experience in  signal modeling  (MFCNN) and noise modeling (WaveFormer), we are gradually building an offline pipeline capable of searching for signals in complete GW observation data and calculating FARs.

Real-time GW searches for GW150914

He Wang, et al. PRD 101, 10 (2020): 104003

He Wang, et al. MLST. 5, 1 (2024): 015046.

Beyond Speed: Generalization and Discovery in GW Detection

• Our primary goal is not speed but the model's ability to generalize and discover new GW signals, including those beyond the reach of matched filtering techniques and General Relativity (GR).
• Leveraging our experience in  signal modeling  (MFCNN) and noise modeling (WaveFormer), we are gradually building an offline pipeline capable of searching for signals in complete GW observation data and calculating FARs.

He Wang, et al. PRD 101, 10 (2020): 104003

He Wang, et al. MLST. 5, 1 (2024): 015046.

Real-time GW searches for GW150914

数据增益 -> GWToolkit （缺一个好看的高吞吐性能图和loss/acc性能图）+ Data Protal (缺一个可视化和性能表现CKAN?)

Pipeline -> MFCNN + WaveFormer（缺雪藏event的corner图）; bGR（缺结果图）

信号探测（缺各种文章中的探测统计量图像）-> 引出新的理论缺失： machine learning GW statistics

Beyond Speed: Generalization and Discovery in GW Detection

• Our primary goal is not speed but the model's ability to generalize and discover new GW signals, including those beyond the reach of matched filtering techniques and General Relativity (GR).
• Leveraging our experience in  signal modeling  (MFCNN) and noise modeling (WaveFormer), we are gradually building an offline pipeline capable of searching for signals in complete GW observation data and calculating FARs.

Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012)

Feature extraction

Convolutional Neural Network (ConvNet or CNN)

Classification

GW150914

GW151226

GW151012

• CNNs always works pretty good on stimulated noises.
• However, when on real noises from LIGO, this approach does not work that well.
• It's too sensitive against the background + hard to find GW events)

Beyond Speed: Generalization and Discovery in GW Detection

• Our primary goal is not speed but the model's ability to generalize and discover new GW signals, including those beyond the reach of matched filtering techniques and General Relativity (GR).
• Leveraging our experience in  signal modeling  (MFCNN) and noise modeling (WaveFormer), we are gradually building an offline pipeline capable of searching for signals in complete GW observation data and calculating FARs.

He Wang, et al. PRD 101, 10 (2020): 104003

GW150914

GW151226

GW151012

Matched-filtering (cross-correlation with the templates) can be regarded as a convolutional layer with a set of predefined kernels.

• In practice, we use matched filters as an essential component of feature extraction in the first part of the CNN for GW detection.

Beyond Speed: Generalization and Discovery in GW Detection

• Our primary goal is not speed but the model's ability to generalize and discover new GW signals, including those beyond the reach of matched filtering techniques and General Relativity (GR).
• Leveraging our experience in  signal modeling  (MFCNN) and noise modeling (WaveFormer), we are gradually building an offline pipeline capable of searching for signals in complete GW observation data and calculating FARs.

He Wang, et al. PRD 101, 10 (2020): 104003

• Transform matched-filtering method from frequency domain to time domain.
• The square of matched-filtering SNR for a given data \(d(t) = n(t)+h(t)\):

Frequency domain

Beyond Speed: Generalization and Discovery in GW Detection

• Our primary goal is not speed but the model's ability to generalize and discover new GW signals, including those beyond the reach of matched filtering techniques and General Relativity (GR).
• Leveraging our experience in  signal modeling  (MFCNN) and noise modeling (WaveFormer), we are gradually building an offline pipeline capable of searching for signals in complete GW observation data and calculating FARs.

He Wang, et al. PRD 101, 10 (2020): 104003

• Transform matched-filtering method from frequency domain to time domain.
• The square of matched-filtering SNR for a given data \(d(t) = n(t)+h(t)\):

Frequency domain

Time domain

(normalizing)

(matched-filtering)

(whitening)

where

Beyond Speed: Generalization and Discovery in GW Detection

• Our primary goal is not speed but the model's ability to generalize and discover new GW signals, including those beyond the reach of matched filtering techniques and General Relativity (GR).
• Leveraging our experience in  signal modeling  (MFCNN) and noise modeling (WaveFormer), we are gradually building an offline pipeline capable of searching for signals in complete GW observation data and calculating FARs.

He Wang, et al. PRD 101, 10 (2020): 104003

• Transform matched-filtering method from frequency domain to time domain.
• The square of matched-filtering SNR for a given data \(d(t) = n(t)+h(t)\):

Frequency domain

Time domain

(normalizing)

(matched-filtering)

(whitening)

where

Deep Learning Framework

• In the 1-D convolution (\(*\)) on Apache MXNet, given input data with shape [batch size, channel, length] :

FYI:       \(N_\ast = \lfloor(N-K+2P)/S\rfloor+1\)

（A schematic illustration for a unit of convolution layer)

Beyond Speed: Generalization and Discovery in GW Detection

• Our primary goal is not speed but the model's ability to generalize and discover new GW signals, including those beyond the reach of matched filtering techniques and General Relativity (GR).
• Leveraging our experience in  signal modeling  (MFCNN) and noise modeling (WaveFormer), we are gradually building an offline pipeline capable of searching for signals in complete GW observation data and calculating FARs.

He Wang, et al. PRD 101, 10 (2020): 104003

• Transform matched-filtering method from frequency domain to time domain.
• The square of matched-filtering SNR for a given data \(d(t) = n(t)+h(t)\):

Frequency domain

Time domain

(normalizing)

(matched-filtering)

(whitening)

where

Deep Learning Framework

Beyond Speed: Generalization and Discovery in GW Detection

• Our primary goal is not speed but the model's ability to generalize and discover new GW signals, including those beyond the reach of matched filtering techniques and General Relativity (GR).
• Leveraging our experience in  signal modeling  (MFCNN) and noise modeling (WaveFormer), we are gradually building an offline pipeline capable of searching for signals in complete GW observation data and calculating FARs.

He Wang, et al. PRD 101, 10 (2020): 104003

Time domain

(normalizing)

(matched-filtering)

(whitening)

where

Deep Learning Framework

Beyond Speed: Generalization and Discovery in GW Detection

• Our primary goal is not speed but the model's ability to generalize and discover new GW signals, including those beyond the reach of matched filtering techniques and General Relativity (GR).
• Leveraging our experience in  signal modeling  (MFCNN) and noise modeling (WaveFormer), we are gradually building an offline pipeline capable of searching for signals in complete GW observation data and calculating FARs.

He Wang, et al. PRD 101, 10 (2020): 104003

The available codes: https://gist.github.com/iphysresearch/a00009c1eede565090dbd29b18ae982c

1 sec duration

35 templates used

Various types of Glitch

• The improvement of data quality is a very complex issue, with data from over 20,000 sensor channels determining the quality of the gravitational wave science data channel.

• Reducing non-Gaussian short-duration pulse interference (Glitches) in gravitational wave data will help reduce the false alarm rate of gravitational wave signals.

• Removing Glitches from gravitational wave detection data is a multi-classification problem.

  • Traditional machine learning algorithms​ ​Powell J, et al. CQG, 2015
  • Deep learning algorithms Zevin, M, et al. C

Ormiston R, et al. PRR, 2020

• DeepClean: One-dimensional Convolutional Neural Network which takes a specified set of witness channels and subsequently outputs the predicted noise in strain.

Credit: Marco Cavaglià 

CQG. 37 (2020) 055002

• The WaveFormer, a billion-scale transformer-based model, excels in suppressing realistic noise and recovering injections or GW events, thereby significantly improving data quality.
• In its application, it treats each overlapping time-domain data subsequence as an individual token, akin to tokenization in natural language processing (NLP).

Given $d=h+n$, we can normalize $d$ as follows:

• Implementations:
  • PSD sampling from real noise.
  • input size: 8.0625 sec
  • fs = ​2048Hz
  • Band-pass: 20~2048Hz
  • Masked loss

• Noise level percentile amplitude is significantly reduced, by approximately two orders.
• Further ASD analysis shows that WaveFormer effectively eliminates both narrowband and broadband spectral information, substantially lowering frequency contributions.
• Using the Gravity Spy database for glitches with SNR > 10 and confidence > 0.95, results show significant suppression of glitches in real advanced LIGO-Virgo noise.

• Overlap and matched-filtering signal-to-noise are calculated to represent phase and amplitude recovery performance.
• Among the intermediate frequency range (20–200 Hz) that covers rich BBH signal information, the ASD distribution of denoised waveform is evidently consistent with that of target signal.

(Upper panels: Signal amplitude recovery performance

(Bottom panels: Signal phase recovery performance​)

• These results show that our denoising algorithm outperformed others by capturing the characteristic chirping morphology of BBH evolution, and can denoise signals in realistic detection scenarios without affecting signal characteristics such as phase and amplitude.
• For the event GW191204_171526, classified as either an NSBH or a low-mass BBH candidate in GWTC-3, the overlap with IMRPhenomXPHM achieved 0.93 and 0.95 on H1 and L1, respectively, which are marked improvements over those achieved by BayesWave and cWB (with overlaps between 0.82–0.86).

GW191204_171526

• Firstly, we obtain the denoised output by utilizing Waveformer.
  Then, triggers are defined and identified by three steps including:

  1. Find Peaks. Locate triggers on a single detector by finding its maximum all local-maximum (0.2s away from neighboring maximum/local-maximum).

  2. By constraining triggers that exist on both two detectors, we get VALID triggers. (consist 3~4 segments)

  3. Calculate the cross-correlation of the to-be-evaluated trigger across channels or within a single channel, between its noisy and corresponding denoised segments, as well as between denoised segments themselves.

• Firstly, we obtain the denoised output by utilizing Waveformer. Then, triggers are defined and identified by three steps including,

  • Find Peaks. Locate triggers on a single detector by finding its maximum all local-maximum (0.2s away from neighboring maximum/local-maximum).

  • By constraining triggers that exist on both two detectors, we get VALID triggers. (consist 3~4 segments)

• Firstly, we obtain the denoised output by utilizing Waveformer. Then, triggers are defined and identified by three steps including,

  • Find Peaks. Locate triggers on a single detector by finding its maximum all local-maximum (0.2s away from neighboring maximum/local-maximum).

  • By constraining triggers that exist on both two detectors, we get VALID triggers. (consist 3~4 segments)

  • Calculate the correlation of the to-be-evaluated trigger across channels or within a single channel, between its noisy and corresponding denoised segments, as well as between denoised segments themselves.

• Firstly, we obtain the denoised output by utilizing Waveformer. Then, triggers are defined and identified by three steps including,

  • Find Peaks. Locate triggers on a single detector by finding its maximum all local-maximum (0.2s away from neighboring maximum/local-maximum).

  • By constraining triggers that exist on both two detectors, we get VALID triggers. (consist 3~4 segments)

  • Calculate the correlation of the to-be-evaluated trigger across channels or within a single channel, between its noisy and corresponding denoised segments, as well as between denoised segments themselves.

• Through time shift, background analysis is done on other triggers around the target trigger. (time-shift interval 0.1 sec)
• Finally, by counting the number of false alarm trigger pairs, we obtain the IFAR value of the target trigger, which represents the reported or candidate BBH event in this experiment.

OURs

(PyCBC) Davies​, et al. PRD 2020

• Developed an AI-based workflow with WaveFormer, combining convolutional neural network and transformer for effective GW noise suppression and hierarchical feature extraction across a wide frequency range.
• Achieved significant noise suppression and signal recovery performance improvements, including state-of-the-art results on real observational data and BBH events, leading to dramatic data quality improvement and significant IFAR enhancement on 75 reported BBH events.

• Challenges in Model Interpretability
  • The black-box nature of AI models complicates interpretability, challenging the comparison of AI-generated detection statistics with traditional matched filtering chi-square distributions.
  • Convincing the scientific community of the pipeline's validity and the statistical significance of new discoveries remains difficult despite the model's ability to identify potential gravitational wave signals.

OURs

LVK. PRD (2016). arXiv:1602.03839

The negative log-likelihood cost function always strongly penalizes the most active incorrect prediction. And the correctly classified examples will contribute little to the overall training cost."
—— I. Goodfellow, Y. Bengio, A. Courville. Deep Learning. 2016. (book)

OURs

LVK. PRD (2016). arXiv:1602.03839

• Challenges in Model Interpretability
  • The black-box nature of AI models complicates interpretability, challenging the comparison of AI-generated detection statistics with traditional matched filtering chi-square distributions.
  • Convincing the scientific community of the pipeline's validity and the statistical significance of new discoveries remains difficult despite the model's ability to identify potential gravitational wave signals.

Exploring Beyond General Relativity
 

• Much of the discussion on model generalization has been within the GR framework. Our collaboration with 东北大学 on beyond General Relativity (bGR) aims to demonstrate AI's potential advantages in detecting signals that surpass GR's limitations.

Harsh Narola, et al. “Beyond General Relativity: Designing a Template-Based Search for Exotic Gravitational Wave Signals.” PRD 107, 2 (2023): 024017.

Yu-Xin Wang​, et al. "Draft in Progress"

• GW Astronomy
• AI for Science · GW Data Analysis
• GW search · Pipeline
• Parameter estimation · Scientific discovery
• Key Takeaways
• (Space-based GW Detection)

Credit: LIGO Magazine.

• Traditional parameter estimation (PE) techniques rely on Bayesian analysis methods (posteriors + evidence)

• Computing the full 15-dimensional posterior distribution estimate is very time-consuming:
  • Calculating likelihood function
  • Template generation time-consuming
• Machine learning algorithms are expected to speed up! 

• A complete 15-dimensional posterior probability distribution, taking about 1 s (<< \(10^4\) s).

• Prior Sampling: 50,000 Posterior samples in approximately 8 Seconds.

• Capable of calculating evidence
• Processing time: (using 64 CPU cores)
  • less than 1 hour with IMRPhenomXPHM,
  • approximately 10 hours with SEOBNRv4PHM

Nature Physics 18, 1 (2022) 112–17

Big Data Mining and Analytics 5, 1 (2021) 53–63.

（Based on 1912.02762）

【【机器学习】白板推导系列(三十三) ～ 流模型(Flow based Model)】 

（Based on 1912.02762）

• Data: target data \(\mathbf{y}\in\mathbb{R}^{15}\) (with condition data \(\mathbf{x}\)).
• Task:
  • Fitting a flow-based model \(p_{\mathrm{y}}(\mathbf{y} ; \boldsymbol{\theta})\) to a target distribution \(p_{\mathrm{y}}^{*}(\mathbf{y})\)
  • by minimizing KL divergence with respect to the model’s parameters \(\boldsymbol{\theta}=\{\boldsymbol{\phi}, \boldsymbol{\psi}\}\),
  • where \(\boldsymbol{\phi}\) are the parameters of \(T\) and \(\boldsymbol{\psi}\) are the parameters of \(p_{\mathrm{z}}(\mathbf{z})=\mathcal{N}(0,\mathbb{I})\).
• Loss function:
  
  
  
  
   
• Assuming we have a set of samples \(\left\{\mathbf{y}_{n}\right\}_{n=1}^{N}\sim p_{\mathrm{y}}^{*}(\mathbf{y})\),
  
  
  
  Minimizing the above Monte Carlo approximation of the KL divergence is equivalent to fitting the flow-based model to the samples \(\left\{\mathbf{y}_{n}\right\}_{n=1}^{N}\) by maximum likelihood estimation.

归一化流模型示意图

DINGO  要特别快，要足够准

（缺DINGO技术相关的前沿动态）

Enrico 只要够快就足够了，可以做引力检验

（调研所有和最新的AI+引力检验文章）

（缺PyGWB的引入、数据描述、结果）-> 同样遇到如何做statistics的问题

• Bayesian inference, the Holy Grail of gravitational-wave data analysis,
  enables astrophysical interpretation and scientific discoveries.
   

Simulation-Based Inference (SBI)

• SBI \(\Rightarrow\) Fast and precise parameter estimation.
• SBI \(\Rightarrow\) TGR / Cosmology / PTA ...

PRL 127, 24 (2021) 241103.

PRL 130, 17 (2023) 171403.

Real-time gravitational wave science with neural posterior estimation

Sampling with prior knowledge for high-dimensional gravitational wave data analysis

He Wang, et al. Big Data Min. Anal. (2021)

He Wang, et al.  (2024)

PRD 108, 4 (2023): 044029.

Appreciating the Ringdown Overtone Test of GW150914

• A notable work involves ringdown overtone testing, which, acknowledging the difficulty in achieving DINGO-like precision for complex waveforms, leverages the speed advantage of AI.
• By simulating the signal and \(10^3\) realizations of LIGO noise for each pixel, it accomplishes what is impossible for MCMC methods, prioritizing speed over precision in a strategic trade-off.

arXiv:2404.14286

• 進撃のnflow model in GW inference area.

  • 2002.07656: 5D toy model [1] (PRD)

  • 2008.03312: 15D binary black hole inference [1] (MLST)

  • 2106.12594: Amortized inference and group-equivariant neural posterior estimation [2] (PRL)

  • 2111.13139: Group-equivariant neural posterior estimation [2]

  • 2210.05686: Importance sampling [2] (PRL)

  • 2211.08801: Noise forecasting [2] (PRD)

  • 2305.17161: FMPE

  • 2404.14286: eccentricity of BBHs

1. https://github.com/stephengreen/lfi-gw  (2020)

2. https://github.com/dingo-gw/dingo   (2023.03)

Exploring Stochastic Gravitational Wave Background with AI

• Utilizing AI for parameter estimation in the stochastic gravitational wave background (SGWB) presents a fascinating blend of rich theoretical content and the potential for optimizing current data processing methods.
• While still preliminary and ongoing, our work shows promising results for high SNR SGWB scenarios, where AI-based posterior probabilities are notably more precise and narrower compared to traditional cross-correlation methods used in PyGWB.

Our result (preliminary)

Exploring Stochastic Gravitational Wave Background with AI

• Performance saturation is observed between SNR levels of \(10^{-6}\) to \(10^{-7}\), indicating a plateau in model effectiveness in low SNR conditions.
• Unlike PyGWB, which can accumulate cross-correlation data from SGWB to further constrain the power spectrum, AI model outputs do not readily provide statistically meaningful information for aggregation. Multiplying posterior probabilities from multiple segments leads to ambiguous, and potentially biased, results due to the lack of statistically significant fluctuations across different posterior distributions.

Abbott R, et al. PRD 104, 2 (2021): 022004.

PyGWB result

Our result (preliminary)

• Exploring the importance of understanding how AI models make predictions in scientific research.
  • The critical role of generative models (生成模型是关键)
  • Quantifying uncertainty: a key aspect (不确定性量化问题)
  • Fostering controllable and reliable models (模型的可控可信问题)

"A running dog"

• The most common and direct approach, from Artificial Intelligence Generated Content (AIGC) to GW statistical inference: pixel point \(\Rightarrow\) inferred parameter.

• Exploring the importance of understanding how AI models make predictions in scientific research.
  • The critical role of generative models (生成模型是关键)
  • Quantifying uncertainty: a key aspect (不确定性量化问题)
  • Fostering controllable and reliable models (模型的可控可信问题)

"A corgi running on the street"

A picture is worth a thousand words.

A fraction of a thousand words.

Credit: 李宏毅

"A running dog"

• The most common and direct approach, from Artificial Intelligence Generated Content (AIGC) to GW statistical inference: pixel point \(\Rightarrow\) inferred parameter.

• GW Astronomy
• AI for Science · GW Data Analysis
• GW search · Pipeline
• Parameter estimation · Scientific discovery
• Key Takeaways
• (Space-based GW Detection)

On-going

• Agentic Reasoning for Inference
• ...

Insights

• AI is not just a tool; it is a revolutionary pathway for scientific discoveries.
• Theoretical Advancements in ML for GW Statistics
  • There is a pressing need for the theoretical refinement of ML applications in GW statistics, aiming to bridge current gaps and enhance model reliability.
• Improve the interpretability of AI models, as it is essential for enhanced and trustworthy discoveries.

This silde: https://slides.com/iphysresearch/2024july_lzu

For further reference or to cite the work presented today,
please cite this silde: https://slides.com/iphysresearch/2024mar_bnuz

https://twitter.com/chipro/status/1768388213008445837?s=46&t=JmDXWgIucgr_FlsBFTvuRQ

如何把大模型这个概念在合适的地方讲出来？（开头？最后？）

高维、多模态的inference挑战（是整个引力波相关的科学研究的技术难点）

PTMCMC 的算法描述图

Multi-agent 的概念图，及其相关结果的对比图

DINGO: A Leap Forward

• DINGO and related works represent the cutting edge in this field within the LIGO frequency band.
• Tested on 42 BBH events from GWTC-3
• Being deployed for O4, with the potential to become a new gravitational wave signal search pipeline
• Capable of calculating evidence
• Processing time: (using 64 CPU cores)
  • less than 1 hour with IMRPhenomXPHM,
  • approximately 10 hours with SEOBNRv4PHM
• ​Evidence for eccentricity in the population of BBH observed by LVK. (LIGO-G2400750, 2024 Mar)

• 進撃のnflow model in GW inference area.

  • 2002.07656: 5D toy model [1] (PRD)

  • 2008.03312: 15D binary black hole inference [1] (MLST)

  • 2106.12594: Amortized inference and group-equivariant neural posterior estimation [2] (PRL)

  • 2111.13139: Group-equivariant neural posterior estimation [2]

  • 2210.05686: Importance sampling [2] (PRL)

  • 2211.08801: Noise forecasting [2] (PRD)

[1]. https://github.com/stephengreen/lfi-gw  (published @2020)

[2]. https://github.com/dingo-gw/dingo   (published @2023.03)

Wang H, Du M H, Xu P, Zhou Y F. Sci Sin-Phys Mech Astron, 2024, 54, doi: 10.1360/SSPMA-2024-0087

The analysis of scientific data from space-based GW detection differs significantly from ground-based detection:

• A superposition of overlapping signals ( \(\neq\) isolated event)
• Observations of more waveform periods over different time scales ( \(\neq\) short-duration signals)
• Signal-dominated detection ( \(\neq\) noise-dominated)
• Reliance on more complex techniques for noise assessment
  ( \(\neq\) regular acquisition of signal-free data)

Analyses cannot treat sources independently and sequentially work through a list of candidate detections.

The analysis of scientific data from space-based GW detection differs significantly from ground-based detection:

• A superposition of overlapping signals ( \(\neq\) isolated event)
• Observations of more waveform periods over different time scales ( \(\neq\) short-duration signals)
• Signal-dominated detection ( \(\neq\) noise-dominated)
• Reliance on more complex techniques for noise assessment
  ( \(\neq\) regular acquisition of signal-free data)

• The data analyses depend on the simultaneous fitting of complete astrophysical, cosmological, and instrument models to the observed data.
• The need for this so-called “global fit” was identified as the primary challenge to the data analysis early in the space-borne mission formulation.

(Sec.8.6 The Red Book)

• Global vs. Individual Analysis: While global-fit techniques effectively manage the dense overlapping of signals in space-based GW data, individual pipelines are crucial for detecting unique events.

• Role of Individual Pipelines: These pipelines act as a pre-processing step, focusing on particular types of sources and diving deeper into the data. They refine the analysis by working on the latest best-fit residuals from the global fit.

• Case Study - MBHB Mergers: Mergers of MBHBs often exhibit high SNR between \(10^2\) to \(10^3\), appearing as distinct peaks in data time series. 

• Data curation

  • Model: frequency domain; PhenomD;  TDI-A/E response

  • Input: 1 day length; 15Hz; shape=(2, 3, 2877)

  • Noise: Gaussian stationary from the noise PSD (for training/test) + GB confusion noise (for test)

  • Project: Taiji program

M. Du, B. Liang, HW, P. Xu, Z. Luo, Y. Wu. SCPMA 67, 230412 (2024).

• The top section of the illustration shows the solar system barycenter (SSB) and Taiji frames, with two black dashed arrows symbolizing not two separate GW signals, but rather indicating how the sky location and arrival time of the same GW signal take different values in these two frames.

• The “positive” problem translates the SSB-frame parameters to their Taiji-frame counterparts via a time-dependent mapping \(f_1\), then to the TDI outputs through a time-independent mapping \(f_2\), and an exponential term.

• These steps can be schematically summarized as:
  
  
  where \(\mathcal{T}_\alpha^{A, E}(f)\) is often referred to as the transfer function.

M. Du, B. Liang, HW, P. Xu, Z. Luo, Y. Wu. SCPMA 67, 230412 (2024).

• Consequently, even if the network has only learned the time-dependent relationship between \(\boldsymbol{\theta}_S\) and the TDI response at a specific tref (the 30th day in our case), with the aid of coordinate transformation, it has essentially learned the time-invariant mapping \(f_2\), and can be then generalized to make parameter estimation at any other reference time.
   

• It is worth noting that our method relies on analytical orbits and
  the time-independence of the coordinate transformation \(f_2\).

• The top section of the illustration shows the solar system barycenter (SSB) and Taiji frames, with two black dashed arrows symbolizing not two separate GW signals, but rather indicating how the sky location and arrival time of the same GW signal take different values in these two frames.

• The “positive” problem translates the SSB-frame parameters to their Taiji-frame counterparts via a time-dependent mapping \(f_1\), then to the TDI outputs through a time-independent mapping \(f_2\), and an exponential term.

M. Du, B. Liang, HW, P. Xu, Z. Luo, Y. Wu. SCPMA 67, 230412 (2024).

• Methodology: Utilization of the Kolmogorov-Smirnov (KS) test to compare one-dimensional distributions generated by our algorithms, ensuring the accuracy of parameter estimation.

• Empirical Validation: Conducted extensive testing on simulated signals, injecting 1000 waveforms from the prior with added confusion noise and varying reference times between 1 and 365 days.

• Results: The tests assessed the frequency at which true parameters fell within certain confidence levels, confirming that our credible intervals are well-calibrated and reflect true confidence in the signal parameters.

M. Du, B. Liang, HW, P. Xu, Z. Luo, Y. Wu. SCPMA 67, 230412 (2024).

• Overview of Findings: Nested sampling results indicate minimal expected multimodality in ecliptic coordinates. However, distinct peaks identified in the time of coalescence (\(t_c\)), labeled as NF-1 (dominant) and NF-2 (subdominant), highlight unique multimodal behavior.

• Impact on PE: The presence of these peaks affects the posterior distributions of extrinsic parameters, potentially leading to inaccuracies in \(t_c\) and subsequent parameters due to phase term associations and inherent degeneracies.
• Model Performance: Despite the multimodality, the best-fit values from the NF model closely align with true values within the \(1\sigma\) range for most parameters, and at least \(2\sigma\) for others.
• Comparative Analysis: The ML pipeline tends to produce broader posteriors compared to the Bayesian nested sampling approach. 

（NF = Normalizing Flow model）

M. Du, B. Liang, HW, P. Xu, Z. Luo, Y. Wu. SCPMA 67, 230412 (2024).

Earth-based GW detection

1. A Python Toolbox for Gravitational Wave Astronomy: GWToolkit​
   • This toolbox is powered by Ray/JAX and supports both CPU and GPU. It is designed specifically for machine learning applications.
2. Can AI identify new GW events from LIGO data?
   • Could this be a GW signal beyond General Relativity (GR)?
3. How can we address the issue of strong or unacceptable biases that occur when outputs from AI models are used jointly or in combination to measure properties of a population, sub-population, or ensemble?
   (also addressed by 2405.18095)

Space-based GW detection

• “Global fit” challenge
  • How can we achieve and accelerate the Bayesian inference through algorithmic innovations?
    • Flow-based proposal?
    • Transdimensional Nested Sampling?
  • How can we leverage powerful LLM-based methods to accomplish this?

中国科学院计算机网络信息中心“东方”超级计算系统 (全国产CPU/GPU)

This silde: https://slides.com/iphysresearch/2024july_lzu

• GW Astronomy
• AI for Science · GW Data Analysis
• GW search · Pipeline
• Parameter estimation · Scientific discovery
• Key Takeaways
• Space-based GW Detection
• 引力波数据分析之知识框架（GWDA Knowledge Framework）

“国际理论物理中心(亚太地区)” 经联合国教科文组织第38届大会审议通过。由中国科学院、基金委和国际理论物理中心共同建设，是进行基础科学前沿与相关交叉科学领域高水平科研、教育和培训的非营利性组织，是联合国教科文组织基础科学方面的在国内的第一个二类中心。

引力波天文学 & 引力波多信使天文学

• 目前，国内外正在计划进行各种引力波探测项目，包括利用宇宙微波背景辐射的B模极化的CMB-S4、LiteBIRD、BICEP3/Keck Array、AliCPT；利用脉冲星定时阵列的NANOGrav、EPTA、PPTA、CPTA；利用空间引力波星间激光干涉仪的LISA、太极计划、天琴计划等；以及用于地面引力波干涉仪的ET、BBO。最近，NANOGrav、EPTA和CPTA共同宣布了存在纳赫兹级随机引力波背景的强有力证据。

Chinese Journal of Space Science,  2023, 43(4): 589-599.

LIGO-G2300554

空基引力波探测科学数据的分析与地基相比差距很大：

• 大量的混叠波源 ( \(\neq\) 孤立事件)
• 在不同时间尺度上观测到更大的波形周期 ( \(\neq\) 短时信号)
• 信号主导的探测 ( \(\neq\) 噪声主导)
• 依赖更复杂的技术评估噪声 ( \(\neq\) 定期获取无信号数据)

空间引力波观测频段内含有大量的波源和多种波源类型：

• \(10^4\) 可探测的银河系内致密双星绕转 (UCB, VGB)
• \(10\sim10^2\) 超大质量黑洞双星合并 (SMBH)
• \(10\sim10^3\) 极端质量比黑洞双星绕转 (EMRI)
• 恒星级质量黑洞双星的绕转 (SOBH)
• 随机引力波背景 (SGWB)
• 未建模的波源事件 (Burst...)

空间太极计划

• 空间引力波探测主要有以下 4 类波源：

  • 恒星级质量的致密双星 (黑洞、中子星、白矮星以及它们的两两组合) 的旋近

  • 双白矮星的并合、​超大质量双黑洞的并合

  • 极端质量比旋进 (通常是一个恒星级致密天体绕着一个超大质量黑洞的旋进）

  • 宇宙中可能存在的中等质量双黑洞以及前面这些源的信号叠加形成的引力波背景

天琴计划

空基引力波探测科学数据的分析与地基相比差距很大：

• 大量的混叠波源 ( \(\neq\) 孤立事件)
• 在不同时间尺度上观测到更大的波形周期 ( \(\neq\) 短时信号)
• 信号主导的探测 ( \(\neq\) 噪声主导)
• 依赖更复杂的技术评估噪声 ( \(\neq\) 定期获取无信号数据)

空间引力波观测频段内含有大量的波源和多种波源类型：

• \(10^4\) 可探测的银河系内致密双星绕转 (UCB, VGB)
• \(10\sim10^2\) 超大质量黑洞双星合并 (SMBH)
• \(10\sim10^3\) 极端质量比黑洞双星绕转 (EMRI)
• 恒星级质量黑洞双星的绕转 (SOBH)
• 随机引力波背景 (SGWB)
• 未建模的波源事件 (Burst...)

天琴计划

空间太极计划

(Sec.8.3.1 Red Book)

空间引力波探测获得的是什么样的 (科学) 数据？

• 来自几十个激光干涉仪的分数频率偏差 (相对多普勒频移)
• 预计天线重新定位 (9天) 和激光重新锁定 (几周) 会导致短时间 (＜小时) 的间隔，而类似LPF的意外长时间间隔 (∼天) 经常被考虑在内，总占空比约80-90%。 (日心轨道)
• 主要受激光噪声的影响
• 经过预处理后，得到3个时间延迟干涉 (TDI) 数据流 (X, Y, Z)
• ...

Analyses cannot treat sources independently and sequentially work through a list of candidate detections.

空间引力波探测获得的是什么样的 (科学) 数据？

• 来自几十个激光干涉仪的分数频率偏差 (相对多普勒频移)
• 预计天线重新定位 (9天) 和激光重新锁定 (几周) 会导致短时间 (＜小时) 的间隔，而类似LPF的意外长时间间隔 (∼天) 经常被考虑在内，总占空比约80-90%。 (日心轨道)
• 主要受激光噪声的影响
• 经过预处理后，得到3个时间延迟干涉 (TDI) 数据流 (X, Y, Z)
• ...

其他重要的科学数据处理的技术挑战：

• 随机噪声（Stochastic noise）

• 仪器瞬变（glitches）

• 频谱线（Spectral lines）

• 数据间断（Data gaps）

• 非平稳性（Non-stationarities）

Baghi et al., Phys. Rev. D (2019)