12月28日 @武汉大学

王赫

hewang@ucas.ac.cn

International Centre for Theoretical Physics Asia-Pacific (ICTP-AP), UCAS

Taiji Laboratory for Gravitational Wave Universe (Beijing/Hangzhou), UCAS

On behalf of the LIGO-VIRGO-KAGRA collaborations

• GW Astronomy
• AI for Science · GW Data Analysis
• GW search · Pipeline
• Parameter estimation · Scientific discovery
• Space-based GW Detection

• In 1916, A. Einstein proposed the GR and predicted the existence of GW.

• Gravitational waves (GW) are a strong field effect in the GR.

  • 2015: the first experimental detection of GW from the merger of two black holes was achieved.

  • 2017: the first multi-messenger detection of a BNS signal was achieved, marking the beginning of multi-messenger astronomy.

  • 2017: the Nobel Prize in Physics was awarded for the detection of GW.

  • As of now: more than 90 gravitational wave events have been discovered.

  • O4, which began on May 24th 2023, is currently in progress.

Gravitational waves generated by binary black holes system

GW detector

LIGO-VIRGO-KAGRA network

2017 Nobel Prize in Physics

• 引力波探测打开了探索宇宙的新窗口

• 不同波源，频率跨越 20 个数量级，不同探测器

• 多信使天文学

• Fundamental Physics
  • Existence of gravitational waves
  • To put constraints on the properties of gravitons
• Astrophysics
  • Refine our understanding of stellar evolution
  • and the behavior of matter under extreme conditions.
• Cosmology
  • The measurement of the Hubble constant
  • Dark energy

The first GW event of GW150914

• Detecting gravitational waves require a mix of FIVE key ingredients:
  1. good detector technology
  2. good waveform predictions
  3. good data analysis methodology and technology
  4. coincident observations in several independent detectors
  5. coincident observations in electromagnetic astronomy

—— Bernard F. Schutz

​​DOI: 10.1063/1.1629411 

GWTC-3

• Fundamental Physics
  • Existence of gravitational waves
  • To put constraints on the properties of gravitons
• Astrophysics
  • Refine our understanding of stellar evolution
  • and the behavior of matter under extreme conditions.
• Cosmology
  • The measurement of the Hubble constant
  • Dark energy

• Detecting gravitational waves require a mix of FIVE key ingredients:
  1. good detector technology
  2. good waveform predictions
  3. good data analysis methodology and technology
  4. coincident observations in several independent detectors
  5. coincident observations in electromagnetic astronomy

—— Bernard F. Schutz

​​DOI: 10.1063/1.1629411 

GWTC-3

©Floor Broekgaarden (repo)

Matched filtering techniques (匹配滤波方法)
 

• In Gaussian and stationary noise environments, the optimal linear algorithm for extracting weak signals

• Works by correlating a known signal model \(h(t)\) (template) with the data.
• Starting with data: \(d(t) = h(t) + n(t)\).
• Defining the matched-filtering SNR \(\rho(t)\):
  \(\rho^2(t)\equiv\frac{1}{\langle h|h \rangle}|\langle d|h \rangle(t)|^2 \) , where \(\langle d|h \rangle (t) = 4\int^\infty_0\frac{\tilde{d}(f)\tilde{h}^*(f)}{S_n(f)}e^{2\pi ift}df \) ,
  \(\langle h|h \rangle = 4\int^\infty_0\frac{\tilde{h}(f)\tilde{h}^*(f)}{S_n(f)}df \), \(S_n(f)\) is noise power spectral density (one-sided).

LISA / Taiji project

• Frequentist hypothesis testing and likelihood princple:

  • make some assumptions about signal and noise hypothesis

  • write down the likelihood function for a signal in noise

  • find the parameters that maximise it

  • define a corresponding detection statistic

    \(\rightarrow\) recover the MF
• Bayesian hypothesis testing:
  • start from the same likelihood
  • define a prior over signal parameters
  • marginalise over them to arrive at a Bayes factor
  • Often the dirty secret: just treat this as a Frequentist detection statistic
    \(\rightarrow\) recover the MF (for certain prior choices)

• GW Astronomy
• AI for Science · GW Data Analysis
• GW search · Pipeline
• Parameter estimation · Scientific discovery
• Space-based GW Detection

• 2016年，AlphaGo 第一版发表在了 Nature 杂志上

• 2021年，AIphaFold 预测蛋白质结构登上 Science、Nature 年度技术突破

• 2022年，DeepMind团队通过游戏训练AI发现矩阵乘法算法问题​

• 《达摩院2022十大科技趋势》将 AI for Science 列为重要趋势

  • “人工智能成为科学家的新生产工具，催生科研新范式”

• 2023年，DeepMind发布AI工具GNoME (Nature)，成功预测220万种晶体结构

• 2023年3月，为贯彻落实国家《新一代人工智能发展规划》，科技部会同自然科学基金委启动“人工智能驱动的科学研究”（AI for Science）专项部署工作，布局“人工智能驱动的科学研究”前沿科技研发体系​。

• 2016年，AlphaGo 第一版发表在了 Nature 杂志上

• 2021年，AIphaFold 预测蛋白质结构登上 Science、Nature 年度技术突破

• 2022年，DeepMind团队通过游戏训练AI发现矩阵乘法算法问题​

• 《达摩院2022十大科技趋势》将 AI for Science 列为重要趋势

  • “人工智能成为科学家的新生产工具，催生科研新范式”

• 2023年，DeepMind发布AI工具GNoME (Nature)，成功预测220万种晶体结构

• 2023年3月，为贯彻落实国家《新一代人工智能发展规划》，科技部会同自然科学基金委启动“人工智能驱动的科学研究”（AI for Science）专项部署工作，布局“人工智能驱动的科学研究”前沿科技研发体系​。

Pioneering works utilizing CNN

• The most common and direct approach, from Computer Vision (CV) to GW signal processing: pixel point \(\Rightarrow\) sampling point.

• Convolutional neural networks (CNN) can achieve comparable performance to Matched Filtering and surpass them in terms of execution speed (with GPU support) under Gaussian stationary noise.

AI for Science \(\rightarrow\) AI for GW Astronomy






 

• Artificial Intelligence (AI) has great potential to revolutionize gravitational wave astronomy by improving data analysis, modeling, and detector development.
• Representation and supervised learning crucially extract features from GW signals, autonomously identifying informative features and leveraging labeled data for accuracy.

• GW Astronomy
• AI for Science · GW Data Analysis
• GW search · Pipeline
• Parameter estimation · Scientific discovery
• Space-based GW Detection

Matched-filtering Convolutional Neural Network (MFCNN)

• GW templates can be utilized as recognizable features for signal detection.
• It is feasible to generalize both matched-filtering and neural networks.
• Linear filters (i.e., matched-filtering) in signal processing can be reformulated as neural layers (i.e., CNNs).

MLGWSC-1

• The majority of AI algorithms used for testing are highly sensitive to non-Gaussian real noise backgrounds, resulting in high false positive rates.

CL.M., W.W., H.W., et al. PRD (2022)

Ensemble learning

• Leverages statistical approaches to utilize more information for making informed decisions by combining multiple models.

Expanding the dimension of the output

• is to call more information to make decisions in improving AI models.

• GW Astronomy
• AI for Science · GW Data Analysis
• GW search · Pipeline
  • Data ​Denoising for Signal Detection
• Parameter estimation · Scientific discovery
• Space-based GW Detection

• Developed an AI-based workflow with WaveFormer, combining convolutional neural network and transformer for effective GW noise suppression and hierarchical feature extraction across a wide frequency range.
• Achieved significant noise suppression and signal recovery performance improvements, including state-of-the-art results on real observational data and BBH events, leading to dramatic data quality improvement and significant IFAR enhancement on 75 reported BBH events.

• Challenges in Model Interpretability
  • The black-box nature of AI models complicates interpretability, challenging the comparison of AI-generated detection statistics with traditional matched filtering chi-square distributions.
  • Convincing the scientific community of the pipeline's validity and the statistical significance of new discoveries remains difficult despite the model's ability to identify potential gravitational wave signals.

OURs

LVK. PRD (2016). arXiv:1602.03839

He Wang et al 2024 Mach. Learn.: Sci. Technol. 5 015046

Exploring Beyond General Relativity
 

• Much of the discussion on model generalization has been within the GR framework. Our collaboration with 东北大学 on beyond General Relativity (bGR) aims to demonstrate AI's potential advantages in detecting signals that surpass GR's limitations.

Yu-Xin Wang​, et al. "Draft in Progress"

B. P. Abbott et al. (LIGO-Virgo), PRD 100, 104036 (2019). 

• GW Astronomy
• AI for Science · GW Data Analysis
• GW search · Pipeline
• Parameter estimation · Scientific discovery
• Space-based GW Detection

Credit: LIGO Magazine.

• Traditional parameter estimation (PE) techniques rely on Bayesian analysis methods (posteriors + evidence)

• Computing the full 15-dimensional posterior distribution estimate is very time-consuming:
  • Calculating likelihood function
  • Template generation time-consuming
• Machine learning algorithms are expected to speed up! 

• A complete 15-dimensional posterior probability distribution, taking about 1 s (<< \(10^4\) s).

• Prior Sampling: 50,000 Posterior samples in approximately 8 Seconds.

• Capable of calculating evidence
• Processing time: (using 64 CPU cores)
  • less than 1 hour with IMRPhenomXPHM,
  • approximately 10 hours with SEOBNRv4PHM

Nature Physics 18, 1 (2022) 112–17

Big Data Mining and Analytics 5, 1 (2021) 53–63.

（Based on 1912.02762）

【【机器学习】白板推导系列(三十三) ～ 流模型(Flow based Model)】 

（Based on 1912.02762）

• Data: target data \(\mathbf{y}\in\mathbb{R}^{15}\) (with condition data \(\mathbf{x}\)).
• Task:
  • Fitting a flow-based model \(p_{\mathrm{y}}(\mathbf{y} ; \boldsymbol{\theta})\) to a target distribution \(p_{\mathrm{y}}^{*}(\mathbf{y})\)
  • by minimizing KL divergence with respect to the model’s parameters \(\boldsymbol{\theta}=\{\boldsymbol{\phi}, \boldsymbol{\psi}\}\),
  • where \(\boldsymbol{\phi}\) are the parameters of \(T\) and \(\boldsymbol{\psi}\) are the parameters of \(p_{\mathrm{z}}(\mathbf{z})=\mathcal{N}(0,\mathbb{I})\).
• Loss function:
  
  
  
  
   
• Assuming we have a set of samples \(\left\{\mathbf{y}_{n}\right\}_{n=1}^{N}\sim p_{\mathrm{y}}^{*}(\mathbf{y})\),
  
  
  
  Minimizing the above Monte Carlo approximation of the KL divergence is equivalent to fitting the flow-based model to the samples \(\left\{\mathbf{y}_{n}\right\}_{n=1}^{N}\) by maximum likelihood estimation.

归一化流模型示意图

• Bayesian inference, the Holy Grail of gravitational-wave data analysis,
  enables astrophysical interpretation and scientific discoveries.
   

Simulation-Based Inference (SBI)

• SBI \(\Rightarrow\) Fast and precise parameter estimation.
• SBI \(\Rightarrow\) TGR / Cosmology / PTA ...

PRL 127, 24 (2021) 241103.

PRL 130, 17 (2023) 171403.

Real-time gravitational wave science with neural posterior estimation

Sampling with prior knowledge for high-dimensional gravitational wave data analysis

He Wang, et al. Big Data Min. Anal. (2021)

Fast Parameter Inference on Pulsar Timing Arrays with Normalizing Flows

Exact coverage first!

Paradigm

New
discovery

first!

PRD 108, 4 (2023): 044029.

Appreciating the Ringdown Overtone Test of GW150914

• A notable work involves ringdown overtone testing, which, acknowledging the difficulty in achieving DINGO-like precision for complex waveforms, leverages the speed advantage of AI.
• By simulating the signal and \(10^3\) realizations of LIGO noise for each pixel, it accomplishes what is impossible for MCMC methods, prioritizing speed over precision in a strategic trade-off.

arXiv:2404.14286

• 進撃のnflow model in GW inference area.

  • 2002.07656: 5D toy model [1] (PRD)

  • 2008.03312: 15D binary black hole inference [1] (MLST)

  • 2106.12594: Amortized inference and group-equivariant neural posterior estimation [2] (PRL)

  • 2111.13139: Group-equivariant neural posterior estimation [2]

  • 2210.05686: Importance sampling [2] (PRL)

  • 2211.08801: Noise forecasting [2] (PRD)

  • 2305.17161: FMPE

  • 2404.14286: eccentricity of BBHs

1. https://github.com/stephengreen/lfi-gw  (2020)

2. https://github.com/dingo-gw/dingo   (2023.03)

• GW Astronomy
• AI for Science · GW Data Analysis
• GW search · Pipeline
• Parameter estimation · Scientific discovery
• Space-based GW Detection

The analysis of scientific data from space-based GW detection differs significantly from ground-based detection:

• A superposition of overlapping signals ( \(\neq\) isolated event)
• Observations of more waveform periods over different time scales ( \(\neq\) short-duration signals)
• Signal-dominated detection ( \(\neq\) noise-dominated)
• Reliance on more complex techniques for noise assessment
  ( \(\neq\) regular acquisition of signal-free data)

空间引力波探测科学数据处理的挑战与人工智能技术的应用

王赫, 杜明辉, 徐鹏, 周宇峰

2024年, 第54卷, 第7期, 270403

https://doi.org/10.1360/SSPMA-2024-0087

Analyses cannot treat sources independently and sequentially work through a list of candidate detections.

• The data analyses depend on the simultaneous fitting of complete astrophysical, cosmological, and instrument models to the observed data.
• The need for this so-called “global fit” was identified as the primary challenge to the data analysis early in the space-borne mission formulation.

(Sec.8.6 The Red Book)

Global Fit

• The idea of the global fit method is to comprehensively model all astrophysical and instrumental features present in the space-borne gravitational wave data.
   

Technical challenges:

• High dimension
• Highly correlation
• Multimodality
• Trans-dimension

• Global vs. Individual Analysis: While global-fit techniques effectively manage the dense overlapping of signals in space-based GW data, individual pipelines are crucial for detecting unique events.

• Role of Individual Pipelines: These pipelines act as a pre-processing step, focusing on particular types of sources and diving deeper into the data. They refine the analysis by working on the latest best-fit residuals from the global fit.

• Case Study - MBHB Mergers: Mergers of MBHBs often exhibit high SNR between \(10^2\) to \(10^3\), appearing as distinct peaks in data time series. 

• Data curation

  • Model: frequency domain; PhenomD;  TDI-A/E response

  • Input: 1 day length; 15Hz; shape=(2, 3, 2877)

  • Noise: Gaussian stationary from the noise PSD (for training/test) + GB confusion noise (for test)

  • Project: Taiji program

M. Du, B. Liang, HW, P. Xu, Z. Luo, Y. Wu. SCPMA 67, 230412 (2024).

• The top section of the illustration shows the solar system barycenter (SSB) and Taiji frames, with two black dashed arrows symbolizing not two separate GW signals, but rather indicating how the sky location and arrival time of the same GW signal take different values in these two frames.

• The “positive” problem translates the SSB-frame parameters to their Taiji-frame counterparts via a time-dependent mapping \(f_1\), then to the TDI outputs through a time-independent mapping \(f_2\), and an exponential term.

• These steps can be schematically summarized as:
  
  
  where \(\mathcal{T}_\alpha^{A, E}(f)\) is often referred to as the transfer function.

M. Du, B. Liang, HW, P. Xu, Z. Luo, Y. Wu. SCPMA 67, 230412 (2024).

• Consequently, even if the network has only learned the time-dependent relationship between \(\boldsymbol{\theta}_S\) and the TDI response at a specific tref (the 30th day in our case), with the aid of coordinate transformation, it has essentially learned the time-invariant mapping \(f_2\), and can be then generalized to make parameter estimation at any other reference time.
   

• It is worth noting that our method relies on analytical orbits and
  the time-independence of the coordinate transformation \(f_2\).

• The top section of the illustration shows the solar system barycenter (SSB) and Taiji frames, with two black dashed arrows symbolizing not two separate GW signals, but rather indicating how the sky location and arrival time of the same GW signal take different values in these two frames.

• The “positive” problem translates the SSB-frame parameters to their Taiji-frame counterparts via a time-dependent mapping \(f_1\), then to the TDI outputs through a time-independent mapping \(f_2\), and an exponential term.

1 year length

can infer at any other reference time

trained on the 30th day only

M. Du, B. Liang, HW*, P. Xu, Z. Luo, Y. Wu*. SCPMA 67, 230412 (2024).

• Methodology: Utilization of the Kolmogorov-Smirnov (KS) test to compare one-dimensional distributions generated by our algorithms, ensuring the accuracy of parameter estimation.

• Empirical Validation: Conducted extensive testing on simulated signals, injecting 1000 waveforms from the prior with added confusion noise and varying reference times between 1 and 365 days.

• Results: The tests assessed the frequency at which true parameters fell within certain confidence levels, confirming that our credible intervals are well-calibrated and reflect true confidence in the signal parameters.

M. Du, B. Liang, HW, P. Xu, Z. Luo, Y. Wu. SCPMA 67, 230412 (2024).

• Overview of Findings: Nested sampling results indicate minimal expected multimodality in ecliptic coordinates. However, distinct peaks identified in the time of coalescence (\(t_c\)), labeled as NF-1 (dominant) and NF-2 (subdominant), highlight unique multimodal behavior.

• Impact on PE: The presence of these peaks affects the posterior distributions of extrinsic parameters, potentially leading to inaccuracies in \(t_c\) and subsequent parameters due to phase term associations and inherent degeneracies.
• Model Performance: Despite the multimodality, the best-fit values from the NF model closely align with true values within the \(1\sigma\) range for most parameters, and at least \(2\sigma\) for others.
• Comparative Analysis: The ML pipeline tends to produce broader posteriors compared to the Bayesian nested sampling approach. 

（NF = Normalizing Flow model）

M. Du, B. Liang, HW, P. Xu, Z. Luo, Y. Wu. SCPMA 67, 230412 (2024).

Bo Liang, Minghui Du*, He Wang*, et al. “Rapid Parameter Estimation for Merging Massive Black Hole Binaries Using Continuous Normalizing Flows.” arXiv, October 7, 2024. http://arxiv.org/abs/2407.07125. (MLST accepted)

• GW Astronomy
• AI for Science · GW Data Analysis
• GW search · Pipeline
• Parameter estimation · Scientific discovery
• Space-based GW Detection
  • Global-fit

Data Analysis for LISA/Taiji/Tianqin

Credit: Natalia Korsakova
(LISA Symposium 2024 Dubin)

Galactic Compact Binaries​

• Simple signals, fast to evaluate waveform with response.
  1. Verification Binaries (high SNR, more than a dozen) 
  2. tractable .. (higher SNR, nearly ten thousand)
  3. untractable .. (lower SNR, nearly a million)
• Problem is the confusion between signals:
  • Overlapping signals
  • Unknown number of signals
  • Signals that gradually transition to the stochastic background

New catalog with parameters can be found here: https://gitlab.in2p3.fr/LISA/lisa-verification-binaries

The analysis of the best currently known LISA binaries, even making maximal use of the available information about the sources, is susceptible to ambiguity or biases when not simultaneously fitting to the rest of the galactic population.

(copied from Littenberg et al. 2404.03046)

credit: Kupfer et al, arXiv:2302.12719

credit: Kupfer et al, arXiv:2302.12719

Massive Black Hole Binaries

• MBHBs for different masses have very different morphologies.
• Important to develop methods to accelerate data analysis, particularly for low mass MBHBs.
  • reduced order modelling, reduced order qudratures, accelerated likelihood evaluations...

Credit: Geraint Pratten​ (LISA Symposium 2024 Dubin)

Credit: Sylvain Marsat
 (LISA Symposium 2024 Dubin)

Massive Black Hole Binaries

The addition of GBs biases the parameter recovery of masses and spins away from the injected values, reinforcing the need for a global fit pipeline which will simultaneously fit the parameters of the GB signals before estimating the parameters of MBHBs.  (Copied from Weaving, et al. CQG, 2023, 41(2): 025006.)

(Sec.8.6 Red Book)

Global Fit

• The idea of the global fit method is to comprehensively model all astrophysical and instrumental features present in the space-borne gravitational wave data.
• This approach not only focuses on the signal from a single source, but attempts to capture the combined effects of all sources in the data, conducting a comprehensive analysis of the entire dataset to identify and model all potential signal and noise sources.
   

Technical challenges:

• High dimension
• Highly correlation
• Multimodality
• Trans-dimension

GLASS (Littenberg&Cornish 2023)

• MCMC-based

Strub et al. (2024,2023)

• MLE-based + FIM

GLASS (Littenberg&Cornish 2023)

• MCMC-based

Strub et al. (2024,2023)

• MLE-based + FIM

GLASS (Littenberg&Cornish 2023)

• MCMC-based

Strub et al. (2024,2023)

• MLE-based + FIM

GLASS (Littenberg&Cornish 2023)

• MCMC-based

Strub et al. (2024,2023)

• MLE-based + FIM

GLASS (Littenberg&Cornish 2023)

• MCMC-based
  1. 资源优化挑战：这涉及到如何在最小化等待时间和最大化计算节点效率的过程中, 进行资源分配和优化策略. 在处理大规模、高维度的数据时, 这是一个复杂的平衡问题.
  2. 迭代成本平衡难题：在不同计算步骤（如对MBHB和GB 内部的跨维度模型更新）之间平衡成本是一个挑战.
     需要考虑到整体收敛时间和计算资源, 以优化整个分析过程.
  3. 具体的技术限制:
     • Heterodyned步骤限制：在MBHB模型处理中，Heterodyned步骤成为了一个瓶颈, 其在每次迭代中的成本对整体计算成本有重要影响.
     • 收敛时间过长：对于频率最高的UCB, 其收敛时间较长, 这增加了迭代次数和整体分析的时间和资源消耗.

Strub et al. (2024,2023)

• MLE-based + FIM
  • The found MLEs of both MBHBs and GBs can be used as proposals of computationally expensive MCMC searches to improve the sampling efficiency.

空间引力波探测科学数据处理的挑战与人工智能技术的应用

王赫, 杜明辉, 徐鹏, 周宇峰

2024年, 第54卷, 第7期, 270403

https://doi.org/10.1360/SSPMA-2024-0087

GLASS (Littenberg&Cornish 2023)

• MCMC-based

中国科学院计算机网络信息中心“东方”超级计算系统 (全国产CPU/GPGPU)

Use Ray for distributed computing on both CPU anD GPUs

Neural density estimation

• Density fit for posterior distributions
  • use the old posterior to form a proposal for the extended data.
• Density fit for the Galaxy
  • fitt a Galaxy model for joint distribution for \((A, \beta, \lambda)\).
• ...

Neural density estimation

• Density fit for posterior distributions
  • use the old posterior to form a proposal for the extended data.
• Density fit for the Galaxy
  • fitt a Galaxy model for joint distribution for \((A, \beta, \lambda)\).
• ...

Ref:

• Ashton, G, and C Talbot. MNRAS 507, no. 2 (2021): 2037–51.
• Korsakova, N, et al. (2402.13701)
• Wouters, T, et al. (2404.11397​)