Nov. 7, 2023, 17:05-17:20

International Workshop on Intelligent Computing in Astronomy

"Computing Senses Cosmos" (2023) @Zhejiang Lab, Hangzhou, China

Gravitational Wave Detection and AI Technology:
New Methods for Unveiling the Mysteries of the Universe

He Wang (王赫)

hewang@ucas.ac.cn

International Centre for Theoretical Physics Asia-Pacific (ICTP-AP), UCAS

Taiji Laboratory for Gravitational Wave Universe, UCAS
on behalf of LVK Collaboration

  • Thriving Advancements in AI for GW Astronomy
    • GW astronomy 1
    • AI4GW 1
  • AI for Science: GW Astronomy
    • GW Searches (Earth/Space-based)
      • MF -> CNN -> MFCNN -> more dims 2 (Discriminative model)
    • Noise Characterization (Earth/Space-based)
      • Transformer 1
      • NC. 1
  • AI Predicting the Universe: Opportunities and Challenges (Parameter Estimation)
    • SBI -> ​​​​ ​TGR / SGWB / Cosmology / EOS ...
    • Challenges: 可解释性、表征能力(space-based)
  • Key Takeaways 2
    • GWToolkit (Ray+Redis)
  • Thriving Advancements in AI for GW Astronomy
    • GW Searches
    • Parameter Estimation
    • Noise Characterization
  • Space-based Gravitational Wave Data Analysis
  • AI for Science: GW Astronomy
  • Key Takeaways

Taiji

BEFORE

AFTER

  • In 1916, A. Einstein proposed the GR and predicted the existence of GW.

  • Gravitational waves (GW) are a strong field effect in the GR.

    • 2015: the first experimental detection of GW from the merger of two black holes was achieved.

    • 2017: the first multi-messenger detection of a BNS signal was achieved, marking the beginning of multi-messenger astronomy.

    • 2017: the Nobel Prize in Physics was awarded for the detection of GW.

    • As of now: more than 90 gravitational wave events have been discovered.

    • O4, which began on May 24th 2023, is currently in progress.

Gravitational waves generated by binary black holes system

GW detector

LIGO-VIRGO-KAGRA network

2017 Nobel Prize in Physics

Gravitational Wave Astronomy

Gravitational Wave Astronomy

  • Fundamental Physics
    • Existence of gravitational waves
    • To put constraints on the properties of gravitons
  • Astrophysics
    • Refine our understanding of stellar evolution
    • and the behavior of matter under extreme conditions.
  • Cosmology
    • The measurement of the Hubble constant
    • Dark energy

©Floor Broekgaarden (repo)

GWTC-3

  • Detecting gravitational waves require a mix of FIVE key ingredients:
    1. good detector technology
    2. good waveform predictions
    3. good data analysis methodology and technology
    4. coincident observations in several independent detectors
    5. coincident observations in electromagnetic astronomy

—— Bernard F. Schutz

​​DOI: 10.1063/1.1629411 

Technical Challenges: Data Processing for GW

GW Data characteristics










 

  • ​Noise: non-Gaussian and non-stationary

  • Signal:

    • (Earth-based) A low signal-to-noise ratio (SNR) which is typically about 1/100 of the noise amplitude (-60 dB).

    • (Space-based) A superposition of all GW signals (e.g.: \(10^4\) of GBs, \(10\sim10^2\) of SMBHs, and \(10\sim10^3\) of EMRIs, etc.) received during the mission's observational run.

Matched filtering techniques (匹配滤波方法)
 

  • In Gaussian and stationary noise environments, the optimal linear algorithm for extracting weak signals

  • Works by correlating a known signal model \(h(t)\) (template) with the data.
  • Starting with data: \(d(t) = h(t) + n(t)\).
  • Defining the matched-filtering SNR \(\rho(t)\):
    \(\rho^2(t)\equiv\frac{1}{\langle h|h \rangle}|\langle d|h \rangle(t)|^2 \) , where \(\langle d|h \rangle (t) = 4\int^\infty_0\frac{\tilde{d}(f)\tilde{h}^*(f)}{S_n(f)}e^{2\pi ift}df \) ,
    \(\langle h|h \rangle = 4\int^\infty_0\frac{\tilde{h}(f)\tilde{h}^*(f)}{S_n(f)}df \), \(S_n(f)\) is noise power spectral density (one-sided).

Text

Text

LIGO-VIRGO-KAGRA

LISA / Taiji project

AI for Gravitational Wave Astronomy

Pioneering works utilizing CNN







 

  • The most common and direct approach, from Computer Vision (CV) to GW signal processing: pixel point \(\Rightarrow\) sampling point.









 

  • Convolutional neural networks (CNN) can achieve comparable performance to Matched Filtering and surpass them in terms of execution speed (with GPU support) under Gaussian stationary noise.

Text

PRL, 2018, 120(14): 141103.

PRD, 2018, 97(4): 044039.

AI for Science \(\rightarrow\) AI for GW Astronomy















 

  • Artificial Intelligence (AI) has great potential to revolutionize gravitational wave astronomy by improving data analysis, modeling, and detector development.
  • Representation and supervised learning crucially extract features from GW signals, autonomously identifying informative features and leveraging labeled data for accuracy.

Exported: Oct, 2023 (in preparation)

Text

GW Detection: Discriminative Learning

CL.M., W.W., H.W., et al. PRD (2022)

Ensemble learning

  • Leverages statistical approaches to utilize more information for making informed decisions by combining multiple models.

Text

MLGWSC-1

  • The majority of AI algorithms used for testing are highly sensitive to non-Gaussian real noise backgrounds, resulting in high false positive rates.

Text

(MFCNN group) H.W., et al. PRD (2023)

Expanding the dimension of the output

  • is to call more information to make decisions in improving AI models.

Text

CL.M., W.W., H.W., et al. PRD (2023)

Matched-filtering Convolutional Neural Network (MFCNN)

  • GW templates can be utilized as recognizable features for signal detection.
  • It is feasible to generalize both matched-filtering and neural networks.
  • Linear filters (i.e., matched-filtering) in signal processing can be reformulated as neural layers (i.e., CNNs).

Text

Real-time GW searches for GW150914

H.W., et al. PRD (2020)

GW Denoising: Discriminative Learning

Large Language Model (LLM)

  • Representation learning is conducted using a billion-scale transformer-based model (encoder-only).
  • Segments of sampling points are considered as the processing units for each character: tokens \(\Rightarrow\) segments of sampling point.

H.W., et al. arXiv:2212.14283
DOI: 10.21203/rs.3.rs-2452860/v1

TY.Z., R.L., H.W., et al. Commun. Phys. (2023)

Text

Large Language Model (LLM)

  • Noise suppression and recovery of GW events in actual LIGO recordings.
  • "One Model to Rule Them All":
    • Signal extraction of EMRI / MBHB / GBs / SGWB in the LISA mock challenge (LDC).

Text

H.W., et al. arXiv:2212.14283
DOI: 10.21203/rs.3.rs-2452860/v1

TY.Z., R.L., H.W., et al. Commun. Phys. (2023)

MBHB

EMRI

GB

GW Denoising: Discriminative Learning

AI Predicting the Universe: Opportunities and Challenges

  • Bayesian inference, the Holy Grail of gravitational-wave data analysis,
    enables astrophysical interpretation and scientific discoveries.

Simulation-Based Inference (SBI)

  • SBI \(\Rightarrow\) Fast and precise parameter estimation.
  • SBI \(\Rightarrow\) Test of General Relativity (TGR) / Cosmology / PTA ...

PRL 131, 17 (2023): 171403.

Angular Power Spectrum of Gravitational-Wave Transient Sources as a Probe of the Large-Scale Structure

Angular Power Spectrum of Gravitational-Wave Transient Sources as a Probe of the Large-Scale Structure

Text

arXiv:2310.12209

Fast Parameter Inference on Pulsar Timing Arrays with Normalizing Flows

PRD 108, 4 (2023): 044029.

Neural Posterior Estimation with Guaranteed Exact Coverage: The Ringdown of GW150914

arXiv:2310.13405, LIGO-P2300306

Cosmological Inference using Gravitational Waves and Normalising Flows

PRL 127, 24 (2021) 241103.

PRL 130, 17 (2023) 171403.

Real-time gravitational wave science with neural posterior estimation

Sampling with prior knowledge for high-dimensional gravitational wave data analysis

H.W., et al. Big Data Min. Anal. (2021)

AI Predicting the Universe: Opportunities and Challenges

  • Exploring the importance of understanding how AI models make predictions in scientific research.
    • The critical role of generative models (生成模型是关键)
    • Quantifying uncertainty: a key aspect (不确定性量化问题)
    • Fostering controllable and reliable models (模型的可控可信问题)

Text-to-image

AI or Bayes

"A running dog"
  • The most common and direct approach, from Artificial Intelligence Generated Content (AIGC) to GW statistical inference: pixel point \(\Rightarrow\) inferred parameter.

AI Predicting the Universe: Opportunities and Challenges

"A running dog"

Text-to-image

"A corgi running on the street"

AI or Bayes

  • The most common and direct approach, from Artificial Intelligence Generated Content (AIGC) to GW statistical inference: pixel point \(\Rightarrow\) inferred parameter.

A picture is worth a thousand words.

A fraction of a thousand words.

Credit: 李宏毅

  • Exploring the importance of understanding how AI models make predictions in scientific research.
    • The critical role of generative models (生成模型是关键)
    • Quantifying uncertainty: a key aspect (不确定性量化问题)
    • Fostering controllable and reliable models (模型的可控可信问题)

Key Takeaways

Nature Physics 18, 1 (2022): 9–11

arXiv:2308.05510, accepted by SCPMA.

On-going

  • About Data: We are developing a software toolkit called GWToolkit that integrates gravitational wave signal processing and generic asynchronous data pipeline capabilities.
  • About models: Let's continue to leverage representation learning and explore the use of GPT-like language models for scientific discovery.

Text

Insights

  • AI is not just a tool; it is a revolutionary pathway for scientific discoveries.
  • The future is filled with technical challenges, even when it comes to using AI, including:
    • Super high-dimensional parameter inference
    • Super high-dimension of GW data strains
  • "Gravitational Wave Astrostatistics" has the potential to become a new field of knowledge.
  • Improve the interpretability of AI models, as it is essential for enhanced and trustworthy discoveries.

Text

Dynamic training samples in memory.

Key Takeaways

Nature Physics 18, 1 (2022): 9–11

arXiv:2308.05510, accepted by SCPMA.

On-going

  • About Data: We are developing a software toolkit called GWToolkit that integrates gravitational wave signal processing and generic asynchronous data pipeline capabilities.
  • About models: Let's continue to leverage representation learning and explore the use of GPT-like language models for scientific discovery.

Text

Insights

  • AI is not just a tool; it is a revolutionary pathway for scientific discoveries.
  • The future is filled with technical challenges, even when it comes to using AI, including:
    • Super high-dimensional parameter inference
    • Super high-dimension of GW data strains
  • "Gravitational Wave Astrostatistics" has the potential to become a new field of knowledge.
  • Improve the interpretability of AI models, as it is essential for enhanced and trustworthy discoveries.

Text

Dynamic training samples in memory.

for _ in range(num_of_audiences):
    print('Thank you for your attention! 🙏')

This slide: https://slides.com/iphysresearch/2023ai4astro_zhejianglab

Key Takeaways

  1. Gravitational-wave astronomy turns to AI:
    • A thriving and highly competitive research field on the international stage.
  2. Is AI just a tool? Certainly not! It's a revolutionary pathway for scientific discoveries:
    • Enabling new discoveries and insights into the universe.
  3. Addressing Challenges in AI for GW astronomy: essential for enhanced discoveries
    • LLM
    • Scientific infrastructure
    • Interpretability
for _ in range(num_of_audiences):
    print('Thank you for your attention! 🙏')

This slide: https://slides.com/iphysresearch/2023ml_astromoy

Smith, Rory. Nature Physics 18, 1 (2022): 9–11

  • The use of calibration techniques in gravitational wave astronomy
    • high dims of params
    • 过多的数据长度 vs params   (提及我们的PE工作)
    • High-Dimensional Gravitational Wave Astrostatistics
  • 数据：数据增益；提及 GWToolkit
  • 模型：继续发挥 Representation learning，尝试 GPT类的语言模型 for scientific discovery
  • Interpretability is essential

AI for Gravitational Wave: Parameter Estimation

Bayesian statistics

  • Traditional parameter estimation (PE) techniques rely on Bayesian analysis methods (posteriors + evidence)

  • Computing the full 15-dimensional posterior distribution estimate is very time-consuming:
    • Calculating likelihood function
    • Template generation time-consuming
  • Machine learning algorithms are expected to speed up! 

Data quality improvement

Credit: Marco Cavaglià 

LIGO-Virgo data processing

GW searches

Astrophsical interpretation of GW sources

CQG. 37 (2020) 055002

AI for Gravitational Wave: Parameter Estimation

  • A complete 15-dimensional posterior probability distribution, taking about 1 s (<< \(10^4\) s).

LIGO-Virgo data processing

Nature Physics 18, 1 (2022) 112–17

  • Prior Sampling: 50,000 Posterior samples in approximately 8 Seconds.

Big Data Mining and Analytics 5, 1 (2021) 53–63.

  • Capable of calculating evidence
  • Processing time: (using 64 CPU cores)
    • less than 1 hour with IMRPhenomXPHM,
    • approximately 10 hours with SEOBNRv4PHM

PRL 127, 24 (2021) 241103.

PRL 130, 17 (2023) 171403.

Taiji

Space-based Gravitational Wave Data Analysis

  • 空间引力波探测主要有以下 4 类波源：

    • 恒星级质量的致密双星 (黑洞、中子星、白矮星以及它们的两两组合) 的旋近

    • 双白矮星的并合、​超大质量双黑洞的并合

    • 极端质量比旋进 (通常是一个恒星级致密天体绕着一个超大质量黑洞的旋进）

    • 宇宙中可能存在的中等质量双黑洞以及前面这些源的信号叠加形成的引力波背景

LIGO-G2300554

Credit: ESA, K. Holley-Bockelmann

Space-based Gravitational Wave Data Analysis

  • Global-fit approach + AI-powered approach
    • Searches: Discriminative model
    • Inference: Generative model

PLB 841 (2023) 137904.

Communications Physics, 2023, 6(1): 212.

arXiv:2308.05510

Universe, 2023, 9(9): 407.

Nature Astronomy, 2022, 6(12): 1356-1363.

Nature Astronomy, 2022, 6(12): 1334-1338.

Schematic view of the global fit approach.

  • AI serves as a valuable tool in gravitational wave astronomy:
    (Big data & Computational Complexity)
    • Enhancing data analysis,
    • Noise reduction, and
    • Parameter estimation.
    • It streamlines the research process and allows scientists to focus on the most relevant information.
  • Beyond a Tool: AI transcends its role as a mere tool by enabling scientific discovery in GW astronomy.
    • Characterization of GW signals involves
      • Exploring beyond the scope of GR ,
      • Enabling real-time inference
    • "Curse of Dimensionality" in inference
      • Overlapping signal  (In progress)
      • Hierarchical Bayesian Analysis (In progress)
    • Test of GR
      • Tighter parameter constraints of variance
      • Guaranteed exact coverage
    • ...

Challenge in GW Data Analysis: Lessons and Future

GW170817
GW190412
GW190814

Bayes factor (MCMC)

PRD 101, 10 (2020) 104003.

(In preparation)

Challenge in GW Data Analysis: Lessons and Future

arXiv: 2211.01304

  • AI serves as a valuable tool in gravitational wave astronomy:
    (Big data & Computational Complexity)
    • Enhancing data analysis,
    • Noise reduction, and
    • Parameter estimation.
    • It streamlines the research process and allows scientists to focus on the most relevant information.
  • Beyond a Tool: AI transcends its role as a mere tool by enabling scientific discovery in GW astronomy.
    • Characterization of GW signals involves
      • Exploring beyond the scope of GR , 
      • Enabling real-time inference
    • "Curse of Dimensionality" in inference
      • Overlapping signal  (In progress)
      • Hierarchical Bayesian Analysis (In progress)
    • Test of GR
      • Tighter parameter constraints of variance
      • Guaranteed exact coverage
    • ...

Combining inferences from multiple sources

PRD 99, 124044 (2019)

arXiv:2305.18528

Challenge in GW Data Analysis: Lessons and Future

  • AI serves as a valuable tool in gravitational wave astronomy:
    (Big data & Computational Complexity)
    • Enhancing data analysis,
    • Noise reduction, and
    • Parameter estimation.
    • It streamlines the research process and allows scientists to focus on the most relevant information.
  • Beyond a Tool: AI transcends its role as a mere tool by enabling scientific discovery in GW astronomy.
    • Characterization of GW signals involves
      • Exploring beyond the scope of GR , 
      • Enabling real-time inference
    • "Curse of Dimensionality" in inference
      • Overlapping signal  (In progress)
      • Hierarchical Bayesian Analysis (In progress)
    • Test of GR
      • Tighter parameter constraints of variance
      • Guaranteed exact coverage
    • ...

ICML2023

AI for Science: GW Astronomy

  • Using Large Language Models (LLMs) for scientific discovery.
  • The need for a scientific infrastructure for AI in Science.
  • 以国家天文科学数据中心在线服务平台为基础，开发适用于引力波天文学研究的引力波探测开源数据门户 (2023-2025)。
  • 开发一套集成引力波数据处理与通用异步计算功能的软件工具包：GWToolkit

AI for Science: GW Astronomy

  • Exploring the importance of understanding how AI models make predictions in scientific research.
    • The critical role of generative models (生成模型是关键)
    • Quantifying uncertainty: a key aspect (不确定性量化问题)
    • Fostering controllable and reliable models (模型的可控可信问题)

Bayes

AI

Credit: 李宏毅

Text-to-image

LIGO-P2300306

AI for Science: GW Astronomy

  • Exploring the importance of understanding how AI models make predictions in scientific research.
    • The critical role of generative models (生成模型是关键)
    • Quantifying uncertainty: a key aspect (不确定性量化问题)
    • Fostering controllable and reliable models (模型的可控可信问题)

Bayes

AI

Credit: 李宏毅

Text-to-image

LIGO-P2300306

Key Takeaways

  1. Gravitational-wave astronomy turns to AI:
    • A thriving and highly competitive research field on the international stage.
  2. Is AI just a tool? Certainly not! It's a revolutionary pathway for scientific discoveries:
    • Enabling new discoveries and insights into the universe.
  3. Addressing Challenges in AI for GW astronomy: essential for enhanced discoveries
    • LLM
    • Scientific infrastructure
    • Interpretability
for _ in range(num_of_audiences):
    print('Thank you for your attention! 🙏')

This slide: https://slides.com/iphysresearch/2023ml_astromoy

Smith, Rory. Nature Physics 18, 1 (2022): 9–11

Key Takeaways

“国际理论物理中心(亚太地区)” 经联合国教科文组织第38届大会审议通过。由中国科学院、基金委和国际理论物理中心共同建设，是进行基础科学前沿与相关交叉科学领域高水平科研、教育和培训的非营利性组织，是联合国教科文组织基础科学方面的在国内的第一个二类中心。

Outlook

  • 值得关注的 AI 技术：
    • Large Language Model (LLM)
    • AI generated content (AIGC)

WaveFormer

Transformer: 750x / 2yrs

AI for Science

  • 2016年，AlphaGo 第一版发表在了 Nature 杂志上

  • 2021年，AI预测蛋白质结构登上 Science、Nature 年度技术突破，潜力无穷

  • 2022年，DeepMind团队通过游戏训练AI发现矩阵乘法算法问题​

  • 《达摩院2022十大科技趋势》将 AI for Science 列为重要趋势

    • “人工智能成为科学家的新生产工具，催生科研新范式”

  • AI for Science：为科学带来了模型与数据双驱动的新的研究范式

    • AI + 数学、AI + 化学、AI + 医药、AI + 物理、AI + 天文 ...

AlphaGo 围棋机器人

AlphaTensor 发现矩阵算法

AlphaFold 蛋白质结构预测