He Wang PRO
Knowledge increases by sharing but not by saving.
引力波数据分析系列报告
时间:2023年6月11日(周日)下午14:00
中国科学院力学研究所怀柔园区1号楼430会议室
机器学习在引力波数据分析中的应用
王赫
hewang@ucas.ac.cn
中国科学院大学 · 国际理论物理中心（亚太地区）
——参数估计和数据降噪
- Gravitational Wave Astronomy
-
Gravitational Wave Parameter Estimation
- DINGO
-
Gravitational Wave Observational Data Denoising
- WaveFormer
-
Outlook
- LLM / ChatGPT
- Software development
- PE
Content
-
In 1916, Einstein proposed the GR and predicted the existence of GW.
-
Gravitational waves (GW) are a strong field effect in the GR.
-
2015: the first experimental detection of GW from the merger of two black holes was achieved.
-
2017: the first multi-messenger detection of a BHS merger was achieved, marking the beginning of multi-messenger astronomy.
-
2017: the Nobel Prize in Physics was awarded for the detection of GW.
-
As of now: more than 90 gravitational wave events have been discovered.
-
双星并合系统产生的引力波波源
引力波振幅的测量
地面引力波探测器网络
2017 年诺贝尔物理学奖
Gravitational Wave Astronomy
Gravitational Wave Astronomy
-
Fundamental physics
- Existence of gravitational waves
- To put constraints on the properties of gravitons
-
Astrophysics
- Refine our understanding of stellar evolution
- and the behavior of matter under extreme conditions.
-
Cosmology
- The measurement of the Hubble constant
- Dark energy
GWTC-3
The First GW Event: GW150914
- Detecting gravitational waves require a mix of FIVE key ingredients:
- good detector technology
- good waveform predictions
- good data analysis methodology and technology
- coincident observations in several independent detectors
- coincident observations in electromagnetic astronomy
—— Bernard F. Schutz
DOI:10.1063/1.1629411
AI for Gravitational Wave
- AI for Science \(\rightarrow\) AI for GW
- Artificial Intelligence (AI) has great potential to revolutionize gravitational wave astronomy by improving data analysis, modeling, and detector development.
AI for Gravitational Wave
-
GW Data characteristics:
-
Noise: non-Gaussian and non-stationary
-
Signal: A low signal-to-noise ratio (SNR) which is typically about 1/100 of the noise amplitude (-60 dB)
-
Data quality improvement
Credit: Marco Cavaglià
LIGO-Virgo data processing
GW waveform modeling
GW searches
Astrophsical interpretation of GW sources
Gravitational Wave Parameter Estimation
Gravitational Wave Parameter Estimation
Thrane, Eric, and Colm Talbot. “An Introduction to Bayesian Inference in Gravitational-Wave Astronomy: Parameter Estimation, Model Selection, and Hierarchical Models.” Publications of the Astronomical Society of Australia 36 (September 2019): e010. https://doi.org/10.1017/pasa.2019.2.
Likelihood
-
Traditional parameter estimation (PE) techniques rely on Bayesian analysis methods (posteriors + evidence)
-
For CBC, LIGO-Virgo parameter estimation software:
-
Bilby / LALInference / PyCBC Inference / RIFT
-
- Computing the full 15-dimensional posterior distribution estimate is very time-consuming:
- Calculating likelihood function
- Template generation time-consuming
- Machine learning algorithms are expected to speed up! If it can be achieved in real-time, it will be more helpful for signal detection.
Gravitational Wave Parameter Estimation
- Deep Generative Models: Conditional Variational Autoencoder (CVAE)
- Noise Power Spectrum Based on Design Sensitivity, Gaussian Simulated Noise (Proof-of-principle studies)
- A complete 15-dimensional posterior probability distribution, taking about 1 second
An example: Posterior probability distribution of the complete 15-dimensional parameters
Gravitational Wave Parameter Estimation
- Deep Generative Models: Normalizing Flow Models (Nflow)
- Noise Power Spectra Based on GW150914 Nearby Noise Estimation
- First Implementation of Full Posterior Parameter Estimation for Real Gravitational Wave Event GW150914
- 50,000 Posterior Samples in Approximately 8 Seconds
Wang H, Cao Z, et al. Big Data Mining and Analytics, 2021
Gravitational Wave Parameter Estimation
- 深度生成模型：归一化流模型 (Nflow)
-
DINGO
- 测试 GWTC-1 的 BBH 事件
- 耗时 < 1 min (≈ 20 s, IMRPhenomPv2)
- 开始为 O4 部署，有望成为新的引力波信号搜寻流水线
-
DINGO-IS
- 测试 GWTC-3 中 42 BBH 事件
- 耗时 ≲ 1 h (IMRPhenomXPHM)，≈ 10 h (SEOBNRv4PHM, 64 CPU cores)
- 能够计算 evidence
AI for Gravitational Wave
-
GW Data characteristics:
-
Noise: non-Gaussian and non-stationary
-
Signal: A low signal-to-noise ratio (SNR) which is typically about 1/100 of the noise amplitude (-60 dB)
-
Data quality improvement
Credit: Marco Cavaglià
LIGO-Virgo data processing
GW waveform modeling
GW searches
Astrophsical interpretation of GW sources
Gravitational Wave Detection
PRL, 2018, 120(14): 141103.
-
Matched filtering techniques (匹配滤波方法)
-
In Gaussian and stationary noise environments, the optimal linear algorithm for extracting weak signals
-
- Convolutional neural networks (CNN) can achieve comparable performance to MF, and outperform them in terms of execution speed (with GPU support).
... under Gaussian stationary noise.
PRD, 2018, 97(4): 044039.
-
GW Data characteristics:
-
Noise: non-Gaussian and non-stationary
-
Signal: A low signal-to-noise ratio (SNR) which is typically about 1/100 of the noise amplitude (-60 dB)
-
Gravitational Wave Detection
Convolutional Neural Network (ConvNet or CNN)
- Test the CNN model on real LIGO recordings and GW events, the output is very bad 😰
Matched-filtering Convolutional Neural Network (MFCNN)
GW150914
GW151012
MFCNN
MFCNN
GPS time
GW150914
GW151012
GPS time
Wang H, et al. PRD (2020)
Gravitational Wave Detection
- 改进并开发神经网络模型，以适应真实的引力波观测数据的任务
- 匹配滤波算法当中的波形模板 \(\rightarrow\) 卷积层中的卷积核权重参数
- Matched-filtering layer (匹配滤波感知层)
- 可以准确探测到 GWTC-1 中的 11 个真实引力波事件，甚至包括 GW170817
- 可以直接应用于空间引力波数据场景，探测 MBHBs
GW170817
GW190412
GW190814
mass distribution
Ruan WH, Wang H, et al. PLB (2023)
Matched-filtering Convolutional Neural Network (MFCNN)
Wang H, et al. PRD (2020)
Gravitational Wave Detection
- 改进并开发神经网络模型，以适应真实的引力波观测数据的任务
- 匹配滤波算法当中的波形模板 \(\rightarrow\) 卷积层中的卷积核权重参数
- Matched-filtering layer (匹配滤波感知层)
- “神经网络化”的探测统计量 (匹配滤波信噪比)
Frequency domain
\langle h|h \rangle = 4\int^\infty_0\frac{\tilde{h}(f)\tilde{h}^*(f)}{S_n(f)}df
\langle d|h \rangle (t) = 4\int^\infty_0\frac{\tilde{d}(f)\tilde{h}^*(f)}{S_n(f)}e^{2\pi ift}df
(whitening)
Time domain
(normalizing)
(matched-filtering)
\langle h|h \rangle \sim [\bar{h}(t) \ast \bar{h}(-t)]|_{t=0}
\langle d|h \rangle (t) \sim \,\bar{d}(t)\ast\bar{h}(-t)
where \(S_n(|f|)\) is the one-sided average PSD of \(d(t)\)
\bar{S_n}(t)=\int^{+\infty}_{-\infty}S_n^{-1/2}(f)e^{2\pi ift}df
\left\{\begin{matrix}
\bar{d}(t) = d(t) * \bar{S}_n(t) \\
\bar{h}(t) = h(t) * \bar{S}_n(t)
\end{matrix}\right.
In the 1-D convolution (\(*\)), given input data with shape [batch size, channel, length] :
（A schematic illustration for a unit of convolution layer)
\int\tilde{x}_1(f) \cdot \tilde{x}_2(f) e^{2\pi ift}df= x_1(t)*x_2(t)
x_1(t)*x_2^*(-t) = x_1(t)\star x_2(t)
\int\tilde{x}_1(f) \cdot \tilde{x}^*_2(f) e^{2\pi ift}df= x_1(t)\star x_2(t)
Matched-filtering Convolutional Neural Network (MFCNN)
Wang H, et al. PRD (2020)
Gravitational Wave Detection
- 改进并开发神经网络模型，以适应真实的引力波观测数据的任务
- 匹配滤波算法当中的波形模板 \(\rightarrow\) 卷积层中的卷积核权重参数
- Matched-filtering layer (匹配滤波感知层)
- “神经网络化”的探测统计量 (匹配滤波信噪比)
- Insight: 引力波信号处理 \(\rightarrow\) 智能引力波信号处理
Matched-filtering Convolutional Neural Network (MFCNN)
Wang H, et al. PRD (2020)
An example of transfer function:
CNN
RNN
Gravitational Wave Detection
- 改进并开发神经网络模型，以适应真实的引力波观测数据的任务
- 匹配滤波算法当中的波形模板 \(\rightarrow\) 卷积层中的卷积核权重参数
- Matched-filtering layer (匹配滤波感知层)
- “神经网络化”的探测统计量 (匹配滤波信噪比)
- The first machine learning gravitational wave signal search challenge (MLGWSC1) https://github.com/gwastro/ml-mock-data-challenge-1
H1
L1
search scope
(MFCNN group) Wang H, et al. PRD (2023)
Gravitational Wave Observational Data Denoising
-
Billion-scale transformer-based model (WaveFormer)
- Suppression on realistic noise, and
- Recovery of injections / GW events
- Application:
- Data quality improvement
arXiv:2212.14283, DOI: 10.21203/rs.3.rs-2452860/v1
BEFORE
AFTER
Gravitational Wave Observational Data Denoising
- Billion-scale transformer-based model (WaveFormer)
- Suppression on realistic noise, and
- Recovery of injections / GW events
- Application:
- Data quality improvement
BEFORE
AFTER
Gravitational Wave Observational Data Denoising
- Billion-scale transformer-based model (WaveFormer)
- Suppression on realistic noise, and
- Recovery of injections / GW events
- Application:
- Data quality improvement
Bacon P. et al. arXiv: 2205.13513
Gravitational Wave Observational Data Denoising
- Billion-scale transformer-based model (WaveFormer)
- Suppression on realistic noise, and
- Recovery of injections / GW events
- Application:
- Data quality improvement
Bacon P. et al. arXiv: 2205.13513
Murali C & Lumley D. arXiv: 2210.01718
Wei W and Huerta E A. PLB 2020
Chatterjee C, Wen L, et al. PRD 2021
arXiv:2212.14283, DOI: 10.21203/rs.3.rs-2452860/v1
GW170823
Gravitational Wave Observational Data Denoising
- Billion-scale transformer-based model (WaveFormer)
- Suppression on realistic noise, and
- Recovery of injections / GW events
- Data quality improvement
arXiv:2212.14283, DOI: 10.21203/rs.3.rs-2452860/v1
Outlook
- 值得关注的 AI 技术：
- Large Language Model (LLM)
- AI generated content (AIGC)
WaveFormer
Transformer: 750x / 2yrs
Outlook
- 值得关注的 AI 技术：LLM, AIGC
- 软体开发: GWToolkit powered by Ray.
- Taiji Toolkit ?
- 开源 vs 闭源
- 合作 vs 外包
- 学术声誉的共享与共赢
- 人才培养 vs 团队建设
Outlook
Recent Updates to Rapid PE
- Pathak et al. (2210.02706). Rapid reconstruction of compact binary sources using meshfree approximation
- Wofford et al. (2210.07912). Improving performance for GW PE with an efficient and highly-parallelized algorithm
- Islam et al. (2210.16278). Factorized PE for Real-Time GW Inference
- Digman & Cornish. (2212.04600). PE for Stellar-Origin Black Hole Mergers In LISA
- Yelikar et al. (2301.01337). Low-latency PE enabled by a Gaussian likelihood approximation for RIFT
- Wong et al. (2302.05333). Fast GW PE without compromises
- Tiwari et al. (2303.01463). VARAHA: A Fast Non-Markovian sampler for estimating GW posteriors
- Karnesis et al. (2303.02164). Eryn : A multi-purpose sampler for Bayesian inference
- Fairhurst et al. (2304.03731). Fast inference of CBC properties using the information encoded in the GW signal
- ...
- 值得关注的 AI 技术：LLM, AIGC
- 软体开发: GWToolkit powered by Ray.
- 参数估计：(引力波数据分析领域的“圣杯”)
- Bayes inference
- "Curse of Dimensionality"
- Test of GR
Neural Posterior Estimation with guaranteed exact coverage: the ringdown of GW150914
Normalizing Flows as an Avenue to Studying Overlapping Gravitational Wave Signals
-
2002.07656: 5D toy model [1] (PRD)
-
2008.03312: 15D binary black hole inference [1] (MLST)
-
2106.12594: Amortized inference and group-equivariant neural PE [2] (PRL)
-
2111.13139: Group-equivariant neural PE [2]
-
2210.05686: Importance sampling [2]
-
2211.08801: Noise forecasting [2]
-
https://github.com/dingo-gw/dingo (2023.03)
PRD 99, 124044 (2019)
Combining inferences from multiple sources
for _ in range(num_of_audiences):
print('Thank you for your attention! 🙏')Outlook
Recent Updates to Rapid PE
- Pathak et al. (2210.02706). Rapid reconstruction of compact binary sources using meshfree approximation
- Wofford et al. (2210.07912). Improving performance for GW PE with an efficient and highly-parallelized algorithm
- Islam et al. (2210.16278). Factorized PE for Real-Time GW Inference
- Digman & Cornish. (2212.04600). PE for Stellar-Origin Black Hole Mergers In LISA
- Yelikar et al. (2301.01337). Low-latency PE enabled by a Gaussian likelihood approximation for RIFT
- Wong et al. (2302.05333). Fast GW PE without compromises
- Tiwari et al. (2303.01463). VARAHA: A Fast Non-Markovian sampler for estimating GW posteriors
- Karnesis et al. (2303.02164). Eryn : A multi-purpose sampler for Bayesian inference
- Fairhurst et al. (2304.03731). Fast inference of CBC properties using the information encoded in the GW signal
- ...
- 值得关注的 AI 技术：LLM, AIGC
- 软体开发: GWToolkit powered by Ray.
- 参数估计：(引力波数据分析领域的“圣杯”)
- Bayes inference
- "Curse of Dimensionality"
- Test of GR
Neural Posterior Estimation with guaranteed exact coverage: the ringdown of GW150914
Normalizing Flows as an Avenue to Studying Overlapping Gravitational Wave Signals
-
2002.07656: 5D toy model [1] (PRD)
-
2008.03312: 15D binary black hole inference [1] (MLST)
-
2106.12594: Amortized inference and group-equivariant neural PE [2] (PRL)
-
2111.13139: Group-equivariant neural PE [2]
-
2210.05686: Importance sampling [2]
-
2211.08801: Noise forecasting [2]
-
https://github.com/dingo-gw/dingo (2023.03)
©Floor Broekgaarden (repo)
Gravitational Wave Astronomy
- Looking towards the future of gravitational wave astronomy: O4 and beyond
LIGO-G2300554
AI for Gravitational Wave
-
2016年，AlphaGo 第一版发表在了 Nature 杂志上
-
2021年，AI预测蛋白质结构登上 Science、Nature 年度技术突破，潜力无穷
-
2022年，DeepMind团队通过游戏训练AI发现矩阵乘法算法问题
-
《达摩院2022十大科技趋势》将 AI for Science 列为重要趋势
-
“人工智能成为科学家的新生产工具，催生科研新范式”
-
-
AI for Science：为科学带来了模型与数据双驱动的新的研究范式
-
AI + 数学、AI + 化学、AI + 医药、AI + 物理、AI + 天文 ...
-
AlphaGo 围棋机器人
AlphaTensor 发现矩阵算法
AlphaFold 蛋白质结构预测
Gravitational Wave Observational Data Denoising
-
数据质量的提升是一个非常复杂的问题，超过 20 万个传感器通道的数据会决定引力波科学数据通道的质量
-
降低引力波数据中非高斯的短时脉冲波干扰 (Glitch)，会有助于减少引力波信号误报率
-
引力波探测数据中去除 Glitch，是一个多分类问题
- 传统机器学习算法 Powell J, et al. CQG, 2015
- 深度学习算法 Zevin, M, et al. CQG, 2017; Razzano M, Cuoco E. CQG, 2018; Ormiston R, et al. PRR, 2020
- 与其消除数据的非高斯性，何不直接把信号重构出来？这有助于发现理论预言之外的引力波信号！
Extremely Loud Helix Koi Fish
Glitch cases
non-Gaussianess
Ormiston R, et al. PRR, 2020
AI For Scientist
AI For Science 创客松
——人工智能驱动的科学研究
Future of AI in
Gravitational wave astronomy
- Potential advancements in AI technology and applications.
- Impact on the field of astronomy and astrophysics.
海量非高斯非稳态
亚原子核级别
超低信噪比
多模态
高维数据结构
引力波观测数据
模式识别
智能降噪
引力波信号识别
智能贝叶斯
参数估计
引力波统计推断
引力理论
量子场论
基本理论检验与颠覆
Interpretability of AI in science
- Exploring the importance of understanding how AI models make predictions in scientific research.
Bayes
AI
- 生成式模型是关键
- 不确定性量化?
- 可控可信的模型?
from 李宏毅
Interpretability of AI in science
- Exploring the importance of understanding how AI models make predictions in scientific research.
Bayes
AI
from 李宏毅
- 生成式模型是关键
- 不确定性量化?
- 可控可信的模型?
Collaboration between AI and human experts
How AI can assist human experts in analyzing and interpreting gravitational wave data (natural science).
- 数据/软件的开源现状
- 学术成果的评判标准
- 专业知识的融合促进
- ...
机器学习在引力波数据分析中的应用——参数估计及数据降噪
By He Wang
机器学习在引力波数据分析中的应用——参数估计及数据降噪
引力波数据分析系列报告 | 时间:2023年6月11日(周日)下午14:00 | 中国科学院力学研究所怀柔园区1号楼430会议室
