He Wang PRO
Knowledge increases by sharing but not by saving.
12月08日 @中国科学院大学雁栖湖校区教学楼一101教室
AI在引力波探测中的机遇与挑战
王赫
hewang@ucas.ac.cn
中国科学院大学 · 国际理论物理中心（亚太地区）
中国科学院大学 · 引力波宇宙太极实验室（北京/杭州）
Content
- GW Astronomy
- AI for Science · GW Data Analysis
- GW search · Pipeline
- Parameter estimation · Scientific discovery
- Space-based GW Detection (Taiji Project)
\(\Rightarrow\) Welcome to join us!
-
In 1916, A. Einstein proposed the GR and predicted the existence of GW.
-
Gravitational waves (GW) are a strong field effect in the GR.
-
2015: the first experimental detection of GW from the merger of two black holes was achieved.
-
2017: the first multi-messenger detection of a BNS signal was achieved, marking the beginning of multi-messenger astronomy.
-
2017: the Nobel Prize in Physics was awarded for the detection of GW.
-
As of now: more than 90 gravitational wave events have been discovered.
-
O4, which began on May 24th 2023, is currently in progress.
-
Gravitational waves generated by binary black holes system
GW detector
LIGO-VIRGO-KAGRA network
2017 Nobel Prize in Physics
Gravitational Wave Astronomy
-
引力波探测打开了探索宇宙的新窗口
-
不同波源，频率跨越 20 个数量级，不同探测器
-
多信使天文学
Gravitational Wave Astronomy
-
Fundamental Physics
- Existence of gravitational waves
- To put constraints on the properties of gravitons
-
Astrophysics
- Refine our understanding of stellar evolution
- and the behavior of matter under extreme conditions.
-
Cosmology
- The measurement of the Hubble constant
- Dark energy
The first GW event of GW150914
- Detecting gravitational waves require a mix of FIVE key ingredients:
- good detector technology
- good waveform predictions
- good data analysis methodology and technology
- coincident observations in several independent detectors
- coincident observations in electromagnetic astronomy
—— Bernard F. Schutz
DOI: 10.1063/1.1629411
GWTC-3
Gravitational Wave Astronomy
Technical Challenges: Data Processing for GW
GW Data characteristics
-
Noise: non-Gaussian and non-stationary
-
Signal:
-
(Earth-based) A low signal-to-noise ratio (SNR) which is typically about 1/100 of the noise amplitude (-60 dB).
-
(Space-based) A superposition of all GW signals (e.g.: \(10^4\) of GBs, \(10\sim10^2\) of SMBHs, and \(10\sim10^3\) of EMRIs, etc.) received during the mission's observational run.
-
Matched filtering techniques (匹配滤波方法)
-
In Gaussian and stationary noise environments, the optimal linear algorithm for extracting weak signals
- Works by correlating a known signal model \(h(t)\) (template) with the data.
- Starting with data: \(d(t) = h(t) + n(t)\).
- Defining the matched-filtering SNR \(\rho(t)\):
\(\rho^2(t)\equiv\frac{1}{\langle h|h \rangle}|\langle d|h \rangle(t)|^2 \) , where \(\langle d|h \rangle (t) = 4\int^\infty_0\frac{\tilde{d}(f)\tilde{h}^*(f)}{S_n(f)}e^{2\pi ift}df \) ,
\(\langle h|h \rangle = 4\int^\infty_0\frac{\tilde{h}(f)\tilde{h}^*(f)}{S_n(f)}df \), \(S_n(f)\) is noise power spectral density (one-sided).
Text
LIGO-VIRGO-KAGRA
LISA / Taiji project
Text
-
Frequentist hypothesis testing and likelihood princple:
-
make some assumptions about signal and noise hypothesis
-
write down the likelihood function for a signal in noise
-
find the parameters that maximise it
-
define a corresponding detection statistic
\(\rightarrow\) recover the MF
-
-
Bayesian hypothesis testing:
- start from the same likelihood
- define a prior over signal parameters
- marginalise over them to arrive at a Bayes factor
- Often the dirty secret: just treat this as a Frequentist detection statistic
\(\rightarrow\) recover the MF (for certain prior choices)
Content
- GW Astronomy
- AI for Science · GW Data Analysis
- GW search · Pipeline
- Parameter estimation · Scientific discovery
- Space-based GW Detection (Taiji Project)
\(\Rightarrow\) Welcome to join us!
科学智能：AI for Science
-
2016年，AlphaGo 第一版发表在了 Nature 杂志上
-
2021年，AIphaFold 预测蛋白质结构登上 Science、Nature 年度技术突破
-
2022年，DeepMind团队通过游戏训练AI发现矩阵乘法算法问题
-
《达摩院2022十大科技趋势》将 AI for Science 列为重要趋势
-
“人工智能成为科学家的新生产工具，催生科研新范式”
-
-
2023年，DeepMind发布AI工具GNoME (Nature)，成功预测220万种晶体结构
-
2023年3月，为贯彻落实国家《新一代人工智能发展规划》，科技部会同自然科学基金委启动“人工智能驱动的科学研究”（AI for Science）专项部署工作，布局“人工智能驱动的科学研究”前沿科技研发体系。
AlphaGo 围棋机器人
AlphaTensor 发现矩阵算法
AlphaFold 蛋白质结构预测
验证数学猜想
Text
Pioneering works utilizing CNN
- The most common and direct approach, from Computer Vision (CV) to GW signal processing: pixel point \(\Rightarrow\) sampling point.
- Convolutional neural networks (CNN) can achieve comparable performance to Matched Filtering and surpass them in terms of execution speed (with GPU support) under Gaussian stationary noise.
AI for Science \(\rightarrow\) AI for GW Astronomy
- Artificial Intelligence (AI) has great potential to revolutionize gravitational wave astronomy by improving data analysis, modeling, and detector development.
- Representation and supervised learning crucially extract features from GW signals, autonomously identifying informative features and leveraging labeled data for accuracy.
Text
Exported: Oct, 2023 (in preparation)
PRL, 2018, 120(14): 141103.
PRD, 2018, 97(4): 044039.
引力波数据处理：人工智能技术应用
Content
- GW Astronomy
- AI for Science · GW Data Analysis
- GW search · Pipeline
- Parameter estimation · Scientific discovery
- Space-based GW Detection (Taiji Project)
\(\Rightarrow\) Welcome to join us!
GW search · Pipeline
Text
Beyond Speed: Generalization and Discovery in GW Detection
- Our primary goal is not speed but the model's ability to generalize and discover new GW signals, including those beyond the reach of matched filtering techniques and General Relativity (GR).
- Leveraging our experience in signal modeling (MFCNN) and noise modeling (WaveFormer), we are gradually building an offline pipeline capable of searching for signals in complete GW observation data and calculating FARs.
Real-time GW searches for GW150914
He Wang, et al. PRD 101, 10 (2020): 104003
He Wang, et al. MLST. 5, 1 (2024): 015046.
- Developed an AI-based workflow with WaveFormer, combining convolutional neural network and transformer for effective GW noise suppression and hierarchical feature extraction across a wide frequency range.
- Achieved significant noise suppression and signal recovery performance improvements, including state-of-the-art results on real observational data and BBH events, leading to dramatic data quality improvement and significant IFAR enhancement on 75 reported BBH events.
Text
-
Challenges in Model Interpretability
- The black-box nature of AI models complicates interpretability, challenging the comparison of AI-generated detection statistics with traditional matched filtering chi-square distributions.
- Convincing the scientific community of the pipeline's validity and the statistical significance of new discoveries remains difficult despite the model's ability to identify potential gravitational wave signals.
OURs
LVK. PRD (2016). arXiv:1602.03839
GW151226
GW151012
GW search · Pipeline
He Wang et al 2024 Mach. Learn.: Sci. Technol. 5 015046
Gebru et al. ICCV (2017)
Zhou et al. CVPR (2018)
Shen et al. CVPR (2018)
Image courtesy of Tesla (2020)
太学术了。。。下面简单说。。。
- 无处不在的卷积神经网络
Content
- GW Astronomy
- AI for Science · GW Data Analysis
- GW search · Pipeline
- Parameter estimation · Scientific discovery
- Space-based GW Detection (Taiji Project)
\(\Rightarrow\) Welcome to join us!
Parameter estimation · Scientific discovery
Credit: LIGO Magazine.
-
Traditional parameter estimation (PE) techniques rely on Bayesian analysis methods (posteriors + evidence)
- Computing the full 15-dimensional posterior distribution estimate is very time-consuming:
- Calculating likelihood function
- Template generation time-consuming
- Machine learning algorithms are expected to speed up!
AI for Gravitational Wave: Parameter Estimation
Bayesian statistics
Data quality improvement
Credit: Marco Cavaglià
LIGO-Virgo data processing
GW searches
Astrophsical interpretation of GW sources
AI for Gravitational Wave: Parameter Estimation
- A complete 15-dimensional posterior probability distribution, taking about 1 s (<< \(10^4\) s).
- Prior Sampling: 50,000 Posterior samples in approximately 8 Seconds.
- Capable of calculating evidence
- Processing time: (using 64 CPU cores)
- less than 1 hour with IMRPhenomXPHM,
- approximately 10 hours with SEOBNRv4PHM
PRL 127, 24 (2021) 241103.
PRL 130, 17 (2023) 171403.
Nature Physics 18, 1 (2022) 112–17
Big Data Mining and Analytics 5, 1 (2021) 53–63.
A diagram of prior sampling between feature space and physical parameter space
（Based on 1912.02762）
【【机器学习】白板推导系列(三十三) ～ 流模型(Flow based Model)】
Normalizing Flow Model (1/4)
The main idea of flow-based modeling is to express \(\mathbf{y}\in\mathbb{R}^D\) as a transformation \(T\) of a real vector \(\mathbf{z}\in\mathbb{R}^D\) sampled from \(p_{\mathrm{z}}(\mathbf{z})\):
\mathbf{y}=T(\mathbf{z}) \quad \text { where } \quad \mathbf{z} \sim p_{\mathrm{y}}(\mathbf{z})
Note: The invertible and differentiable transformation \(T\) and the base distribution \(p_{\mathrm{z}}(\mathbf{z})\) can have parameters \(\{\boldsymbol{\phi}, \boldsymbol{\psi}\}\) of their own, i.e. \( T_{\phi} \) and \(p_{\mathrm{z},\boldsymbol{\psi}}(\mathbf{z})\).
Change of Variables:
p_{\mathrm{y}}(\mathbf{y})=p_{\mathrm{z}}(\mathbf{z})\left|\operatorname{det} J_{T}(\mathbf{z})\right|^{-1} \quad \text { where } \quad \mathbf{u}=T^{-1}(\mathbf{x}) .
J_{T}(\mathbf{z})=\left[\begin{array}{ccc}
\frac{\partial T_{1}}{\partial \mathrm{z}_{1}} & \cdots & \frac{\partial T_{1}}{\partial \mathrm{z}_{D}} \\
\vdots & \ddots & \vdots \\
\frac{\partial T_{D}}{\partial \mathrm{z}_{1}} & \cdots & \frac{\partial T_{D}}{\partial \mathrm{z}_{D}}
\end{array}\right]
Equivalently,
The Jacobia \(J_{T}(\mathbf{u})\) is the \(D \times D\) matrix of all partial derivatives of \(T\) given by:
p_{\mathrm{y}}(\mathbf{y})=p_{\mathrm{z}}\left(T^{-1}(\mathbf{y})\right)\left|\operatorname{det} J_{T^{-1}}(\mathbf{y})\right|
p_{\mathrm{y}}(\mathbf{y})
p_{\mathrm{z}}(\mathbf{z})
\mathbf{z}
\mathbf{y}
T
T^{-1}
base density
target density
（Based on 1912.02762）
Normalizing Flow Model (2/4)
- Data: target data \(\mathbf{y}\in\mathbb{R}^{15}\) (with condition data \(\mathbf{x}\)).
- Task:
- Fitting a flow-based model \(p_{\mathrm{y}}(\mathbf{y} ; \boldsymbol{\theta})\) to a target distribution \(p_{\mathrm{y}}^{*}(\mathbf{y})\)
- by minimizing KL divergence with respect to the model’s parameters \(\boldsymbol{\theta}=\{\boldsymbol{\phi}, \boldsymbol{\psi}\}\),
- where \(\boldsymbol{\phi}\) are the parameters of \(T\) and \(\boldsymbol{\psi}\) are the parameters of \(p_{\mathrm{z}}(\mathbf{z})=\mathcal{N}(0,\mathbb{I})\).
- Loss function:
- Assuming we have a set of samples \(\left\{\mathbf{y}_{n}\right\}_{n=1}^{N}\sim p_{\mathrm{y}}^{*}(\mathbf{y})\),
Minimizing the above Monte Carlo approximation of the KL divergence is equivalent to fitting the flow-based model to the samples \(\left\{\mathbf{y}_{n}\right\}_{n=1}^{N}\) by maximum likelihood estimation.
\mathcal{L}(\boldsymbol{\theta}) \approx-\frac{1}{N} \sum_{n=1}^{N} \log p_{\mathrm{z}}\left(T^{-1}\left(\mathbf{y}_{n} ; \boldsymbol{\phi}\right) ; \boldsymbol{\psi}\right)+\log \left|\operatorname{det} J_{T^{-1}}\left(\mathbf{y}_{n} ; \boldsymbol{\phi}\right)\right|+\mathrm{const.}
p_{\mathrm{y}}(\mathbf{y})
p_{\mathrm{z}}(\mathbf{z})
\mathbf{z}
\mathbf{y}
T
T^{-1}
base density
target density
\begin{aligned}
\mathcal{L}(\boldsymbol{\theta}) &=D_{\mathrm{KL}}\left[p_{\mathrm{y}}^{*}(\mathbf{y}) \| p_{\mathrm{y}}(\mathbf{y} ; \boldsymbol{\theta})\right] \\
&=-\mathbb{E}_{p_{\mathbf{y}}^{*}(\mathbf{y})}\left[\log p_{\mathbf{y}}(\mathbf{y} ; \boldsymbol{\theta})\right]+\text { const. } \\
&=-\mathbb{E}_{p_{\mathbf{y}}^{*}(\mathbf{y})}\left[\log p_{\mathrm{z}}\left(T^{-1}(\mathbf{y} ; \boldsymbol{\phi}) ; \boldsymbol{\psi}\right)+\log \left|\operatorname{det} J_{T^{-1}}(\mathbf{y} ; \boldsymbol{\phi})\right|\right]+\mathrm{const} .
\end{aligned}
\mathbb{E}_{p_{\mathbf{y}}^{*}(\mathbf{y})}\left[\log p_{\mathbf{y}}^{*}(\mathbf{y} ; \boldsymbol{\theta})\right]
Rational Quadratic Neural Spline Flows
(RQ-NSF)
Train
\vec\theta = (m_1,m_2,d_L, ...) \in P_{prior}
\vec{x}=\vec{h}_{\vec{\theta}} + \vec{n}
nflow
\vec{z} \Rightarrow \mathbb{N}(0,\mathbb{I})
Normalizing Flow Model
归一化流模型示意图
Test
\vec\theta = (m_1,m_2,d_L, ...) \in P_{posterior}
\vec{x}=\vec{h}_{\vec{\theta}} + \vec{n}
nflow
\vec{z} \in \mathbb{N}(0,\mathbb{I})
Train
\vec\theta = (m_1,m_2,d_L, ...) \in P_{prior}
\vec{x}=\vec{h}_{\vec{\theta}} + \vec{n}
nflow
\vec{z} \Rightarrow \mathbb{N}(0,\mathbb{I})
Normalizing Flow Model
归一化流模型示意图
Test
\vec\theta = (m_1,m_2,d_L, ...) \in P_{posterior}
\vec{x}=\vec{h}_{\vec{\theta}} + \vec{n}
nflow
\vec{z} \in \mathbb{N}(0,\mathbb{I})
Train
\vec\theta = (m_1,m_2,d_L, ...) \in P_{prior}
\vec{x}=\vec{h}_{\vec{\theta}} + \vec{n}
nflow
\vec{z} \Rightarrow \mathbb{N}(0,\mathbb{I})
Normalizing Flow Model
-
Bayesian inference, the Holy Grail of gravitational-wave data analysis,
enables astrophysical interpretation and scientific discoveries.
Simulation-Based Inference (SBI)
- SBI \(\Rightarrow\) Fast and precise parameter estimation.
- SBI \(\Rightarrow\) TGR / Cosmology / PTA ...
Text
PRL 127, 24 (2021) 241103.
PRL 130, 17 (2023) 171403.
Real-time gravitational wave science with neural posterior estimation
Sampling with prior knowledge for high-dimensional gravitational wave data analysis
He Wang, et al. Big Data Min. Anal. (2021)
PRD 108, 4 (2023): 044029.
Neural Posterior Estimation with Guaranteed Exact Coverage: The Ringdown of GW150914
arXiv:2310.13405, LIGO-P2300306
Cosmological Inference using Gravitational Waves and Normalising Flows
Normalizing Flows as an Avenue to Studying Overlapping Gravitational Wave Signals
Parameter Estimation · Scientific Discovery
arXiv:2310.12209
Fast Parameter Inference on Pulsar Timing Arrays with Normalizing Flows
arXiv:2404.14286
Exact coverage first!
Paradigm
New
discovery
first!
还是太学术了。。。下面简单点。。。
AI for Sci 的内在驱动力几乎都与 “内插” 泛化特性相关
还是太学术了。。。下面简单点。。。
AI for Sci 的内在驱动力几乎都与 “内插” 泛化特性相关
还是太学术了。。。下面简单点。。。
AI for Sci 的内在驱动力几乎都与 “内插” 泛化特性相关
Test of General Relatively
- GW Astronomy
- AI for Science · GW Data Analysis
- GW search · Pipeline
- Parameter estimation · Scientific discovery
- Space-based GW Detection (Taiji Project)
\(\Rightarrow\) Welcome to join us!
Content
“国际理论物理中心(亚太地区)” 经联合国教科文组织第38届大会审议通过。由中国科学院、基金委和国际理论物理中心共同建设，是进行基础科学前沿与相关交叉科学领域高水平科研、教育和培训的非营利性组织，是联合国教科文组织基础科学方面的在国内的第一个二类中心。
ICTP-AP & Taiji Project
GW Data Analysis?
引力波数据分析？
程序猿？
数据分析师？
运维工程师？
数字信号处理
R.C. Cofer, Benjamin F. Harding, in Rapid System Prototyping with FPGAs, 2006
Dieter Rasch, Dieter Schott. Mathematical Statistics, (2018)
数理统计
-
理论基础：
-
引力波物理
-
数字信号处理
-
数理统计
-
-
编程基础：
-
-
硬件基础：
-
Miller, M.C., Yunes, N. The new frontier of gravitational waves. Nature 568, 469–476 (2019).
引力波物理与引力波天文学
引力波数据分析：知识框架
-
理论基础：
-
引力波物理 (pycbc, lalsuite, lisacode, bilby, ... )
-
数字信号处理 (scipy, stat, ...)
-
数理统计 (bilby, emcee, ptemcee, ptmcmc, …)
-
-
编程基础：
-
Python (numpy, pandas; matplotlib; ...),
-
AI (scikit-learn, XGBoost, PyTorch, TensorFlow, JAX, ...)
-
Linux (docker, github, bash, vim, emacs …)
-
-
硬件基础：
-
主板、内存，GPUs，显存 ...
- AI时代程序员都应该了解的GPU基础知识
-
引力波数据分析：知识框架
引力波数据分析：课程体系
引力波太极实验室:《引力波数据探索：编程与分析实战训练营》 2023.11 - 2024.1
- 第 3 部分：机器学习基础
- 机器学习算法之应用起步
- 机器学习算法之应用进阶
- 实战项目：基于 LIGO 的 Glitch 元数据完成多分类任务
- 实战项目：基于 LIGO 的 Glitch 时频图数据实现聚类分析
- 第 4 部分：深度学习基础
- 深度学习技术概述与神经网络基础
- 实战项目：训练一个3层神经网络（手撸版）
- 卷积神经网络与引力波信号探测
- 实战项目：使用 CNN 识别双黑洞系统引力波信号
- Kaggle数据科学竞赛 (黑客马拉松): Can you find the GW signals?
- 【公开课】AI发展全景与GPT前沿解析 (特邀嘉宾：高民权)
- 第 0 部分：通向自我实现之路
- 第 1 部分：编程开发环境与工作流
- 基础运维技术
- 容器化技术
- 实战项目：Python / Jupyter 开发环境搭建 + 远程连接 VS Code
- 实战项目：LALsuite / LISAcode 的源码编译 (optional)
- Git 分布式版本控制系统
- 【公开课】数据技术演进与现实应用 (特邀嘉宾：田昕峣)
- 第 2 部分：基于 Python 的数据分析基础
- 数据科学语言 Python 从入门到熟悉
- 数据分析实训之 Numpy / Pandas
- 实战项目：GW Event Catalog 的探索性数据分析
- 实战项目：股票数据分析案例 (optional)
- 基于 Python 的数据可视化理论与实践之 Matplotlib / Seaborn
- 实战项目：GWTC 论文中的 Figures
- 实战项目：针对 GW150914 信号处理与匹配滤波数据分析
- 【公开课】贝叶斯推断在引力波科学中的应用 (特邀嘉宾：赵俊杰)
引力波数据分析：课程体系
引力波太极实验室:《引力波数据探索：编程与分析实战训练营》 2023.11 - 2024.1
- 第 3 部分：机器学习基础
- 机器学习算法之应用起步
- 机器学习算法之应用进阶
- 实战项目：基于 LIGO 的 Glitch 元数据完成多分类任务
- 实战项目：基于 LIGO 的 Glitch 时频图数据实现聚类分析
- 第 4 部分：深度学习基础
- 深度学习技术概述与神经网络基础
- 实战项目：训练一个3层神经网络（手撸版）
- 卷积神经网络与引力波信号探测
- 实战项目：使用 CNN 识别双黑洞系统引力波信号
- Kaggle数据科学竞赛 (黑客马拉松): Can you find the GW signals?
- 【公开课】AI发展全景与GPT前沿解析 (特邀嘉宾：高民权)
- 第 0 部分：通向自我实现之路
- 第 1 部分：编程开发环境与工作流
- 基础运维技术
- 容器化技术
- 实战项目：Python / Jupyter 开发环境搭建 + 远程连接 VS Code
- 实战项目：LALsuite / LISAcode 的源码编译 (optional)
- Git 分布式版本控制系统
- 【公开课】数据技术演进与现实应用 (特邀嘉宾：田昕峣)
- 第 2 部分：基于 Python 的数据分析基础
- 数据科学语言 Python 从入门到熟悉
- 数据分析实训之 Numpy / Pandas
- 实战项目：GW Event Catalog 的探索性数据分析
- 实战项目：股票数据分析案例 (optional)
- 基于 Python 的数据可视化理论与实践之 Matplotlib / Seaborn
- 实战项目：GWTC 论文中的 Figures
- 实战项目：针对 GW150914 信号处理与匹配滤波数据分析
- 【公开课】贝叶斯推断在引力波科学中的应用 (特邀嘉宾：赵俊杰)
GW Data Analysis!
引力波数据分析！
引力波物理科学家
数据科学家
人工智能
算法工程师
for _ in range(num_of_audiences):
print('Thank you for your attention! 🙏')引力波数据分析！
引力波物理科学家
数据科学家
人工智能
算法工程师
GW data analysis?
AI在引力波探测中的机遇与挑战
By He Wang
AI在引力波探测中的机遇与挑战
2024.12.08 @中国科学院大学雁栖湖校区教学楼一101教室。 人工智能技术在引力波信号处理中的应用，涵盖去噪，多频段数据分析及参数反演的研究成果。
