WaveFormer: transformer-based denoising method for gravitational-wave data

Various types of Glitch

• The improvement of data quality is a very complex issue, with data from over 20,000 sensor channels determining the quality of the gravitational wave science data channel.

• Reducing non-Gaussian short-duration pulse interference (Glitches) in gravitational wave data will help reduce the false alarm rate of gravitational wave signals.

• Removing Glitches from gravitational wave detection data is a multi-classification problem.

  • Traditional machine learning algorithms​ ​Powell J, et al. CQG, 2015
  • Deep learning algorithms Zevin, M, et al. C

Ormiston R, et al. PRR, 2020

• DeepClean: One-dimensional Convolutional Neural Network which takes a specified set of witness channels and subsequently outputs the predicted noise in strain.

• Extraction and denoising GW signals using deep learning:
  • Both Wei et al. [PLB 2020] and Chatterjee et al. [PRD 2021] have shown that considering phase overlaps yields excellent results.
• Detecting and denoising GW signals using deep learning:
  • Both Bacon et al. [2205.13513] and Murali et al. [PRD 2023] could recover the phase of original GW signal with certain cycles but failed to recover the complete evaluation in amplitude scale.

• The WaveFormer, a billion-scale transformer-based model, excels in suppressing realistic noise and recovering injections or GW events, thereby significantly improving data quality.
• In its application, it treats each overlapping time-domain data subsequence as an individual token, akin to tokenization in natural language processing (NLP).

Given $d=h+n$, we can normalize $d$ as follows:

• Implementations:
  • PSD sampling from real noise.
  • input size: 8.0625 sec
  • fs = ​2048Hz
  • Band-pass: 20~2048Hz
  • Masked loss

• Noise level percentile amplitude is significantly reduced, by approximately two orders.
• Further ASD analysis shows that WaveFormer effectively eliminates both narrowband and broadband spectral information, substantially lowering frequency contributions.
• Using the Gravity Spy database for glitches with SNR > 10 and confidence > 0.95, results show significant suppression of glitches in real advanced LIGO-Virgo noise.

• Overlap and matched-filtering signal-to-noise are calculated to represent phase and amplitude recovery performance.
• Among the intermediate frequency range (20–200 Hz) that covers rich BBH signal information, the ASD distribution of denoised waveform is evidently consistent with that of target signal.

(Upper panels: Signal amplitude recovery performance

(Bottom panels: Signal phase recovery performance​)

• These results show that our denoising algorithm outperformed others by capturing the characteristic chirping morphology of BBH evolution, and can denoise signals in realistic detection scenarios without affecting signal characteristics such as phase and amplitude.
• For the event GW191204_171526, classified as either an NSBH or a low-mass BBH candidate in GWTC-3, the overlap with IMRPhenomXPHM achieved 0.93 and 0.95 on H1 and L1, respectively, which are marked improvements over those achieved by BayesWave and cWB (with overlaps between 0.82–0.86).

GW191204_171526

• Firstly, we obtain the denoised output by utilizing Waveformer. Then, triggers are defined and identified by three steps including,

  • Find Peaks. Locate triggers on a single detector by finding its maximum all local-maximum (0.2s away from neighboring maximum/local-maximum).

• An search algorithm for GW require that: [cite: 2010.07244]

  1. the same signal is seen in the detectors; (the same signal is seen by time-shifting in single detector)

  2. the same waveform must be present both detectors;

  3. and the signal’s time of arrival must be consistent with the GW travel time between the observatories.

• Firstly, we obtain the denoised output by utilizing Waveformer. Then, triggers are defined and identified by three steps including,

  • Find Peaks. Locate triggers on a single detector by finding its maximum all local-maximum (0.2s away from neighboring maximum/local-maximum).

  • By constraining triggers that exist on both two detectors, we get VALID triggers. (consist 3~4 segments)

• Firstly, we obtain the denoised output by utilizing Waveformer. Then, triggers are defined and identified by three steps including,

  • Find Peaks. Locate triggers on a single detector by finding its maximum all local-maximum (0.2s away from neighboring maximum/local-maximum).

  • By constraining triggers that exist on both two detectors, we get VALID triggers. (consist 3~4 segments)

  • Calculate the correlation of the to-be-evaluated trigger across channels or within a single channel, between its noisy and corresponding denoised segments, as well as between denoised segments themselves.

• Firstly, we obtain the denoised output by utilizing Waveformer. Then, triggers are defined and identified by three steps including,

  • Find Peaks. Locate triggers on a single detector by finding its maximum all local-maximum (0.2s away from neighboring maximum/local-maximum).

  • By constraining triggers that exist on both two detectors, we get VALID triggers. (consist 3~4 segments)

  • Calculate the correlation of the to-be-evaluated trigger across channels or within a single channel, between its noisy and corresponding denoised segments, as well as between denoised segments themselves.

• Through time shift, background analysis is done on other triggers around the target trigger. (time-shift interval 0.1 sec)
• Finally, by counting the number of false alarm trigger pairs, we obtain the IFAR value of the target trigger, which represents the reported or candidate BBH event in this experiment.

(PyCBC) Davies​, et al. PRD 2020

Ours

• Assessed denoising workflow performance by comparing with GWTC-1, GWTC-2, GWTC2.1, and GWTC-3 catalogs and associated data releases.

• Noted significant divergence in IFAR distribution between our results and those from GWTC and OGC catalogs.

• Achieved significant IFAR improvement across all 75 reported BBH events, indicating effective suppression of loud terrestrial noise.

  • Example: For low SNR (\(10.8_{-0.4}^{+0.3}\)) event GW200208_130117, obtained an IFAR of 8916 years, surpassing maximum IFAR of <4000 years in other catalogs.

• Variability in IFAR improvement linked to the original data's noise nature, including its non-Gaussian, non-stationary characteristics, and different signal recognition strategies by pipelines.

• IFAR performance significantly depends on the reduction of non-Gaussian noise near each event.

  • Events with substantial IFAR improvement had misleading non-Gaussian noise effectively eliminated.

  • Events where IFAR underperforms retained non-Gaussian characteristics, possibly due to WaveFormer's inherent systematic errors.

• Developed an AI-based workflow with WaveFormer, combining convolutional neural network and transformer for effective GW noise suppression and hierarchical feature extraction across a wide frequency range.
• Achieved significant noise suppression and signal recovery performance improvements, including state-of-the-art results on real observational data and BBH events, leading to dramatic data quality improvement and significant IFAR enhancement on 75 reported BBH events.

• Challenges in Model Interpretability
  • The black-box nature of AI models complicates interpretability, challenging the comparison of AI-generated detection statistics with traditional matched filtering chi-square distributions.
  • Convincing the scientific community of the pipeline's validity and the statistical significance of new discoveries remains difficult despite the model's ability to identify potential gravitational wave signals.
     
• Future Directions
  • Construct a comprehensive GW signal search pipeline for BBH/BNS/NSBH events.
  • Explore the use of ensemble learning and other statistical methods to enhance the interpretability of the AI detection pipeline and address issues related to its validity.