Anomaly Detection meets
Deep Learning
Berkeley Statistics and Machine Learning Forum
Find the slides at slides.com/eiffl/ano
On the menu today
- Auto-Encoders
- GANs
- Likelihood-Based models
Disclaimer: This is NOT an exhaustive review, there are many hundreds of anomaly detection papers out there
Also, remember our session last year led by Ben Nachman
The AutoEncoder Approach
Variational Autoencoder based Anomaly Detection
An & Cho 2015
The simple autoencoder approach
The Variational AutoEncoder
Because a number of samples are drawn from the latent variable distribution, this allows the reconstruction probability to take into account the variability of the latent variable space, which is one of the main distinctions between the proposed method and the autoencoder based anomaly detection.
Difference from an autoencoder based anomaly detection
- Latent variables are stochastic variables, capturing not only means but also variance of the model
- Reconstructions are stochastic variables, capturing variance of the signal
- Reconstructions are probability measures, can incorporate heterogenous data
Experiments
Applications in Particle Physics
- Searching for New Physics with Deep Autoencoders, Farina et al. 2018
-
Variational Autoencoders for New Physics Mining
at the Large Hadron Collider, Cerri et al. 2019
Another interesting paper using autoencoders with a latent space model
Latent Space Autoregression for Novelty Detection, Abati et al. 2019
Are autoencoders so great?
A Lipschitz-constrained anomaly discriminator framework, Tong et al. 2019
Anomaly Detection,
the GAN way
Unsupervised Anomaly Detection with
Generative Adversarial Networks to Guide Marker Discovery
The idea
Building an anomaly score
- Mapping new images to latent space by minimizing the residual loss:
- And a discrimination loss:
- For each image x, z is optimized to minimize:
This loss after optimization will be the anomaly score
Experiments
Application on astronomical images
Work led by the amazing Kate Storey-Fisher (NYU) @kstoreyf
Real HSC images
GAN generated images
Low and high anomaly scores
Anomaly Detection with Likelihood models
DETECTING OUT-OF-DISTRIBUTION INPUTS TO DEEP
GENERATIVE MODELS USING TYPICALITY
I recommend checking out this great set of slides:
http://www.gatsby.ucl.ac.uk/~balaji/mluq-talk-balaji.pdf