Hyperparameter Optimization with Optuna

Srijith Rajamohan, Ph.D.

Generative Modeling

•  Discriminative Modeling has dominated most of Machine Learning or Deep Learning
• Discriminative Modeling: E.g. for a classifier, learn a manifold that separates the data space
• Generative Modeling: Learn the underlying distribution of data so you can generate new data
• Generative Modeling is harder 'but closer to true Artificial Intelligence than simply Discriminative Learning'
• Two of the popular techniques in Deep Learning
  • Variational Auto-encoders
  • Generative Adversarial Networks

What are Variational Autoencoders?

• Unsupervised Machine Learning algorithm
  • Supervised  and  semi-supervised versions exist as well
  • E.g. Conditional Variational Autoencoders are supervised
• Most people like to think of it in terms of a regular autoencoder - Encoder and a Decoder
• Mathematically, the motivation is to understand the underlying latent space of high-dimensional data
• Think of it as dimensionality reduction

Uses of Variational Autoencoders?

• Anomaly detection
• Dimensionality reduction
• Physics-Informed GANs
  • Physics-based laws encoded into the GANs
• Data augmentation
  • Medical data where you have limited data
  • Data where privacy concerns require synthetic data
  • Challenging to obtain labeled data

Variables

The Math?

The input data

The latent representation

Conditional probability distribution of the input

Probability of the latent space variable

Conditional probability of the latent space variable given the input