# Uncertainty in Neural Networks

Methods and Applications

### Berkeley Statistics and Machine Learning Forum

# What uncertainties are we talking about?

## A Motivating Example: Probabilistic Regression

From this excellent tutorial: https://medium.com/tensorflow/regression-with-probabilistic-layers-in-tensorflow-probability-e46ff5d37baf

- Linear regression

- Aleatoric Uncertainties

- Epistemic Uncertainties

- Epistemic+ Aleatoric Uncertainties

\hat{y} = a x

\hat{y} \sim \mathcal{N}(a x, \sigma^2)

\hat{y} = w x \quad w \sim p(w | \{x_i, y_i\})

\hat{y} \sim \mathcal{N}(w x, \sigma^2) \\ w, \sigma \sim p(w, \sigma | \{x_i, y_i\})

## Quiz: Which uncertainty dominates?

Predicting cluster masses from velocity dispersion

# Bayesian Neural Networks

### As tools to model Epistemic Uncertainties

# Bayes by Backprop

## Weight Uncertainty in Neural Networks

Blundel et al. 2015

## Beyond point estimates of weights

Maximum likelihood:

Bayesian Posterior by Variational Inference:

## Practical Algorithm

## Test on MNIST

## Regression Example

# MC Dropout

## Dropout as a Bayesian Approximation:

Representing Model Uncertainty in Deep Learning

Gal & Ghahramani, 2015

## What is dropout?

Hinton 2012, Srivastava 2014

## The idea

Let's express the predictive probability of the model:

Parameterize q(w) in the following way:

## Experiments

# Example of applications

### Image Segmentation: https://arxiv.org/abs/1703.04977

### Regression from images: https://arxiv.org/abs/1708.08843

Time Series Classification: https://arxiv.org/abs/1901.06384

## Some other methods

- Noise contrastive priors: https://arxiv.org/abs/1807.09289

#### Bayesian Neural Networks

By eiffl

# Bayesian Neural Networks

Session on modeling uncertainties with neural networks, for the Berkeley Statistics and Machine Learning Forum

- 885