Experiments

Experiments

1 ImageNet Classification

Experiments

1 ImageNet Classification

ImageNet 2012

Experiments

1 ImageNet Classification

ImageNet 2012

  • Total number of class : 1000
  • Number of trained images : 1.28 million
  • Number of evaluated images : 50k
  • Number of test images : 100k

Experiments

1 ImageNet Classification

Plain Networks

Experiments

1 ImageNet Classification

Plain Networks

Experiments

1 ImageNet Classification

Plain Networks

Experiments

1 ImageNet Classification

Plain Networks

Experiments

1 ImageNet Classification

Batch Norm

Experiments

1 ImageNet Classification

Batch Norm

Experiments

1 ImageNet Classification

Experiments

1 ImageNet Classification

Experiments

1 ImageNet Classification

Experiments

1 ImageNet Classification

Experiments

1 ImageNet Classification

Model

SoftMax

Sort

Dog

Cat

Fish

Pig

Bird

.5000

.2500

.1250

.0625

.0625

Top-1

Top-5

Experiments

1 ImageNet Classification

Residual Networks

Experiments

1 ImageNet Classification

Residual Networks

Experiments

1 ImageNet Classification

Residual Networks

Experiments

1 ImageNet Classification

Residual Networks

Theoretical Results

Theoretical Results

2 Convergence of Algorithm 1

Experiments

Experiments

MNIST

Experiments

Toronto Face Database

Experiments

CIFAR-10

Experiments

log-likelihood

Parzen window-based

Experiments

log-likelihood

Parzen window-based

Experiments

log-likelihood

Parzen window-based

L = \frac{1}{N} \sum_{i}^{N} \log{P_G(x^i)}
\{x_1, x_2,\dots x_i\} \text{ is sample from } P_{data}
\{\hat{x}_1, \hat{x}_2,\dots \hat{x}_i\} \text{ is sample from } P_{G}

Experiments

log-likelihood

Parzen window-based

L = \frac{1}{N} \sum_{i}^{N} \log{P_G(x^i)}
P_G(\hat{x}_1)=0.2
P_G(\hat{x}_2)=0.4
P_G(\hat{x}_3)=0.3
P_G(\hat{x}_4)=0.1

Experiments

log-likelihood

Parzen window-based

L = \frac{1}{N} \sum_{i}^{N} \log{P_G(x^i)}
P_G(x_i)=[0, 1]
P_G(\hat{x}_1)=0.2
P_G(\hat{x}_2)=0.4
P_G(\hat{x}_3)=0.3
P_G(\hat{x}_4)=0.1

Experiments

log-likelihood

Parzen window-based

L = \frac{1}{N} \sum_{i}^{N} \log{P_G(x^i)}
P_G(x_i)=[0, 1]
P_G(\hat{x}_1)=0.2
P_G(\hat{x}_2)=0.4
P_G(\hat{x}_3)=0.3
P_G(\hat{x}_4)=0.1

Experiments

log-likelihood

Parzen window-based

Experiments

log-likelihood

Parzen window-based

K(x)=\frac{1}{\sqrt{2\pi}} \exp (-\frac{x^2}{2})

Experiments

log-likelihood

Parzen window-based

K(x)=\frac{1}{\sqrt{2\pi}} \exp (-\frac{x^2}{2})
f_n(x)=\frac{1}{n\sigma }\sum_{n}^{i=1}K(\frac{x-\hat{x}_i}{\sigma })

Experiments

log-likelihood

Parzen window-based

f_n(x)=\frac{1}{n\sigma }\sum_{n}^{i=1}K(\frac{x-\hat{x}_i}{\sigma })
K(x)=\frac{1}{\sqrt{2\pi}} \exp (-\frac{x^2}{2})
\sigma =1

Experiments

log-likelihood

Parzen window-based

f_n(x)=\frac{1}{n\sigma }\sum_{n}^{i=1}K(\frac{x-\hat{x}_i}{\sigma })
\sigma=1
K(x)=\frac{1}{\sqrt{2\pi}} \exp (-\frac{x^2}{2})
\sigma =10

cross_val on val_set

Experiments

Visualization of samples from the model

Experiments

Visualization of samples from the model

Experiments

Visualization of samples from the model

z_a
z_b

Experiments

Visualization of samples from the model

z_a
z_b

 linearly interpolating

Experiments

Visualization of samples from the model

z_a
z_b

Experiments

Experiments

1 SETUP

Experiments

1 SETUP

Pre-train Data Set

Image net 2012 1k calss 1.3M

Image net 21k 21k class 14M

JFT 18k classes 303 M

Experiments

1 SETUP

CIFAR-10/100

Oxford-IIIT Pets, Oxford-IIIT Flowers

ImageNet original val_labels & cleaned-up ReaL labels

Fine-tune Data Set

Experiments

1 SETUP

VTAB

Natural

Oxford-IIIT Pet

Small NORB

102  Flower

Caltech101

CIFAR-100

SUN397

SVHN

DTD

Specialized

Diabetic Rethinopathy

EuroSAT

PCam

Structured

Dmlab Frames

Resisc45

dSprites

CLEVR

KITTI

Experiments

1 SETUP

Data Pre-Processing

Experiments

1 SETUP

Data Pre-Processing

Experiments

1 SETUP

Data Pre-Processing

Experiments

1 SETUP

Data Pre-Processing

Experiments

1 SETUP

Data Pre-Processing

random chiose

Experiments

1 SETUP

Model Variants

BERT

ViT

(Patch_{size}^{2})^{-1} \propto Complexity \Leftarrow

Experiments

1 SETUP

Model Variants

GN

WS

Experiments

2  COMPARISON TO SOTA

Experiments

2  COMPARISON TO SOTA

Experiments

2  COMPARISON TO SOTA

2 \text{useTPUcores} \times traingTime_{days}

Experiments

2  COMPARISON TO SOTA

2 \text{useTPUcores} \times traingTime_{hour}

Experiments

2  COMPARISON TO SOTA

Previous

2 \text{useTPUcores} \times traingTime_{hour}

Experiments

2  COMPARISON TO SOTA

Previous

2 \text{useTPUcores} \times traingTime_{hour}

JFT

Experiments

2  COMPARISON TO SOTA

Previous

2 \text{useTPUcores} \times traingTime_{hour}

JFT

same dataset

Experiments

2  COMPARISON TO SOTA

Previous

2 \text{useTPUcores} \times traingTime_{hour}

JFT

Experiments

2  COMPARISON TO SOTA

Previous

2 \text{useTPUcores} \times traingTime_{hour}

Experiments

2  COMPARISON TO SOTA

Made with Slides.com