Introduction into Generative Models

and Embeddings

1

Alexander Isenko

06.06.2017

Yvonne Hoffmann

Master Seminar for Artificial Intelligence

Classification of images

2

Alexander Isenko

06.06.2017

Yvonne Hoffmann

Challenge: Creating new images

3

Alexander Isenko

06.06.2017

Yvonne Hoffmann

Neural

Net

x
xx

Problem: How to train nets?

4

Alexander Isenko

06.06.2017

Yvonne Hoffmann

Task

5

Alexander Isenko

06.06.2017

Yvonne Hoffmann

+

GAN

6

Alexander Isenko

06.06.2017

Yvonne Hoffmann

noise ~

yes / no

generator sample

discriminator

data sample

data sample?

generator

x
xx

How it works - "No Weenies allowed"

7

Alexander Isenko

06.06.2017

Yvonne Hoffmann

How it works - No Weenies allowed

8

Alexander Isenko

06.06.2017

Yvonne Hoffmann

How it works - No Weenies allowed

9

Alexander Isenko

06.06.2017

Yvonne Hoffmann

How it works - No Weenies allowed

10

Alexander Isenko

06.06.2017

Yvonne Hoffmann

How it works - No Weenies allowed

11

Alexander Isenko

06.06.2017

Yvonne Hoffmann

How it works - No Weenies allowed

12

Alexander Isenko

06.06.2017

Yvonne Hoffmann

How it works - No Weenies allowed

13

Alexander Isenko

06.06.2017

Yvonne Hoffmann

How it works - No Weenies allowed

14

Alexander Isenko

06.06.2017

Yvonne Hoffmann

How it works - No Weenies allowed

15

Alexander Isenko

06.06.2017

Yvonne Hoffmann

How it works - No Weenies allowed

16

Alexander Isenko

06.06.2017

Yvonne Hoffmann

How it works - No Weenies allowed

17

Alexander Isenko

06.06.2017

Yvonne Hoffmann

18

Alexander Isenko

06.06.2017

Yvonne Hoffmann

GAN

noise

yes / no

generator sample

discriminator

data sample

data sample?

generator

19

Alexander Isenko

06.06.2017

Yvonne Hoffmann

GAN - Learning process

noise

yes / no

discriminator

data sample?

generator

20

Alexander Isenko

06.06.2017

Yvonne Hoffmann

GAN - Learning process

noise

yes / no

discriminator

data sample?

generator

21

Alexander Isenko

06.06.2017

Yvonne Hoffmann

GAN - Learning process

noise

yes / no

discriminator

data sample?

generator

Learning

process

22

Alexander Isenko

06.06.2017

Yvonne Hoffmann

Learning Progress

Reward

23

Alexander Isenko

06.06.2017

Yvonne Hoffmann

G \rightarrow +R
G→+RG \rightarrow +R
D \rightarrow -R
D→−RD \rightarrow -R
\}
}\}
R (\theta_{G}, \theta_{D})
R(θG,θD)R (\theta_{G}, \theta_{D})
G := \text{Generator}
G:=GeneratorG := \text{Generator}
D := \text{Discriminator}
D:=DiscriminatorD := \text{Discriminator}
\theta_{X} := \text{parametrization of } X
θX:=parametrization of X\theta_{X} := \text{parametrization of } X
R^{X} := \text{Reward of } X
RX:=Reward of XR^{X} := \text{Reward of } X
\theta_{G} \rightarrow \text{data} \rightarrow \theta_{D} \rightarrow R
θG→data→θD→R\theta_{G} \rightarrow \text{data} \rightarrow \theta_{D} \rightarrow R
\frac{\partial R}{\partial \theta_{G}}
∂R∂θG\frac{\partial R}{\partial \theta_{G}}
\frac{\partial R}{\partial \theta_{D}}
∂R∂θD\frac{\partial R}{\partial \theta_{D}}

G trains

D trains

GAN Example

24

Alexander Isenko

06.06.2017

Yvonne Hoffmann

x = 0, \lim_{x \rightarrow 1} x
x=0,limx→1xx = 0, \lim_{x \rightarrow 1} x

I. Problem - Trouble with counting

25

Alexander Isenko

06.06.2017

Yvonne Hoffmann

II. Problem - Trouble with perspective

26

Alexander Isenko

06.06.2017

Yvonne Hoffmann

III. Problem - Misunderstanding global structure

27

Alexander Isenko

06.06.2017

Yvonne Hoffmann

Outlook - Variational Autoencoder

28

Alexander Isenko

06.06.2017

Yvonne Hoffmann

Encoder

Decoder

1
11
0
00
0
00
\vdots
⋮\vdots
z
zz
\mu
μ\mu
\sigma
σ\sigma
z
zz

Outlook - Adaptive Image Compression

29

Alexander Isenko

06.06.2017

Yvonne Hoffmann

Applications

30

Alexander Isenko

06.06.2017

Yvonne Hoffmann

References

31

Alexander Isenko

06.06.2017

Yvonne Hoffmann

• https://blog.openai.com/generative-models/
• http://gabgoh.github.io/ThoughtVectors/
• https://ishmaelbelghazi.github.io/ALI/
• http://kvfrans.com/variational-autoencoders-explained/
• https://arxiv.org/pdf/1701.00160.pdf
• https://research.googleblog.com/2016/11/zero-shot-translation-with-googles.html

• https://medium.com/@awjuliani/generative-adversarial-networks-explained-with-a-classic-spongebob-squarepants-episode-54deab2fce39

• https://arxiv.org/pdf/1705.05823.pdf

33

Alexander Isenko

06.06.2017

Yvonne Hoffmann

Recap - GAN

noise

yes / no

generator sample

discriminator

data sample

data sample?

generator

Recap - VAE

34

Alexander Isenko

06.06.2017

Yvonne Hoffmann

Encoder

Decoder

1
11
0
00
0
00
\vdots
⋮\vdots
z
zz
\mu
μ\mu
\sigma
σ\sigma
z
zz

VAE with better colors

28

Alexander Isenko

06.06.2017

Yvonne Hoffmann

Encoder

Decoder

\mu
μ\mu
\sigma
σ\sigma
z
zz

Sampling

AE with better colors

28

Alexander Isenko

06.06.2017

Yvonne Hoffmann

Encoder

Decoder

z
zz
-0.1
−0.1-0.1
0
00
0.3
0.30.3
1.7
1.71.7
\vdots
⋮\vdots