Learning actions from data using invertible neural networks
Claudia Merger in joint work with Alexandre René, Kirsten Fischer and Peter Bouss
supervised by Moritz Helias and Carsten Honerkamp
in collaboration with David Dahmen and Christian Keup
Invertible neural networks on unsupervised learning tasks
data
INN
Invertible neural networks on unsupervised learning tasks
choose output
data distribution
data
INN
f_{\theta}(x)
Invertible neural networks on unsupervised learning tasks
data
INN
sample
f_{\theta}(x)
Gaussian
What has the INN learned?
Invertible neural networks on unsupervised learning tasks
Examples
- NICE (Dinh et. al., 2015 )
- RealNVP (Dinh et. al., 2017)
- GLOW (Kingma et. al. , 2018)
- https://openai.com/blog/glow/
data
INN
sample
INN must learn a approximate representation of data distribution.
How can we:
- extract the representation?
- use it to understand how the network learns?
Extract the representation
Extracting representation
- Simplify architecture
- Construct layer transform such that action stays polynomial
- Write down transform rules for coefficients
Application
Application
L=2
L=10
Application
Order in polynomial increases exponentially.
Coefficients become large.
Order in polynomial increases exponentially.
Coefficients become large.
Strategy:
- Store higher rank coefficients implicitly
- Truncate polynomial at specific order and include normalizing diagonal term
Order in polynomial increases exponentially.
Coefficients become large.
Strategy:
- Store higher rank coefficients implicitly
- Truncate polynomial at specific order and include normalizing diagonal term
Affine coupling must also be able to decrease the order.
Affine coupling must also be able to decrease the order.
2nd order action
4th order action
sign change
Summary and outlook
- Extract representation in the form of polynomials
- Test SimpleFLOW architecture on simple tasks
- Implement action mapping via Coefficient transforms
Further investigations:
- Explore lower order actions, using example systems from stat. physics
- Trunctaion scheme
- How do higher order terms contribute to lower orders? (Train with and without bias )
Learning actions from data using invertible neural networks
By merger
