Deep Generative AI
florpi
HerWILL 2/8/24
https://florpi.github.io/
IAIFI Fellow
Carol Cuesta-Lazaro
A 2D animation of a folk music band composed of anthropomorphic autumn leaves, each playing traditional bluegrass instruments, amidst a rustic forest setting dappled with the soft light of a harvest moon
BEFORE
Artificial General Intelligence?
AFTER
p(x)
p(y|x)
p(x|y) = \frac{p(y|x)p(x)}{p(y)}
p(x|y)
https://vitalflux.com/generative-vs-discriminative-models-examples/
Generation vs Discrimination
Maximize the likelihood of the training samples
\hat \theta = \argmax \left[ \log p_\theta (x_\mathrm{train}) \right]
x_1
x_2
Model
p_\theta(x)
Training Samples
x_\mathrm{train}
Generative Models 101
The curse of dimensionality
p_\theta(x) = \frac{e^{f_\theta(x)}}{Z_\theta}
Z_\theta = \int e^{f_\theta(x)}dx
x_1
x_2
Trained Model
p_\theta(x)
Generate Novel Samples
Evaluate probabilities
Anomaly detection, model comparison...
https://parti.research.google
A portrait photo of a kangaroo wearing an orange hoodie and blue sunglasses standing on the grass in front of the Sydney Opera House holding a sign on the chest that says Welcome Friends!
Scaling laws and emergent abilities
"Scaling Laws for Neural Language Models" Kaplan et al
Explicit Density
Implicit Density
Tractable Density
Approximate Density
Normalising flows
Variational Autoencoders
Diffusion models
Generative Adversarial Networks
The zoo of generative models
Base distribution
Target distribution
p_X(x) = p_Z(z) \left| \frac{dz}{dx}\right|
Z \sim \mathcal{N} (0,1) \rightarrow g(z) \rightarrow X
Invertible transformation
z \sim p_Z(z)
p_Z(z)
Normalizing flows
x = f(z), \, z = f^{-1}(x)
p(\mathbf{x}) = p_z(f^{-1}(\mathbf{x})) \left\vert \det J(f^{-1}) \right\vert
(Image Credit: Phillip Lippe)
p(x) = \int dz \, p(x|z)
z: Latent variables
Invertible functions aren't that common!
Splines
arXiv:1911.01429
Simulation-based inference
But ODE solutions are always invertible!
z = x + \int_0^1 \phi (x(t)) dt
x = z + \int_1^0 \phi (x(t)) dt
\log p_X(x) = \log p_Z(z) + \int_0^1 \mathrm{Tr} J_\phi (x(t)) dt
Issues NFs: Lack of flexibility
- Invertible functions
- Tractable Jacobians
Chen et al. (2018), Grathwohl et al. (2018)
z = x + \int_0^1 \phi (x(t)) dt
\log p_X(x) = \log p_Z(z) + \int_0^1 \mathrm{Tr} J_\phi (x(t)) dt
Reverse diffusion: Denoise previous step
Forward diffusion: Add Gaussian noise (fixed)
q_\theta(z_0|z_1)
p(z_1|z_0)
p(z_0)
p(z_T)
p(z_2)
p(z_1)
p(z_2|z_1)
p(z_T|z_2)
q_\theta(z_1|z_2)
q_\theta(z_2|z_T)
Diffusion generative models
s_\theta(x,t) = \nabla_x \log p_t(x)
Score
"Equivariant Diffusion for Molecule Generation in 3D" Hongeboom et al
Speeding up drug discovery
A person half Yoda half Gandalf
Desired molecule properties
Students at MIT are
Large Language Models
Pre-trained on next word prediction
...
OVER-CAFFEINATED
NERDS
SMART
ATHLETIC
https://www.astralcodexten.com/p/janus-simulatorsHow do we encode "helpful" in the loss function?
BEFORE RLHF
AFTER RLHF
RLHF: Reinforcement Learning from Human Feedback
Step 1
Human teaches desired output
Explain RLHF
After training the model...
Step 2
Human scores outputs
+ teaches Reward model to score
it is the method by which ...
Explain means to tell someone...
Explain RLHF
Step 3
Tune the Language Model to produce high rewards!
"Sparks of Artificial General Intelligence: Early experiments with GPT-4" Bubeck et al
Produce Javascript code that creates a random graphical image that looks like a painting of Kandinsky
Draw a unicorn in TikZ
ImageBind: Multimodality
"ImageBind: One Embedding Space To Bind Them All" Girdhar et al
-
Books by Kevin P. Murphy
- Machine learning, a probabilistic perspective
- Probabilistic Machine Learning: advanced topics
- IAIFI Summer school
- Tutorials
- Blogposts
cuestalz@mit.edu
HerWill
By carol cuesta
