From Zero to Generative
IAIFI Fellow, MIT
![](https://iaifi.org/images/iaifi-logo-black-noborder-hires.jpg)
Carolina Cuesta-Lazaro
Art: "The art of painting" by Johannes Vermeer
Learning Generative Modelling from scratch
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484126/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484005/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484007/pasted-from-clipboard.png)
Cuenca
Spain
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10099712/pasted-from-clipboard.png)
Heidelberg
Germany
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10099782/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10099734/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10099740/pasted-from-clipboard.png)
Tokyo
Japan
Durham
England
Boston
US
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484069/pasted-from-clipboard.png)
About Myself
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/9201524/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/9201514/pasted-from-clipboard.png)
Medical Imaging
Epidemiology: Agent Based simulations
OBSERVED
SIMULATED
Cosmology
Simulations
HPC
Science question
Statistics ML
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/9204341/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/6926154/pasted-from-clipboard.png)
Natural Language
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484027/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484070/pasted-from-clipboard.png)
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
BEFORE
Artificial General Intelligence?
AFTER
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10775829/90724_23_openai-drops-massive-upgrade-for-chatgpt-making-it-shockingly-powerful.png)
![](https://c.files.bbci.co.uk/0D9B/production/_88738430_pic1go.jpg)
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10436056/pasted-from-clipboard.png)
https://parti.research.google
![](https://cdn-icons-png.flaticon.com/512/1793/1793332.png)
A portrait photo of a kangaroo wearing an orange hoodie and blue sunglasses standing on the grass in front of the Sydney Opera House holding a sign on the chest that says Welcome Friends!
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10436056/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10436056/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10436056/pasted-from-clipboard.png)
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
Scaling laws and emergent abilities
"Scaling Laws for Neural Language Models" Kaplan et al
![](https://i0.wp.com/bdtechtalks.com/wp-content/uploads/2022/08/LLMs-emergent-abilities.png?ssl=1)
![](https://eliaszwang.com/paper-reviews/scaling-laws-neural-lm/featured.png)
![](https://eliaszwang.com/paper-reviews/scaling-laws-neural-lm/featured.png)
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10436060/pasted-from-clipboard.png)
"Sparks of Artificial General Intelligence: Early experiments with GPT-4" Bubeck et al
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10775810/Screenshot_2023-09-23_at_6.18.02_PM.png)
![](https://cdn-icons-png.flaticon.com/512/1793/1793332.png)
Produce Javascript code that creates a random graphical image that looks like a painting of Kandinsky
![](https://cdn-icons-png.flaticon.com/512/1793/1793332.png)
Draw a unicorn in TikZ
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10775810/Screenshot_2023-09-23_at_6.18.02_PM.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10775810/Screenshot_2023-09-23_at_6.18.02_PM.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10436060/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10436060/pasted-from-clipboard.png)
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484105/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484106/pasted-from-clipboard.png)
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11485373/pasted-from-clipboard.png)
Today's Plan
1. Recap of the Machine Learning building blocks
2. Learning to classify
BREAK
3. Tutorial: Build your first classifier
4. Introduction to Generative Models
5. Tutorial: Build your first generative model
(if time permits)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484591/pasted-from-clipboard.png)
BREAK
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
The building blocks: 1. Data
Cosmic Cartography
(Pointclouds)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11144498/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484022/pasted-from-clipboard.png)
MNIST
(Images)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484023/pasted-from-clipboard.png)
Wikipedia
(Text)
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
![](https://images.openai.com/blob/b196df3a-6fea-4d86-87b2-f9bb50be64c7/leaf.png?trim=0,0,0,0&width=2600)
1024x1024
The curse of dimensionality
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484062/pasted-from-clipboard.png)
Inductive biases!
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
The building blocks: 2. Architectures
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484026/pasted-from-clipboard.png)
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
Multilayer Perceptron
Image Credit: CS231n Convolutional Neural Networks for Visual Recognition
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484077/pasted-from-clipboard.png)
Pixel 1
Pixel 2
Pixel N
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484078/pasted-from-clipboard.png)
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
Convolutional Neural Networks
Inductive bias: Translation Invariance
Data Representation: Images
![](https://miro.medium.com/v2/resize:fit:1400/1*Fw-ehcNBR9byHtho-Rxbtw.gif)
Image Credit: Irhum Shakfat "Intuitively Understanding Convolutions for Deep Learning"
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
Inductive bias: Permutation Invariance
Data Representation: Sets, Pointclouds
Deep sets
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484084/pasted-from-clipboard.png)
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484084/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484084/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484084/pasted-from-clipboard.png)
Transformers might be the unifying architecture!
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484087/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484088/pasted-from-clipboard.png)
Text
Images
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
The building blocks: 3. Loss function
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484089/pasted-from-clipboard.png)
Image Credit: "Visualizing the loss landscape of neural networks" Hao Li et al
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
The building blocks: 4. The Optimizer
![](https://miro.medium.com/v2/resize:fit:1200/1*STiRp7PW5yIrvYZupZA6nw.gif)
Image Credit: "Complete guide to Adam optimization" Hao Li et al
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
Tutorial 1: Learning to classify
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484093/pasted-from-clipboard.png)
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
How do we output a probability?
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484077/pasted-from-clipboard.png)
Pixel 1
Pixel 2
Pixel N
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484078/pasted-from-clipboard.png)
p Class 1
p Class 2
p Class 10
Loss function: Cross entropy
How different are two probability distributions?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484096/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484096/pasted-from-clipboard.png)
Model Prediction
if True class is for i
otherwise
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
Truth: Class = 0
True class
Predicted probability
Loss function: Cross entropy
How different are two probability distributions?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484096/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484096/pasted-from-clipboard.png)
Model Prediction
Truth: Class = 0
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
Predicted probability
True class
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484064/pasted-from-clipboard.png)
Loss function: Cross entropy
How different are two probability distributions?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484096/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484096/pasted-from-clipboard.png)
Model Prediction
Truth: Class = 0
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484078/pasted-from-clipboard.png)
import flax.linen as nn
class MLP(nn.Module):
@nn.compact
def __call__(self, x):
# Linear
x = nn.Dense(features=64)(x)
# Non-linearity
x = nn.silu(x)
# Linear
x = nn.Dense(features=64)(x)
# Non-linearity
x = nn.silu(x)
# Linear
x = nn.Dense(features=2)(x)
return x
model = MLP()
Jax Models
import jax.numpy as jnp
example_input = jnp.ones((1,4))
params = model.init(jax.random.PRNGKey(0), example_input)
y = model.apply(params, example_input)
Architecture
Parameters
Call
![](https://images.openai.com/blob/b196df3a-6fea-4d86-87b2-f9bb50be64c7/leaf.png?trim=0,0,0,0&width=2600)
![](https://cdn-icons-png.flaticon.com/512/1793/1793332.png)
A 2D animation of a folk music band composed of anthropomorphic autumn leaves, each playing traditional bluegrass instruments, amidst a rustic forest setting dappled with the soft light of a harvest moon
Image credit: DALL·E 3
1024x1024
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10436322/pasted-from-clipboard.png)
Generation vs Discrimination
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
![](https://creazilla-store.fra1.digitaloceanspaces.com/cliparts/78147/genie-lamp-clipart-xl.png)
Data
A PDF that we can optimize
Maximize the likelihood of the data
Generative Models
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
Maximize the likelihood of the training samples
Model
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11041175/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11041078/pasted-from-clipboard.png)
Training Samples
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
Generative Models 101
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
Trained Model
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11041175/pasted-from-clipboard.png)
Evaluate probabilities
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109974/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109979/pasted-from-clipboard.png)
Low Probability
High Probability
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109944/pasted-from-clipboard.png)
Generate Novel Samples
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109944/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109944/pasted-from-clipboard.png)
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
Change of variables
sampled from a Gaussian distribution with mean 0 and variance 1
How is
distributed?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484064/pasted-from-clipboard.png)
Base distribution
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10435133/pasted-from-clipboard.png)
Target distribution
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10435133/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10435133/pasted-from-clipboard.png)
Invertible transformation
![](https://static.thenounproject.com/png/2424482-200.png)
Normalizing flows
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10435133/pasted-from-clipboard.png)
![](https://www.statology.org/wp-content/uploads/2021/07/plotUniform2.png)
Box-Muller transform
Normalizing flows in 1934
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/9859134/pasted-from-clipboard.png)
(Image Credit: Phillip Lippe)
z: Latent variables
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10435161/pasted-from-clipboard.png)
Invertible functions aren't that common!
Splines
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484064/pasted-from-clipboard.png)
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
But ODE solutions are always invertible!
Continuous time
Issues NFs: Lack of flexibility
- Invertible functions
- Tractable Jacobians
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
Flow ODE
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11266727/pasted-from-clipboard.png)
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10435145/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10435146/pasted-from-clipboard.png)
Chen et al. (2018), Grathwohl et al. (2018)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10435184/pasted-from-clipboard.png)
Generate
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
Evaluate Probability
Need to solve this expensive integral at each step during training!
Very slow
Can we avoid it?
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
Flow matching
Regress the velocity field directly!
But we need to know u. If we know u, then why learn another one?
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484461/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484462/pasted-from-clipboard.png)
Image Credit: "An Introduction to flow matchig" Tor Fjelde et al
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
Conditional Flow matching
Learn a conditional vector field (known at training time)
Approximate it with an unconditional one
The gradients of the losses are the same!
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484461/pasted-from-clipboard.png)
Tutorial 2
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484474/pasted-from-clipboard.png)
Gaussian
MNIST
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
Students at MIT are
Pre-trained on next word prediction
...
OVER-CAFFEINATED
NERDS
SMART
ATHLETIC
Large Language Models
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
![](https://substackcdn.com/image/fetch/w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%253A%252F%252Fsubstack-post-media.s3.amazonaws.com%252Fpublic%252Fimages%252F521b9799-8eaf-4a35-87d0-97fb7875b40a_559x843.png)
https://www.astralcodexten.com/p/janus-simulators
How do we encode "helpful" in the loss function?
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
Step 1
Human teaches desired output
Explain RLHF
![](https://cdn4.vectorstock.com/i/1000x1000/71/23/woman-head-icon-design-template-isolated-vector-31267123.jpg)
After training the model...
Step 2
Human scores outputs
+ teaches Reward model to score
it is the method by which ...
![](https://cdn-icons-png.flaticon.com/512/1793/1793332.png)
Explain means to tell someone...
![](https://cdn-icons-png.flaticon.com/512/1793/1793332.png)
Explain RLHF
Step 3
Tune the Language Model to produce high rewards!
RLHF: Reinforcement Learning from Human Feedback
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
![](https://substackcdn.com/image/fetch/w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%253A%252F%252Fsubstack-post-media.s3.amazonaws.com%252Fpublic%252Fimages%252F521b9799-8eaf-4a35-87d0-97fb7875b40a_559x843.png)
BEFORE RLHF
![](https://substackcdn.com/image/fetch/w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%253A%252F%252Fsubstack-post-media.s3.amazonaws.com%252Fpublic%252Fimages%252Fa9915da1-ec76-40e8-a258-9ce6bb879912_381x334.png)
AFTER RLHF
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
-
Books by Kevin P. Murphy
- Machine learning, a probabilistic perspective
- Probabilistic Machine Learning: advanced topics
- ML4Astro workshop https://ml4astro.github.io/icml2023/
- ProbAI summer school https://github.com/probabilisticai/probai-2023
- IAIFI Summer school
- Blogposts
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
References
cuestalz@mit.edu
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/9860795/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11484091/pasted-from-clipboard.png)
Carolina Cuesta-Lazaro IAIFI/MIT - From Zero to Generative
From zero to generative
By carol cuesta
From zero to generative
- 155