Deep Generative AI
florpi
HerWILL 2/8/24
![](https://github.githubassets.com/assets/GitHub-Mark-ea2971cee799.png)
![](https://static.vecteezy.com/system/resources/previews/014/391/995/original/transparent-website-icon-free-png.png)
https://florpi.github.io/
IAIFI Fellow
Carol Cuesta-Lazaro
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11039971/pasted-from-clipboard.png)
![](https://iaifi.org/images/iaifi-logo-black-noborder-hires.jpg)
![](https://images.openai.com/blob/b196df3a-6fea-4d86-87b2-f9bb50be64c7/leaf.png?trim=0,0,0,0&width=2600)
![](https://cdn-icons-png.flaticon.com/512/1793/1793332.png)
A 2D animation of a folk music band composed of anthropomorphic autumn leaves, each playing traditional bluegrass instruments, amidst a rustic forest setting dappled with the soft light of a harvest moon
![](https://production-media.paperswithcode.com/tasks/face-generation_o4gNdRv_IvWakKY_eLedwpb.jpg)
![](https://substackcdn.com/image/fetch/w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%253A%252F%252Fsubstack-post-media.s3.amazonaws.com%252Fpublic%252Fimages%252Fa9915da1-ec76-40e8-a258-9ce6bb879912_381x334.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109922/pasted-from-clipboard.png)
BEFORE
Artificial General Intelligence?
AFTER
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10775829/90724_23_openai-drops-massive-upgrade-for-chatgpt-making-it-shockingly-powerful.png)
![](https://c.files.bbci.co.uk/0D9B/production/_88738430_pic1go.jpg)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10436322/pasted-from-clipboard.png)
https://vitalflux.com/generative-vs-discriminative-models-examples/
Generation vs Discrimination
Maximize the likelihood of the training samples
Model
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11041175/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11041078/pasted-from-clipboard.png)
Training Samples
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109934/pasted-from-clipboard.png)
Generative Models 101
The curse of dimensionality
![](https://images.deepai.org/glossary-terms/f99300ef736b4ddba8c5506066903a3d/curse-dimensionality-2.png)
Trained Model
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11041175/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109944/pasted-from-clipboard.png)
Generate Novel Samples
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109944/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109944/pasted-from-clipboard.png)
Evaluate probabilities
Anomaly detection, model comparison...
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109974/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11109979/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10436056/pasted-from-clipboard.png)
https://parti.research.google
![](https://cdn-icons-png.flaticon.com/512/1793/1793332.png)
A portrait photo of a kangaroo wearing an orange hoodie and blue sunglasses standing on the grass in front of the Sydney Opera House holding a sign on the chest that says Welcome Friends!
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10436056/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10436056/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10436056/pasted-from-clipboard.png)
Scaling laws and emergent abilities
"Scaling Laws for Neural Language Models" Kaplan et al
![](https://i0.wp.com/bdtechtalks.com/wp-content/uploads/2022/08/LLMs-emergent-abilities.png?ssl=1)
![](https://eliaszwang.com/paper-reviews/scaling-laws-neural-lm/featured.png)
![](https://eliaszwang.com/paper-reviews/scaling-laws-neural-lm/featured.png)
Explicit Density
Implicit Density
Tractable Density
Approximate Density
Normalising flows
Variational Autoencoders
Diffusion models
Generative Adversarial Networks
The zoo of generative models
Base distribution
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10435133/pasted-from-clipboard.png)
Target distribution
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10435133/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10435133/pasted-from-clipboard.png)
Invertible transformation
![](https://static.thenounproject.com/png/2424482-200.png)
Normalizing flows
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/9859134/pasted-from-clipboard.png)
(Image Credit: Phillip Lippe)
z: Latent variables
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10435161/pasted-from-clipboard.png)
Invertible functions aren't that common!
Splines
![](https://pbs.twimg.com/media/Fip9la7XgAA5OxG.jpg:large)
arXiv:1911.01429
Simulation-based inference
But ODE solutions are always invertible!
Issues NFs: Lack of flexibility
- Invertible functions
- Tractable Jacobians
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10435145/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10435146/pasted-from-clipboard.png)
Chen et al. (2018), Grathwohl et al. (2018)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10435184/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10464337/pasted-from-clipboard.png)
Reverse diffusion: Denoise previous step
Forward diffusion: Add Gaussian noise (fixed)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10464337/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10464337/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10464337/pasted-from-clipboard.png)
Diffusion generative models
![](https://blog.kakaocdn.net/dn/cH1S7j/btqYbH2lmp9/UsC9RqbM9PasAj82mAN6kK/img.png)
Score
"Equivariant Diffusion for Molecule Generation in 3D" Hongeboom et al
Speeding up drug discovery
![](https://d3i71xaburhd42.cloudfront.net/66a7a411696a7bb3e0ad844d5555e75a3aacb094/3-Figure2-1.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10464337/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10464337/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10464337/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10464337/pasted-from-clipboard.png)
A person half Yoda half Gandalf
Desired molecule properties
Students at MIT are
Large Language Models
Pre-trained on next word prediction
...
OVER-CAFFEINATED
NERDS
SMART
ATHLETIC
![](https://substackcdn.com/image/fetch/w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%253A%252F%252Fsubstack-post-media.s3.amazonaws.com%252Fpublic%252Fimages%252F521b9799-8eaf-4a35-87d0-97fb7875b40a_559x843.png)
https://www.astralcodexten.com/p/janus-simulators
How do we encode "helpful" in the loss function?
![](https://substackcdn.com/image/fetch/w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%253A%252F%252Fsubstack-post-media.s3.amazonaws.com%252Fpublic%252Fimages%252F521b9799-8eaf-4a35-87d0-97fb7875b40a_559x843.png)
BEFORE RLHF
![](https://substackcdn.com/image/fetch/w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%253A%252F%252Fsubstack-post-media.s3.amazonaws.com%252Fpublic%252Fimages%252Fa9915da1-ec76-40e8-a258-9ce6bb879912_381x334.png)
AFTER RLHF
RLHF: Reinforcement Learning from Human Feedback
Step 1
Human teaches desired output
Explain RLHF
![](https://cdn4.vectorstock.com/i/1000x1000/71/23/woman-head-icon-design-template-isolated-vector-31267123.jpg)
After training the model...
Step 2
Human scores outputs
+ teaches Reward model to score
it is the method by which ...
![](https://cdn-icons-png.flaticon.com/512/1793/1793332.png)
Explain means to tell someone...
![](https://cdn-icons-png.flaticon.com/512/1793/1793332.png)
Explain RLHF
Step 3
Tune the Language Model to produce high rewards!
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10436060/pasted-from-clipboard.png)
"Sparks of Artificial General Intelligence: Early experiments with GPT-4" Bubeck et al
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10775810/Screenshot_2023-09-23_at_6.18.02_PM.png)
![](https://cdn-icons-png.flaticon.com/512/1793/1793332.png)
Produce Javascript code that creates a random graphical image that looks like a painting of Kandinsky
![](https://cdn-icons-png.flaticon.com/512/1793/1793332.png)
Draw a unicorn in TikZ
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10775810/Screenshot_2023-09-23_at_6.18.02_PM.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10775810/Screenshot_2023-09-23_at_6.18.02_PM.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10436060/pasted-from-clipboard.png)
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/10436060/pasted-from-clipboard.png)
ImageBind: Multimodality
"ImageBind: One Embedding Space To Bind Them All" Girdhar et al
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/11110227/pasted-from-clipboard.png)
-
Books by Kevin P. Murphy
- Machine learning, a probabilistic perspective
- Probabilistic Machine Learning: advanced topics
- IAIFI Summer school
- Tutorials
- Blogposts
cuestalz@mit.edu
![](https://s3.amazonaws.com/media-p.slid.es/uploads/993552/images/9860795/pasted-from-clipboard.png)
HerWill
By carol cuesta
HerWill
- 96