Learning Geometry
&
3D Symmetries
Mario Geiger
![](https://media.slid.es/uploads/948058/images/11394464/nvidia-logo-vert-rgb-blk-for-screen.png)
![](https://media.slid.es/uploads/948058/images/11395399/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11404606/pasted-from-clipboard.png)
This morning Emmanuel Noutahi presented us the different molecular representations
![](https://media.slid.es/uploads/948058/images/11404606/pasted-from-clipboard.png)
This morning Emmanuel Noutahi presented us the different molecular representations
![](https://media.slid.es/uploads/948058/images/9673714/in.gif)
![](https://media.slid.es/uploads/948058/images/9673715/out.gif)
Equivariant Neural Networks
input
output
Illustration of a neural network equivariant to rotations in 3D
Why we want Equivariance?
![](https://media.slid.es/uploads/948058/images/10461018/pasted-from-clipboard.png)
Why we want Equivariance?
![](https://media.slid.es/uploads/948058/images/10461018/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/10461021/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/10461021/pasted-from-clipboard.png)
array([[ 0.28261471, 0.56535286, 1.38205716],
[-0.59397486, 0.04869514, 0.76054154],
[-0.96598984, 0.06802525, 0.77853411],
[ 1.09923518, 1.20586634, 0.92461881],
[-0.35728519, 0.55409651, 0.3251024 ],
[-0.03344675, -0.48225385, -0.294099 ],
[-1.79362192, 1.26634314, -0.27039329]])
array([[ 0.55856046, 1.19226885, 0.60782165],
[ 0.59452502, 0.65252289, -0.4547258 ],
[ 0.72974089, 0.75212044, -0.7877266 ],
[ 1.00189192, 0.76199152, 1.55897624],
[ 1.10428859, 0.32784872, -0.08623213],
[ 0.27944288, -0.59813412, -0.24272213],
[ 2.40641258, 0.25619413, -1.19271872]])
Why we want Equivariance?
![](https://media.slid.es/uploads/948058/images/10461018/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/10461021/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/10461021/pasted-from-clipboard.png)
array([[ 0.28261471, 0.56535286, 1.38205716],
[-0.59397486, 0.04869514, 0.76054154],
[-0.96598984, 0.06802525, 0.77853411],
[ 1.09923518, 1.20586634, 0.92461881],
[-0.35728519, 0.55409651, 0.3251024 ],
[-0.03344675, -0.48225385, -0.294099 ],
[-1.79362192, 1.26634314, -0.27039329]])
array([[ 0.55856046, 1.19226885, 0.60782165],
[ 0.59452502, 0.65252289, -0.4547258 ],
[ 0.72974089, 0.75212044, -0.7877266 ],
[ 1.00189192, 0.76199152, 1.55897624],
[ 1.10428859, 0.32784872, -0.08623213],
[ 0.27944288, -0.59813412, -0.24272213],
[ 2.40641258, 0.25619413, -1.19271872]])
Data-Augmentation
- Inexact
- Expensive
Equivariance
- Exact
- Data-efficient
![](https://media.slid.es/uploads/948058/images/11400570/pasted-from-clipboard.png)
Fluid dynamics
![](https://media.slid.es/uploads/948058/images/11400573/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11400576/pasted-from-clipboard.png)
Mechanics
Electrodynamics
![](https://media.slid.es/uploads/948058/images/11400578/pasted-from-clipboard.png)
Standard Model
Rotation
Translation
Boosts
(Galilean or Lorentz)
Time
translation
![](https://media.slid.es/uploads/948058/images/11400594/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11402671/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403832/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403835/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403861/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403861/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403861/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403861/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403861/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403861/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403861/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403861/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403861/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403861/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403861/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403861/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403861/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403861/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403861/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403861/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/10447893/Screenshot_2023-05-04_at_11.57.34_AM.png)
Example of Equivariance
f: positions → forces
![](https://media.slid.es/uploads/948058/images/10447946/1mPK7Gk9masrTEm9wAJ_-GvJEycKNaJ6MDfA6b5tI9w.gif)
Example of Equivariance
f: positions → Hamiltonian
Where is equivariance used in AI?
Protein Folding
EquiFold Jae Hyeon Lee et al.
Protein Docking
DIFFDOCK Gabriele Corso et al.
Molecular Dynamics
Allegro A. Musaelian et al.
Open Catalyst Project.
Solid State Physics
Prediction of Phonon Density Z. Chen et al.
Molecular Electron Densities
Cracking the Quantum Scaling Limit with Machine Learned Electron Densities J. Rackers
Cosmology, Medical Images and others
![](https://media.slid.es/uploads/948058/images/10452721/Screenshot_2023-05-06_at_1.35.47_PM.png)
44M atoms while taking advantage of up to 5120 GPUs
Albert Musaelian
We observe: Equivariance ⇒ Data Efficient!
(Nequip: Simon Batzner et al. 2021)
![](https://media.slid.es/uploads/948058/images/9663712/pasted-from-clipboard.png)
max L of the messages
Error
Trainset size
Invariant features
(Nequip: Simon Batzner et al. 2021)
![](https://media.slid.es/uploads/948058/images/9663712/pasted-from-clipboard.png)
Equivariant features
Error
Trainset size
Invariant features
We observe: Equivariance ⇒ Data Efficient!
Group and Representations
Group and Representations
"what are the operations"
"how they compose"
Group and Representations
"what are the operations"
"how they compose"
"vector spaces on which the action of the group is defined"
Group and Representations
"what are the operations"
"how they compose"
rotations, parity
scalars, vectors, pseudovectors, ...
"vector spaces on which the action of the group is defined"
Group and Representations
Group G
- identity∈G
- associativity g(hk)=(gh)k
- inverse g−1∈G
"what are the operations" "how they compose"
"vector spaces on which the action of the group is defined"
Group and Representations
"what are the operations" "how they compose"
"vector spaces on which the action of the group is defined"
Group G
- identity∈G
- associativity g(hk)=(gh)k
- inverse g−1∈G
Group and Representations
"what are the operations" "how they compose"
"vector spaces on which the action of the group is defined"
Group G
- identity∈G
- associativity g(hk)=(gh)k
- inverse g−1∈G
Group and Representations
"what are the operations" "how they compose"
"vector spaces on which the action of the group is defined"
Group G
- identity∈G
- associativity g(hk)=(gh)k
- inverse g−1∈G
Representation D(g,x)
- g∈G, x∈V
- Linear D(g,x+y)=D(g,x)+D(g,y)
- Follow the structure of the group
D(gh,x)=D(g,D(h,x))
Group and Representations
"what are the operations" "how they compose"
"vector spaces on which the action of the group is defined"
Group G
- identity∈G
- associativity g(hk)=(gh)k
- inverse g−1∈G
Representation D(g,x)
- g∈G, x∈V
- Linear D(g,x+y)=D(g,x)+D(g,y)
- Follow the structure of the group
D(gh,x)=D(g,D(h,x))
Group and Representations
"what are the operations" "how they compose"
"vector spaces on which the action of the group is defined"
Group G
- identity∈G
- associativity g(hk)=(gh)k
- inverse g−1∈G
Representation D(g,x)
- g∈G, x∈V
- Linear D(g,x+y)=D(g,x)+D(g,y)
- Follow the structure of the group
D(gh,x)=D(g,D(h,x))
Group and Representations
"what are the operations" "how they compose"
"vector spaces on which the action of the group is defined"
Group G
- identity∈G
- associativity g(hk)=(gh)k
- inverse g−1∈G
Representation D(g,x)
- g∈G, x∈V
- Linear D(g,x+y)=D(g,x)+D(g,y)
- Follow the structure of the group
D(gh,x)=D(g,D(h,x))
Group and Representations
"what are the operations" "how they compose"
"vector spaces on which the action of the group is defined"
Group G
- identity∈G
- associativity g(hk)=(gh)k
- inverse g−1∈G
Representation D(g,x)
- g∈G, x∈V
- Linear D(g,x+y)=D(g,x)+D(g,y)
- Follow the structure of the group
D(gh,x)=D(g,D(h,x))
Equivalent notation D(g)x
- D(g):V→V
- D(g)∈Rd×d
- D(gh)=D(g)D(h)
Group and Representations
"what are the operations" "how they compose"
"vector spaces on which the action of the group is defined"
Group G
- identity∈G
- associativity g(hk)=(gh)k
- inverse g−1∈G
Representation D(g,x)
- g∈G, x∈V
- Linear D(g,x+y)=D(g,x)+D(g,y)
- Follow the structure of the group
D(gh,x)=D(g,D(h,x))
Group and Representations
"what are the operations" "how they compose"
"vector spaces on which the action of the group is defined"
Equivalent notation D(g)x
- D(g):V→V
- D(g)∈Rd×d
- D(gh)=D(g)D(h)
Group G
- identity∈G
- associativity g(hk)=(gh)k
- inverse g−1∈G
Representation D(g,x)
- g∈G, x∈V
- Linear D(g,x+y)=D(g,x)+D(g,y)
- Follow the structure of the group
D(gh,x)=D(g,D(h,x))
Group and Representations
"what are the operations" "how they compose"
"vector spaces on which the action of the group is defined"
Equivalent notation D(g)x
- D(g):V→V
- D(g)∈Rd×d
- D(gh)=D(g)D(h)
Group G
- identity∈G
- associativity g(hk)=(gh)k
- inverse g−1∈G
Representation D(g,x)
- g∈G, x∈V
- Linear D(g,x+y)=D(g,x)+D(g,y)
- Follow the structure of the group
D(gh,x)=D(g,D(h,x))
Group and Representations
"what are the operations" "how they compose"
"vector spaces on which the action of the group is defined"
Equivalent notation D(g)x
- D(g):V→V
- D(g)∈Rd×d
- D(gh)=D(g)D(h)
Group G
- identity∈G
- associativity g(hk)=(gh)k
- inverse g−1∈G
Examples of representations
a1a2a3a4a5a6a7a8a9∈R9
Examples of representations
Representations are like data types
It tells you how to interpret the data with respect to the group action
a1a2a3a4a5a6a7a8a9∈R9
Examples of representations
3 scalars (3x0e)
a1a2a3a4a5a6a7a8a9∈R9
Knowing that a1,a2,a3 are scalars tells you that they are not affected by a rotation of your system
Representations are like data types
It tells you how to interpret the data with respect to the group action
Examples of representations
3 scalars (3x0e)
a vector (1o)
a1a2a3a4a5a6a7a8a9∈R9
Representations are like data types
It tells you how to interpret the data with respect to the group action
Examples of representations
3 scalars (3x0e)
a vector (1o)
a1a2a3a4a5a6a7a8a9∈R9
If the system is rotated, the 3 components of the vector change together!
Representations are like data types
It tells you how to interpret the data with respect to the group action
Examples of representations
3 scalars (3x0e)
a vector (1o)
a1a2a3a4a5a6a7a8a9∈R9
a vector (1o)
The two vectors transforms independently
Representations are like data types
It tells you how to interpret the data with respect to the group action
Examples of representations
3 scalars (3x0e)
a vector (1o)
a1a2a3a4a5a6a7a8a9∈R9
a vector (1o)
system rotated by g
a′1a′2a′3a′4a′5a′6a′7a′8a′9=D(g)a1a2a3a4a5a6a7a8a9
Representations are like data types
It tells you how to interpret the data with respect to the group action
Examples of representations
3 scalars
a vector
a1a2a3a4a5a6a7a8a9∈R9
a vector
system rotated by g
a1a2a3a4a5a6a7a8a9
Representations are like data types
It tells you how to interpret the data with respect to the group action
111
R
R
a′1a′2a′3a′4a′5a′6a′7a′8a′9=
Examples of representations
3 scalars
a vector
a1a2a3a4a5a6a7a8a9∈R9
a vector
system rotated by g
Representations are like data types
It tells you how to interpret the data with respect to the group action
a1a2a3a4a5a6a7a8a9
111
R
R
a′1a′2a′3a′4a′5a′6a′7a′8a′9=
Examples of representations
3 scalars
a vector
a1a2a3a4a5a6a7a8a9∈R9
a vector
system rotated by g
Representations are like data types
It tells you how to interpret the data with respect to the group action
a1a2a3a4a5a6a7a8a9
111
R
R
a′1a′2a′3a′4a′5a′6a′7a′8a′9=
Equivariance
V
V′
Equivariance
V
V′
D(g)
D′(g)
V
V′
Equivariance
V
V′
D(g)
D′(g)
V
V′
f
Equivariance
V
V′
D(g)
D′(g)
V
V′
f
f
Equivariance
V
V′
D(g)
D′(g)
V
V′
f
f
f(D(g)x)
Equivariance
V
V′
D(g)
D′(g)
V
V′
f
f
f(D(g)x)
D′(g)f(x)
Equivariance
V
V′
D(g)
D′(g)
V
V′
f
f
f(D(g)x)
D′(g)f(x)
=
![](https://media.slid.es/uploads/948058/images/11400570/pasted-from-clipboard.png)
Fluid dynamics
![](https://media.slid.es/uploads/948058/images/11400573/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11400576/pasted-from-clipboard.png)
Mechanics
Electrodynamics
![](https://media.slid.es/uploads/948058/images/11400578/pasted-from-clipboard.png)
Standard Model
Rotation
Translation
Boosts
(Galilean or Lorentz)
![](https://media.slid.es/uploads/948058/images/11400594/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11402671/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403832/pasted-from-clipboard.png)
![](https://media.slid.es/uploads/948058/images/11403835/pasted-from-clipboard.png)
These models are defined as invariant lagrangians L(state)
L(D(g,state))=L(state)
Time
translation
🔨 Composition
🔧 Basic Arithmetic
🔩 Tensor Product
💡 Linear Mixing
🔨 Composition
two equivariant functions
f:V1→V2
h:V2→V3
h∘f is equivariant!
h(f(D1(g)x))=h(D2(g)f(x))=D3(g)h(f(x))
🔧 Basic Arithmetic +−∗/
two equivariant functions
f:V1→V3
h:V2→V3
h+f is equivariant!
f(D1(g)x)+h(D2(g)x)=D3(g)(f(x)+h(x))
🔧 Basic Arithmetic +−∗/
two equivariant functions
f:V1→V3
h:V2→V3
h+f is equivariant!
f(D1(g)x)+h(D2(g)x)=D3(g)(f(x)+h(x))
equivariant function: f:V1→V2
a scalar: α∈R
αf is equivariant!
αf(D1(g)x)=D2(g)αf(x)
🔩 Tensor Product
x1x2x3
y1y2y3y4y5
transforming with D(g)
transforming with D′(g)
=x1y1x2y1x3y1x1y2x2y2x3y2x1y3x2y3x3y3x1y4x2y4x3y4x1y5x2y5x3y5
🔩 Tensor Product
⊗
=x1y1x2y1x3y1x1y2x2y2x3y2x1y3x2y3x3y3x1y4x2y4x3y4x1y5x2y5x3y5
transforms with D(g)⊗D′(g) 👍
dim(D⊗D′)=dim(D)dim(D′) 👎
x1x2x3
y1y2y3y4y5
Reducible representations
D defined on V
is reducible if
∃W⊂V W=0,V
such that
D∣W is a representation
Reducible representations
Famous Example
x1x2y1x2z1x2x1y2y1y2z1y2x1z2y1z2z1z2
x1y1z1⊗x2y2z2=
D defined on V
is reducible if
∃W⊂V W=0,V
such that
D∣W is a representation
Reducible representations
x1x2+y1y2+z1z2
c(x1z2+z1x2)c(x1y2+y1x2)2y1y2−x1x2−z1z2c(y1z2+z1y2)c(z1z2−x1x2)
y1z2−z1y2z1x2−x1z2x1y2−y1x2
x1x2y1x2z1x2x1y2y1y2z1y2x1z2y1z2z1z2
3×3=1+3+5
D defined on V
is reducible if
∃W⊂V W=0,V
such that
D∣W is a representation
Reducible representations
D defined on V
is irreducible if
only for W=0 or W=V
D∣W is a representation
Irreducible
Irreducible representations
For the group of rotations (SO(3))
They are index by L=0,1,2,…
Of dimension d=2L+1
L=0 | d=1 | scalar |
L=1 | d=3 | vector |
L=2 | d=5 | |
... |
Irreducible representations
For the group of rotations + parity (O(3))
They are index by L=0,1,2,…
and p=±1
Of dimension d=2L+1
Even: p=1
Odd: p=−1
L=0 | d=1 | scalar |
L=1 | d=3 | pseudo vector |
L=2 | d=5 | |
... |
L=0 | d=1 | pseudo scalar |
L=1 | d=3 | vector |
L=2 | d=5 | |
... |
🔩 Tensor Product
L1⊗L2=∣L1−L2∣⊕⋯⊕(L1+L2)
Clebsch-Gordan Theorem
Tells you how to decompose the tensor product of two irreps into irreps
💡 Linear Mixing
3 scalars
a vector
a1a2a3a4a5a6a7a8a9
a vector
3 scalars
a vector
b1b2b3b4b5b6b7b8b9
a vector
Linear map
💡 Linear Mixing
3 scalars
a vector
a1a2a3a4a5a6a7a8a9
a vector
3 scalars
a vector
b1b2b3b4b5b6b7b8b9
a vector
w1
w2
w3
by Schur's lemma
🔨 Composition
🔧 Basic Arithmetic
🔩 Tensor Product
💡 Linear Mixing
🔨 Composition
🔧 Basic Arithmetic
🔩 Tensor Product
💡 Linear Mixing
-
Graph Message Passing
-
Transformer
-
3D Convolution
-
...
Thanks for your Attention!
![](https://media.slid.es/uploads/948058/images/10447946/1mPK7Gk9masrTEm9wAJ_-GvJEycKNaJ6MDfA6b5tI9w.gif)
![](https://media.slid.es/uploads/948058/images/10451946/Screenshot_2023-05-05_at_10.54.12_PM.png)
![](https://media.slid.es/uploads/948058/images/10452721/Screenshot_2023-05-06_at_1.35.47_PM.png)
![](https://media.slid.es/uploads/948058/images/10461383/Screenshot_2023-05-09_at_10.47.11_PM.png)
Data-Efficiency
Protein Folding
Molecular Dynamics
Phonons
Quantum Physics
![](https://media.slid.es/uploads/948058/images/10461392/pasted-from-clipboard.png)
Docking
ML for Drug Discovery Summer School
By Mario Geiger
ML for Drug Discovery Summer School
- 371