Plan

What affects data efficiency in equivariant neural networks?

Group

$a, b, c, e \in G$

• $(ab)c = a(bc)$
• $ea=ae=a$
• $(a^{-1})a=a (a^{-1})=e$

Representations of Rotations

Few examples

Representations of Rotations

The Scalars

$x\longrightarrow x$

Few examples

Representations of Rotations

The Scalars

$x\longrightarrow x$

Few examples

Representations of Rotations

The Scalars

$x\longrightarrow x$

Few examples

Group Representations

$(\rho, V)$

$\rho:G \to (V\to V)$      $g,g_1,g_2 \in G$    $x, y \in V$

• $\rho(g)(x+\alpha y) = \rho(g)(x) + \alpha \rho(g)(y)$
• $\rho(g_2)(\rho(g_1)(x)) = \rho(g_2 g_1)(x)$

The Scalars

$x\longrightarrow x$

Irreducible Representations

The Scalars

$x\longrightarrow x$

irreducible

irreducible

reducible

reducible

Irreducible Representations

reducible

Irreducible Representations

=

$c_1 \times$

$c_2 \times$

$c_3 \times$

$c_4 \times$

$c_5 \times$

irreducible

$c_6 \times$

Irreps of Rotations

Irreps of Rotations

Stress Tensor
(3x3 matrix)

$\}$

$\sigma\longrightarrow R\sigma R^T$

Everything can be decomposed into irreps:

Tensor Product

$\rho_1 \otimes \rho_2$ is a representation

acting on the vector space $V_1 \otimes V_2$

$X \in \mathbb{R}^{\dim V_1\times\dim V_2}$

$X \longrightarrow \rho_1(g) X \rho_2(g)^T$

Tensor Product

$\rho_1 \otimes \rho_2$ is a representation

acting on the vector space $V_1 \otimes V_2$

$X \in \mathbb{R}^{\dim V_1\times\dim V_2}$

$X \longrightarrow \rho_1(g) X \rho_2(g)^T$

($X_{ij} \longrightarrow \rho_1(g)_{ik}\rho_2(g)_{jl} X_{kl}$)

Tensor Product

reducible

=

direct sum of

irreducible

$\rho_1 \otimes \rho_2$

$\rho_3 \oplus \rho_4 \oplus \rho_4$

Tensor Product

$G$

$\rho_1$

$\rho_2$

$\rho_3$

$\rho_4$

$\rho_5$

$\otimes$

$\rho_5$

$\rho_1$

$\rho_2$

Tensor Product of Rotations

Example:

$D_2 \otimes D_1 = D_1 \oplus D_2 \oplus D_3$

$D_L$ is the irreps of order L

Tensor Product of Rotations

$D_L$ is the irreps of order L

General formula:

$D_j \otimes D_k = D_{|j-k|} \oplus \dots \oplus D_{j+k}$

Example:

$D_2 \otimes D_1 = D_1 \oplus D_2 \oplus D_3$

Equivariant Neural Network

Using the tools presented previously you can create any equivariant polynomials

$\rho_1$

$\rho_2$

$\rho_2$

$\rho_3$

$\rho_1$

$\rho_1$

$\rho_2$

$\rho_3$

$\rho_1$

$\rho_2$

$\rho_4$

$\rho_4$

$\rho_1$

$\otimes$

$\otimes$

$\otimes$

$\oplus$

$\oplus$

$\oplus$

$\oplus$

$\otimes$

$\otimes$

$\otimes$

$\oplus$

$\oplus$

$\oplus$

$\oplus$

Equivariant Neural Network

$\theta$

$\theta$

Equivariant Neural Networks Architectures

Library to make ENN for Rotations

We wrote python code to help creating Equivariant Neural Networks

$ pip install e3nn

We wrote python code to help creating Equivariant Neural Networks

Library to make ENN for Rotations

Spherical Harmonics are Equivariant Polynomials

Graph Convolution

Nequip

(TFN: Nathaniel Thomas et al. 2018)

(Nequip: Simon Batzner et al. 2021)

$h$

$\vec r$

$m = h \otimes Y(\vec r)$

$m$

* this formula is missing the parameterized radial function

MACE

(MACE: Ilyes Batatia et al. 2022)

$h_1$

$\vec r_1$

$m = F_\theta(\{h_i\otimes Y(\vec r_i)\}_{i=1}^\nu)$

$m$

$h_2$

$h_\nu$

$\vec r_2$

$\vec r_\nu$

Conclusion

Equivariant Neural Networks are more data efficient if they incorporate Tensor Products of order $L \geq 1$

but not necessary as features (MACE)

Thanks for listening

The slides are available at
https://slides.com/mariogeiger/youth2022