Davide Murari
ICIAM - 22/08/2023
davide.murari@ntnu.no
In collaboration with : Elena Celledoni, Brynjulf Owren, Carola-Bibiane Schönlieb and Ferdia Sherry
Structured neural networks and some applications
Neural networks motivated by dynamical systems
N(x)=fθL∘...∘fθ1(x)
Neural networks motivated by dynamical systems
N(x)=fθL∘...∘fθ1(x)
x˙(t)=F(x(t),θ(t))=:Fs(t)(x(t))
Where Fi(x)=F(x,θi)
θ(t)≡θi,t∈[ti,ti+1),hi=ti−ti−1
Neural networks motivated by dynamical systems
Neural networks motivated by dynamical systems
Accuracy is not all you need
X , Label : Plane
X+δ, ∥δ∥2=0.3 , Label : Cat
GENERAL IDEA
EXAMPLE
Property P
P= Volume preservation
Imposing some structure
GENERAL IDEA
EXAMPLE
Property P
P= Volume preservation
Family F of vector fields that satisfy P
Fθ(x,v)=[Σ(Av+a)Σ(Bx+b) ]
F={Fθ:θ∈P}
Imposing some structure
GENERAL IDEA
EXAMPLE
Property P
P= Volume preservation
Family F of vector fields that satisfy P
Fθ(x,v)=[Σ(Av+a)Σ(Bx+b) ]
F={Fθ:θ∈P}
Integrator Ψh that preserves P
Imposing some structure
Mass-preserving networks
Lipschitz-constrained networks
m=1
m=21
Σ(x)=max{x,2x}
Lipschitz-constrained networks
Lipschitz-constrained networks
Adversarial robustness
Thank you for the attention
- Celledoni, E., Murari, D., Owren B., Schönlieb C.B., Sherry F, preprint (2022). Dynamical systems' based neural networks
davide.murari@ntnu.no
Examples
1-LIPSCHITZ NETWORKS
HAMILTONIAN NETWORKS
VOLUME PRESERVING, INVERTIBLE
Naively constrained networks