Vedant Puri
PhD student at Carnegie Mellon University
Length
Time
Quantum Mechanics
Density Functional Theory
Lattice Boltzmann
Continuum Theory
Molecular Dynamics
1\hspace{0.25em}\dot{\mathrm{A}}
1\hspace{0.25em}\mathrm{nm}
1\hspace{0.25em}\mu\mathrm{m}
1\hspace{0.25em}\mathrm{m}
1\hspace{0.25em}\mathrm{fs}
1\hspace{0.25em}\mathrm{ns}
1\hspace{0.25em}\mu\mathrm{s}
1\hspace{0.25em}\mathrm{s}
The problem of closure modeling
\partial_t \vec{v} + (\vec{v}\cdot\nabla)\vec{v} = -\nabla p + \frac{1}{\mathit{Re}}\Delta \vec{v}\\
\nabla\cdot\vec{v} = 0
i\hbar\partial_t\psi = - \frac{\hbar^2}{2m}\Delta\psi + V\psi
\partial_t F + \nu \cdot \nabla_x F = \mathcal{Q}(F, F)
General Relativity
G_{\mu\nu} + \Lambda g_{\mu\nu} = \kappa T_{\mu\nu}
\mathbf{H}\psi = \mathbf{E}\psi
1\hspace{0.25em}\mathrm{km}
F(X) = M\dot{V}(t)\\
V(t) = \dot{X}(t)
Our goal is to add ML to the simulation pipeline
Mesh
\(A\underline{u} = M\underline{f} \)
Domain
Governing Equation
Boundary Constraint
\( NN_\theta \)
Discretization
\( \dfrac{d}{dt} \underline{u} = f(\underline{u}) \)
Solving
Discrete Problems
\(u(\underline{x},t) \)
\(u(\underline{x}) \)
Solution
Loss
Backpropogation
Data
\( NN_\theta \)
\( NN_\theta \)
Linear Solver
Time-Stepper
Need high-level, fast, AD-compatible software ecosystem!
| Orthogonal Functions | Deep Neural Networks |
|---|---|
|
 |
|
Curse of Dimensionality in Approximation Theory
f = \tilde{f} + \mathcal{O}(h)
\tilde{f} = \Sigma_{i=1}^N f_i \phi_i(x)
\tilde{f} = W_L \circ \sigma(W_{L-1}\circ(\dotsc (\sigma W_0(x))) + b_{L-1})
h \sim N^{-c/d}
h \sim 1 / N\, \mathsf{(for\, 2\, layer\, networks)}
Curse of dimensionality
Dimension independent
Model size scales only with the complexity of the signal.
Landscape of ML for PDEs
Mesh-based
PDE-Based
Neural Ansatz
Data-driven
FEM, FVM, IGA, Spectral
Fourier Neural Operator
Implicit Neural Representations
DeepONet
Physics Informed NNs
Convolution NNs
Graph NNs
Adapted from Núñez, CEMRACS 2023
Neural ODEs
Universal Diff Eq
u =
\dfrac{du}{dt} =
\dfrac{d\tilde{u}}{dt} = \tilde{\mathcal{L}}_p(\tilde{u}) +
\dfrac{du}{dt} = \mathcal{L}_p(u) + \mathcal{N}_p(u)
\begin{cases}
\dfrac{d u}{dt} = \mathcal{L}_p(u) + \mathcal{N}_p(u), & x\in\Omega\\
u|_{\partial\Omega} = g(t)
\end{cases}
Reduced Order Modeling
g_\text{out}
\vec{x}
g_\text{in}
\vec{x}
LPBF Process Parameters
Learned surrogate map
Input geometry
Output geometry
Numerically simulate LPBF
g
\vec{x}
Signed distance field
Deep implicit neural network
Learned embedding
Signed distance field
G_i
G_\text{in}
G_\text{out}
g_\text{in}
\vec{x}
g_\text{out}
\vec{x}
g_i
\vec{x}
Residual deformation prediction model
LPBF process parameters
Finite element simulation of residual deformation
Learn implicit geometry
embedding
Calculate model uncertainty
3
4
1
2
Signed distance field
G_i
G_\text{in}
G_\text{out}
g_\text{in}
\vec{x}
g_\text{out}
\vec{x}
g_i
\vec{x}
2.2 Residual deformation prediction model
LPBF process parameters
Task 1: Generate dataset of residual deformation
2.1 Learn implicit geometry embedding
Task 3: Calculate model uncertainty
Task 2: Machine learning framework
Signed distance field
G_i
G_\text{CAD}
G_\text{RD}
g_\text{CAD}
g_\text{RD}
g_i
x
2.2 Residual deformation prediction model
2.1 Learn low-dimensional geometry representation
\( i \in \{ \text{CAD}, \, \text{RD}\}\)
\( \theta\)
\( \{\)
High-fidelity simulation data
\vec{u}_1
\vec{u}_2
\vec{u}_3
\tilde{u}
Collect and compress data
\tilde{u}(t)
\vec{u}_n(t)
Low-dim system sovle
\(\text{Coordinates}\)
{\tilde{u}}
\boldsymbol{x}
\boldsymbol{u}\left( \boldsymbol{x}, t; \boldsymbol{\mu} \right)
\(\text{Loss function}\)
\(\text{Dynamical}\)
\(\text{PDE system}\)
\(\text{ Parameters}\)
\( \text{and time}\)
\(\text{Neural\, field MLP}\)
\(\text{Backpropagation}\)
\(\text{2D Burgers, }\mathit{Re}=1k:\vec{u}+(\vec{u}\cdot\vec{\nabla})\vec{u} = \nu\Delta \vec{u}\)
\(\text{1D KS: }u_t + u_{xx}+\nu u_{xxxx} + uu_x = 0\)
\(\text{1D Advection: }u_t + c u_x=0\)
\(\text{1D Burgers, }\mathit{Re}=10k:\, u_t + uu_x = \nu u_{xx} \)
\(\text{1D KS: }u_t + u_{xx}+\nu u_{xxxx} + uu_x = 0\)
\(\text{1D Burgers, }\mathit{Re}=10k:\, u_t + uu_x = \nu u_{xx} \)
Task 1: Generate dataset of LPBF simulations
\(\text{Mesh signal encoding}\)
\hat{u}
\hat{x}
\(\text{Attention}\\\text{projector}\)
\times
[N, D]
[N, D]
\hat{c}
[M, D]
[M, N]
\hat{z}
[M, D]
Task 2: Learn mesh signal representation
Task 3: Layer-by-layer auto-regressive build
\times L
\(\text{Mesh}\)
\(\text{Transformer block}\)
\(\hat{z}_{1, t+\Delta t}\)​
\(\hat{z}_{2, t+\Delta t}\)
\(\hat{z}_{\ell-1, t+\Delta t}\)
\(\hat{z}_{\ell, t+\Delta t}\)
\(\hat{z}_{\ell+1, t+\Delta t}\)
\(\text{Mesh}\)
\(\text{Mesh signal decoding}\)
\hat{u}
\hat{x}
\times
[N, D]
[N, D]
[N, M]
\hat{z}
[M, D]
\(\text{encoding}\)
\(\text{decoding}\)
\(\text{Node embeddings}\)
\(\text{Signal} \\ \text{encoding}\)
\(\text{Self attention}\)
\(\text{Cross attention}\)
\(\text{Attention}\\\text{projector}\)
\hat{c}
[M, D]
\(\text{Time } t\)
\(\text{Time } t+1\)
\vdots
\(\hat{z}_{1, t}\)​
\(\hat{z}_{2, t}\)
\(\hat{z}_{\ell-1,t}\)
\(\hat{z}_{\ell, t}\)
\vdots
\(\text{Layerwise disaggregation}\)
\(\text{LPBF process}\\\text{parameters}\)
\(\text{3D shape dataset [32]}\)
\(\text{Layer-by-layer residual}\\ \text{deformation calculation}\)
\(\text{Mesh signal encoding}\)
\hat{u}
\hat{x}
\(\text{Attention}\\\text{projector}\)
\times
[N, D]
[N, D]
\hat{c}
[M, D]
[M, N]
\hat{z}
[M, D]
\(\text{Mesh signal decoding}\)
\hat{u}
\hat{x}
\times
[N, D]
[N, D]
[N, M]
\hat{z}
[M, D]
\(\text{Signal} \\ \text{encoding}\)
\(\text{Attention}\\\text{projector}\)
\hat{c}
[M, D]
\(\text{MLP}\)
\(\text{Cluster Attention}\)
\(\text{block}\)
\(\text{MLP}\)
\times B
\(\text{Mesh signal encoding}\)
\hat{u}
\hat{x}
\(\text{Attention}\\\text{projector}\)
\times
[N, D]
[N, D]
\hat{c}
[M, D]
[M, N]
\hat{z}
[M, D]
\(\text{Mesh signal decoding}\)
\hat{u}
\hat{x}
\times
[N, D]
[N, D]
[N, M]
\hat{z}
[M, D]
\(\text{Attention}\\\text{projector}\)
\hat{c}
[M, D]
\times B_\ell
\(\text{Self attention}\)
\(\text{MLP Block}\)
\(\text{Vanilla}\)\(\text{Transformer block}\)
figs
By Vedant Puri
