Calculus on

Signed Distance Fields

Vedant Puri

Towards PDE Solving

  • State of the Art
    • Spectral Methods
    • Finite Elements
  • This Work - Goals
  • Results
    • Meshing = Machine Learning Problem
    • Spectra = Machine Learning Problem
  • Surrogate Model Architecture
  • Future Work

Anatomy of Finite Elements

For PDE + domain

  • Discretize geometry (meshing)
  • Fill elements with function space

 

 

  • Solve linear system for scaling coefficients using an iterative method
\implies
-\nabla^2 u = f
\underline{\underline{A}} \underline{u} = \underline{f}
u(x) = \Sigma_{e=1}^E\Sigma_{i=1}^n u_{ie} \psi_{ie}(x)\phi_e(x)

Problem: Meshing

To solve PDEs...

  • Need to be able to represent continuous functions in domain
  • But discretizing geometry is a major bottleneck in SciComp
    • manual process
    • mesh determines quality, AND time to solution
    • => mesh iteration very slow process
  • Deep Neural Networks
    • Composition of linear + nonlinear operations
    • Representation power of neural networks is in their structure
    • Good for high dimensional problems (empirical evidence)
    • Gradient Descent - Modify parameters past nonlinearities
  • Finite Elements
    • Special purpose architectures for solving PDEs in complex geometries
    • developed over 5 decades
    • designed for low dimensional (1D/2D/3D) problems
    • Enforced sparsity 

Tools: FEM, ML

Goal: Solve PDEs in any Geometry

  • Use differentiable programming to tackle nested high dimensional problems in computational workflows
    • auto geometry discretization (meshing)/ adaptive refinement
    • what basis do we do computation in?
  • Develop surrogate architectures based on the Finite Elements
    • Separate layers for spatial localization
    • Separate layers for resolving target features
  • Develop fast (near instant), accurate, automated function approximation scheme in arbitrary geometries
    • current
      describe geometry (sketch, CAD) --> meshing --> PDE solving
    • aim
      Signed Distance Function --> auto-localization --> PDE solving

Adaptive Meshing

 

\text{ReLU}(x) = \begin{cases} 0, &x\leq 0\\ x, &x>0 \end{cases}
\phi(x) = 2\text{ReLU}(x+0.5)\\ \hspace{2.0em}- 4\text{ReLU}(x-0.0)\\ \hspace{2.0em}+ 2\text{ReLU}(x-0.5)

Shallow Neural Networks can recover finite element spaces

=> FEM function space is subset of DNN function space

Adaptive Meshing

  • Linear dependence of y=NN(x) on scaling coeffs,
  • Nonlinear dependence on FEM node locations
    • => Fast training strategy (Gradient Descent/Least Squares)
    • Gradient descent on nodes (moving around basis functions)
    • Fast linear solve (least square) on scaling coefficients
    • Repeat

  • Number of linear partitions increase exponentially with layers

    • NeuralODEs: arbitrary depth

    • => arbitrary # of partitions

    • => arbitrary accuracy

  • Neural Networks can recover finite element spaces
    • ​FEM function space is subspace of DNN function space
  • DistMesh.jl
    • Adaptive mesh refinement in 2D/3D with complex geometries

Meshing - Arbitrary Shape Function

\phi(x) =\sin(\frac{\pi}{2}x)
  • Shape function: anything that goes from zero to one in [0,1]

 

  • Can choose so that partition edges have smooth first derivative

 

\phi(x) = x^2

Mesh = Partition of Unity

\psi_e(x)
\phi_e(x) = \mathbb{I}_e(x)
\{ \phi_i:\mathbb{R}\to\mathbb{R}| \forall x\in\Omega,\, \Sigma_{i=1}^n \phi_i(x) = 1 \}
u(x) = \Sigma_{e=1}^E\Sigma_{i=1}^n u_{ie} \psi_{ie}(x)\phi_e(x)
\phi_e(x) = \mathrm{NN}(x)
  • Meshes don't need to be indicator functions over disjoint sets
  • Mesh = any set of functions that sum to unity everywhere
    • Let                                 => Meshing = Machine Learning problem
    • Make partitions localize to target function features

Constrain Mesh w. Distance Func

  • Mesh topology matters
    • Connectivity determines how derivatives are accumulated
    • Can't grab information from outside the domain
  • Utilize Signed Distance Function information to constrain mesh
    • Clip mesh at domain boundary with SDF
    • Incorporate SDF in training process
\text{SDF}(\vec{x}) = \begin{cases} -\text{ve}, \hspace{1em}& \vec{x}\in\Omega\\ +\text{ve}, \hspace{1em}& \vec{x}\notin\Omega \end{cases}

Function Space Tuning

\psi_{ie}(x) = \cos(\alpha_{ie}x + \beta_{ie})
u(x) = \Sigma_{e=1}^E\Sigma_{i=1}^n u_{ie} \psi_{ie}(x)\phi_e(x)

TLDR: Make functions learnable!

\psi_e(x) = \mathrm{NN}(x)

Inspiration: Element specific function transformations in FEM

Tunable frequency, phase shift

Problem:

Given a target function u(x), how to identify dominant frequencies and phase shifts? Gradient descent!

\to
x = x(\hat{x},\hat{y}),\, y = y(\hat{x},\hat{y})

Model Architecture

  • Input shape function
  • Pick space partitioning technique
    • piecewise (any shape func)
    • neural network
  • Pick (trainable) function space
    • Tensor product in 2D
  • Train model, solve for coeffs
u(x) = \Sigma_{e=1}^E\Sigma_{i=1}^n u_{ie} \psi_{ie}(x)\phi_e(x)
U = \Phi^T C \Psi

Constrain Model, Get Performance

Deconstruct FEM, then reconstruct!

  • Claw back performance by adding structure
  • Develop a notion of integration --> residual minimization for PDEs
  • Want: SDF --> fast-ish localization alg --> Direct-ish coefficient solve
    • improve accuracy with training

Ideas

  • Constrain function space --> direct solve for coeffs (GD too slow!)
    • Orthogonal functions? with arbitrary element shapes?
    • Gradient Descent-Least Square
  • NN architectures for space partitioning
    • Partitions should localize to target function features
    • Incorporate Signed Distance Function info - control level sets
    • How to extend piecewise formulation to any 2D domain?
    • Partitioning = recursive process... NeuralODEs?

Constrain Model, Get Performance

Deconstruct FEM, then reconstruct!

  • Claw back performance by adding structure
  • Develop a notion of integration --> residual minimization for PDEs
  • Want: SDF --> fast-ish localization alg --> Direct-ish coefficient solve
    • improve accuracy with training

Ideas

  • Constrain function space --> direct solve for coeffs (GD too slow!)
    • Orthogonal functions? with arbitrary element shapes?
    • Gradient Descent-Least Square
  • NN architectures for space partitioning
    • Partitions should localize to target function features
    • Incorporate Signed Distance Function info - control level sets
    • How to extend piecewise formulation to any 2D domain?
    • Partitioning = recursive process... NeuralODEs?

Future Work

  • DistMesh.jl
    • Adaptive mesh refinement in 2D/3D with complex geometries
  • Train surrogate models with signed distance functions generated in Descartes.jl
    •  
  •  

Misc Ideas

  • understand geometry - shape function coupling
  • look into non-conformal finite elements for ideas
  • learn conformal mapping using SDFs - check out Schwarz Christoffel maps
  • Fourier transforms with random frequencies - direct solve for coeffs? how to maintain orthogonality in arbitrary domains?
  • Develop a notion of integration, fluxes on element boundaries
  • In relation to exterior calculus - what are 0/1/2-vectors?
  • AtlatsNet type patching
  • want to initialize POU so we always have a valid partition
  • want a function space that can always do a direct solve for coeffs

DiffMesh_2x0

By Vedant Puri

DiffMesh_2x0

  • 75

More from Vedant Puri