a differentiable physics engine for robotics

Taylor Howell and Simon Le Cleac'h

Taylor Howell

Simon Le Cleac'h

team

thowell@stanford.edu

simonlc@stanford.edu

Taylor Howell

Simon Le Cleac'h

Jan Brüdigam

Zico Kolter

Mac Schwager

Zachary Manchester

team

existing physics engines

contact physics

existing physics engines

contact physics

LCP

existing physics engines

contact physics

LCP

implicit complementarity

existing physics engines

contact physics

LCP

implicit complementarity

gradients

existing physics engines

contact physics

LCP

implicit complementarity

gradients

samples

existing physics engines

contact physics

LCP

implicit complementarity

gradients

samples

subgradient

existing physics engines

popular simulators

https://leggedrobotics.github.io/SimBenchmark/

motivation

contact dynamics

gradients

open-source

Dojo key ideas

variational integrator

interior-point methods

implicit differentiation

Dojo key ideas

stability at low rates

variational integrator

interior-point methods

implicit differentiation

Dojo key ideas

stability at low rates

variational integrator

interior-point methods

accurate contact dynamics

implicit differentiation

Dojo key ideas

stability at low rates

variational integrator

interior-point methods

accurate contact dynamics

implicit differentiation

smooth gradients

variational integrator

Discrete mechanics and variational integrators. J. E. Marsden and M. West.

S = \int_{t_1}^{t_2}{\mathcal{L} dt}

variational integrator

Discrete mechanics and variational integrators. J. E. Marsden and M. West.

S = \int_{t_1}^{t_2}{\mathcal{L} dt}

Euler-Lagrange

F = m a

variational integrator

Discrete mechanics and variational integrators. J. E. Marsden and M. West.

S = \int_{t_1}^{t_2}{\mathcal{L} dt}

discretize

Euler-Lagrange

F = m a

p_+ = p + h (v + mg)

variational integrator

Discrete mechanics and variational integrators. J. E. Marsden and M. West.

S = \int_{t_1}^{t_2}{\mathcal{L} dt}

S_D = h \sum_{i=1}^{N} \mathcal{L}_i

discretize

Euler-Lagrange

F = m a

p_+ = p + h (v + mg)

variational integrator

m(p_+ -2p +p_-)/h - hmg = 0

Discrete mechanics and variational integrators. J. E. Marsden and M. West.

S = \int_{t_1}^{t_2}{\mathcal{L} dt}

discretize

Euler-Lagrange

F = m a

p_+ = p + h (v + mg)

S_D = h \sum_{i=1}^{N} \mathcal{L}_i

variational integrator

compare astronaut energy and momentum conservation to MuJoCo

variational integrator

compare astronaut energy and momentum conservation to MuJoCo

Dojo performs orders of magnitude better

energy and momentum conservation

maximal-coordinates representation

(x_1,v_1,q_1,\omega_1) \in \mathbf{R}^{13}

\mathbf{R}^{169}

(x_2,v_2,q_2,\omega_2) \in \mathbf{R}^{13}

(x_3,v_3,q_3,\omega_3) \in \mathbf{R}^{13}

(x_4,v_4,q_4,\omega_4) \in \mathbf{R}^{13}

maximal-coordinates representation

Linear-Time Variational Integrators in Maximal Coordinates. J. Brudigam and Z. Manchester.
Linear-Time Contact and Friction Dynamics in Maximal Coordinates using Variational Integrators. 
J. Brudigam and Z. Manchester.

maximal-coordinates representation

accurate contact dynamics

no collision violations

accurate contact dynamics

no collision violations

correct Coulomb friction

MuJoCo linear

Dojo linear

MuJoCo nonlinear

Dojo nonlinear

accurate contact dynamics

no collision violations

correct Coulomb friction

interior-point method

impact → inequalities

friction → second-order cone

cone constraints

MuJoCo linear

Dojo linear

MuJoCo nonlinear

Dojo nonlinear

accurate contact dynamics

nonlinear complementarity problem

accurate contact dynamics

custom interior-point solver

Mehrotra predictor-corrector algorithm
CVXOpt second-order cones
non-Euclidean support for quaternions

interior-point solver

\text{minimize} \quad f(x)

\text{subject to}

\phi(x) \geq 0

interior-point solver

\text{minimize} \quad f(x)

\text{subject to}

\phi(x) \geq 0

\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))

interior-point solver

\text{minimize} \quad f(x)

\text{subject to}

\phi(x) \geq 0

\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))

f(x) = x \\ \phi(x) = x

interior-point solver

\text{minimize} \quad f(x)

\text{subject to}

\phi(x) \geq 0

\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))

f(x) = x \\ \phi(x) = x

interior-point solver

\text{minimize} \quad f(x)

\text{subject to}

\phi(x) \geq 0

\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))

f(x) = x \\ \phi(x) = x

interior-point solver

\text{minimize} \quad f(x)

\text{subject to}

\phi(x) \geq 0

\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))

f(x) -\kappa \:\log(\phi(x))

f(x) = x \\ \phi(x) = x

interior-point solver

\text{minimize} \quad f(x)

\text{subject to}

\phi(x) \geq 0

\kappa \rightarrow 0

\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))

f(x) -\kappa \:\log(\phi(x))

f(x) = x \\ \phi(x) = x

smooth gradients

Lezioni di analisi infinitesimale. U. Dini.

smooth gradients

implicit-function theorem

Lezioni di analisi infinitesimale. U. Dini.

r(w^*; \theta) = 0

→

smooth gradients

sensitivity of solution w.r.t problem data

smooth gradients

sensitivity of solution w.r.t problem data

computation cost of gradient is less than simulation step

smooth gradients

sensitivity of solution w.r.t problem data

computation cost of gradient is less than simulation step

differentiate intermediate barrier problems for smooth gradients

smooth gradients

box push

smooth gradients

box push

non-smooth dynamics

smooth gradients

box push

non-smooth dynamics

gradient comparison

smooth gradients

box push

non-smooth dynamics

gradient comparison

less expensive to compute compared to finite-difference or stochastic sampling

github.com/dojo-sim

open-source implementation

github.com/dojo-sim

Julia package: Dojo.jl
- gym-like environments
Python wrapper: dojopy
- interface w/ PyTorch & JAX

open-source implementation

examples

trajectory optimization

smooth-gradient-based optimization with iterative LQR

trajectory optimization

smooth-gradient-based optimization with iterative LQR

stability at low rates enables 2-5x sample-complexity improvement over MuJoCo

reinforcement learning

train static linear policies for locomotion

reinforcement learning

train static linear policies for locomotion

gradients enable 5-10x sample-complexity improvement over derivative-free method

reinforcement learning

train static linear policies for locomotion

gradients enable 5-10x sample-complexity improvement over derivative-free method

stability at low rates enables 2-5x sample-complexity improvement over MuJoCo

system identification

ContactNets: Learning Discontinuous Contact Dynamics with Smooth, Implicit Representations. 
S. Pfrommer, M. Halm, and M. Posa.

real-word dataset

system identification

ContactNets: Learning Discontinuous Contact Dynamics with Smooth, Implicit Representations. 
S. Pfrommer, M. Halm, and M. Posa.

learned

ground-truth

real-word dataset

Dojo environment

system identification

geometry

friction coefficient

ground-truth

learned

system identification

geometry

friction coefficient

ground-truth

learned

Quasi-Newton method utilizes gradients to

learn parameters to 95% accuracy in 20 steps

related work

model-predictive control

Fast Contact-Implicit Model-Predictive Control. 
S. Le Cleac'h & T. Howell, C. Lee, S. Yang, M. Schwager, Z. Manchester

simulation

push recovery

behavior generation

running Julia-based policy at 200-500Hz

NeRF

Differentiable Physics Simulation of Dynamics-Augmented Neural Objects. 
S. Le Cleac'h, HX. Yu, M. Guo, T. Howell, R. Gao, J. Wu, Z. Manchester, M. Schwager

dynamics-augmented NeRF → complex collision geometries

differentiable collision

Differentiable Collision Detection for a Set of Convex Primitives
K. Tracy, T. Howell, S. Le Cleac'h, Z. Manchester

direct trajectory optimization

CALIPSO: A Differentiable Solver for Trajectory Optimization with Conic and Complementarity Constraints
T. Howell, K. Tracy, S. Le Cleac'h, Z. Manchester