a differentiable physics engine for robotics
Taylor Howell and Simon Le Cleac'h
Taylor Howell
Simon Le Cleac'h
team
thowell@stanford.edu
simonlc@stanford.edu
Taylor Howell
Simon Le Cleac'h
Jan Brüdigam
Zico Kolter
Mac Schwager
Zachary Manchester
team
existing physics engines
contact physics
existing physics engines
contact physics
LCP
existing physics engines
contact physics
LCP
implicit complementarity
existing physics engines
contact physics
LCP
implicit complementarity
gradients
existing physics engines
contact physics
LCP
implicit complementarity
gradients
samples
existing physics engines
contact physics
LCP
implicit complementarity
gradients
samples
subgradient
existing physics engines
popular simulators
https://leggedrobotics.github.io/SimBenchmark/
motivation
contact dynamics
gradients
open-source
Dojo key ideas
variational integrator
interior-point methods
implicit differentiation
Dojo key ideas
stability at low rates
variational integrator
interior-point methods
implicit differentiation
Dojo key ideas
stability at low rates
variational integrator
interior-point methods
accurate contact dynamics
implicit differentiation
Dojo key ideas
stability at low rates
variational integrator
interior-point methods
accurate contact dynamics
implicit differentiation
smooth gradients
variational integrator
Discrete mechanics and variational integrators. J. E. Marsden and M. West.
S = \int_{t_1}^{t_2}{\mathcal{L} dt}
variational integrator
Discrete mechanics and variational integrators. J. E. Marsden and M. West.
S = \int_{t_1}^{t_2}{\mathcal{L} dt}
Euler-Lagrange
F = m a
variational integrator
Discrete mechanics and variational integrators. J. E. Marsden and M. West.
S = \int_{t_1}^{t_2}{\mathcal{L} dt}
discretize
Euler-Lagrange
F = m a
p_+ = p + h (v + mg)
variational integrator
Discrete mechanics and variational integrators. J. E. Marsden and M. West.
S = \int_{t_1}^{t_2}{\mathcal{L} dt}
S_D = h \sum_{i=1}^{N} \mathcal{L}_i
discretize
discretize
Euler-Lagrange
F = m a
p_+ = p + h (v + mg)
variational integrator
m(p_+ -2p +p_-)/h - hmg = 0
Discrete mechanics and variational integrators. J. E. Marsden and M. West.
S = \int_{t_1}^{t_2}{\mathcal{L} dt}
discretize
discretize
Euler-Lagrange
Euler-Lagrange
F = m a
p_+ = p + h (v + mg)
S_D = h \sum_{i=1}^{N} \mathcal{L}_i
variational integrator
-
compare astronaut energy and momentum conservation to MuJoCo
variational integrator
-
compare astronaut energy and momentum conservation to MuJoCo
- Dojo performs orders of magnitude better
energy and momentum conservation
maximal-coordinates representation
(x_1,v_1,q_1,\omega_1) \in \mathbf{R}^{13}
\mathbf{R}^{169}
(x_2,v_2,q_2,\omega_2) \in \mathbf{R}^{13}
(x_3,v_3,q_3,\omega_3) \in \mathbf{R}^{13}
(x_4,v_4,q_4,\omega_4) \in \mathbf{R}^{13}
maximal-coordinates representation
Linear-Time Variational Integrators in Maximal Coordinates. J. Brudigam and Z. Manchester.
Linear-Time Contact and Friction Dynamics in Maximal Coordinates using Variational Integrators.
J. Brudigam and Z. Manchester.
maximal-coordinates representation
accurate contact dynamics
no collision violations
accurate contact dynamics
no collision violations
correct Coulomb friction
MuJoCo linear
Dojo linear
MuJoCo nonlinear
Dojo nonlinear
accurate contact dynamics
no collision violations
correct Coulomb friction
interior-point method
impact → inequalities
friction → second-order cone
cone constraints
MuJoCo linear
Dojo linear
MuJoCo nonlinear
Dojo nonlinear
accurate contact dynamics
nonlinear complementarity problem
accurate contact dynamics
custom interior-point solver
- Mehrotra predictor-corrector algorithm
- CVXOpt second-order cones
- non-Euclidean support for quaternions
interior-point solver
\text{minimize} \quad f(x)
x
\text{subject to}
\phi(x) \geq 0
interior-point solver
\text{minimize} \quad f(x)
x
\text{subject to}
\phi(x) \geq 0
\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))
x
interior-point solver
\text{minimize} \quad f(x)
x
\text{subject to}
\phi(x) \geq 0
\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))
x
x
f(x) = x \\
\phi(x) = x
interior-point solver
\text{minimize} \quad f(x)
x
\text{subject to}
\phi(x) \geq 0
\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))
x
x
f(x) = x \\
\phi(x) = x
interior-point solver
\text{minimize} \quad f(x)
x
\text{subject to}
\phi(x) \geq 0
\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))
x
x
f(x) = x \\
\phi(x) = x
interior-point solver
\text{minimize} \quad f(x)
x
\text{subject to}
\phi(x) \geq 0
\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))
x
x
x
f(x) -\kappa \:\log(\phi(x))
f(x) = x \\
\phi(x) = x
interior-point solver
\text{minimize} \quad f(x)
x
\text{subject to}
\phi(x) \geq 0
\kappa \rightarrow 0
\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))
x
x
x
f(x) -\kappa \:\log(\phi(x))
f(x) = x \\
\phi(x) = x
smooth gradients
Lezioni di analisi infinitesimale. U. Dini.
smooth gradients
implicit-function theorem
Lezioni di analisi infinitesimale. U. Dini.
r(w^*; \theta) = 0
→
smooth gradients
sensitivity of solution w.r.t problem data
smooth gradients
sensitivity of solution w.r.t problem data
computation cost of gradient is less than simulation step
smooth gradients
sensitivity of solution w.r.t problem data
computation cost of gradient is less than simulation step
differentiate intermediate barrier problems for smooth gradients
smooth gradients
box push
smooth gradients
box push
non-smooth dynamics
smooth gradients
box push
non-smooth dynamics
gradient comparison
smooth gradients
box push
non-smooth dynamics
gradient comparison
less expensive to compute compared to finite-difference or stochastic sampling
github.com/dojo-sim
open-source implementation
github.com/dojo-sim
- Julia package: Dojo.jl
- gym-like environments
- Python wrapper: dojopy
- interface w/ PyTorch & JAX
open-source implementation
examples
trajectory optimization
trajectory optimization
smooth-gradient-based optimization with iterative LQR
trajectory optimization
smooth-gradient-based optimization with iterative LQR
stability at low rates enables 2-5x sample-complexity improvement over MuJoCo
reinforcement learning
train static linear policies for locomotion
reinforcement learning
train static linear policies for locomotion
gradients enable 5-10x sample-complexity improvement over derivative-free method
reinforcement learning
train static linear policies for locomotion
gradients enable 5-10x sample-complexity improvement over derivative-free method
stability at low rates enables 2-5x sample-complexity improvement over MuJoCo
system identification
ContactNets: Learning Discontinuous Contact Dynamics with Smooth, Implicit Representations. S. Pfrommer, M. Halm, and M. Posa.
real-word dataset
system identification
ContactNets: Learning Discontinuous Contact Dynamics with Smooth, Implicit Representations. S. Pfrommer, M. Halm, and M. Posa.
learned
ground-truth
real-word dataset
Dojo environment
system identification
geometry
friction coefficient
ground-truth
learned
system identification
geometry
friction coefficient
ground-truth
learned
Quasi-Newton method utilizes gradients to
learn parameters to 95% accuracy in 20 steps
related work
model-predictive control
Fast Contact-Implicit Model-Predictive Control.
S. Le Cleac'h & T. Howell, C. Lee, S. Yang, M. Schwager, Z. Manchester
simulation
push recovery
behavior generation
running Julia-based policy at 200-500Hz
NeRF
Differentiable Physics Simulation of Dynamics-Augmented Neural Objects. S. Le Cleac'h, HX. Yu, M. Guo, T. Howell, R. Gao, J. Wu, Z. Manchester, M. Schwager
dynamics-augmented NeRF → complex collision geometries
differentiable collision
Differentiable Collision Detection for a Set of Convex Primitives
K. Tracy, T. Howell, S. Le Cleac'h, Z. Manchester
direct trajectory optimization
CALIPSO: A Differentiable Solver for Trajectory Optimization with Conic and Complementarity Constraints
T. Howell, K. Tracy, S. Le Cleac'h, Z. Manchester
github.com/dojo-sim
dojo_dm
By taylorhowell
