Dojo:
A differentiable simulator for robotics
Simon Le Cleac'h and Taylor Howell
Taylor Howell
Simon Le Cleac'h
Jan Brüdigam
Zico Kolter
Mac Schwager
Zachary Manchester
team
Dojo is a physics engine
that prioritizes
physical accuracy
and
differentiability
doesn't
MuJoCo/Drake/Bullet/Brax
already do that?
popular simulators
https://leggedrobotics.github.io/SimBenchmark/
github.com/dojo-sim
- Julia package: Dojo.jl
- Python wrapper: dojopy
- gym-like environments
- interface w/ PyTorch & JAX
- maximal-coordinates representation
- variational integrator
- higher-fidelity contact model
- custom interior-point solver
- smooth gradients
Dojo
maximal-coordinates representation
minimal-coordinates
representation
minimal-coordinates
representation
(x,v,q,\omega) \in \mathbf{R}^{13}
minimal-coordinates
representation
(x,v,q,\omega) \in \mathbf{R}^{13}
(\theta_1, \dot{\theta}_1) \in \mathbf{R}^2
minimal-coordinates
representation
(x,v,q,\omega) \in \mathbf{R}^{13}
(\theta_1, \dot{\theta}_1) \in \mathbf{R}^2
(\theta_2, \dot{\theta}_2) \in \mathbf{R}^2
minimal-coordinates
representation
(x,v,q,\omega) \in \mathbf{R}^{13}
(\theta_1, \dot{\theta}_1) \in \mathbf{R}^2
(\theta_2, \dot{\theta}_2) \in \mathbf{R}^2
(\theta_3, \dot{\theta}_3) \in \mathbf{R}^2
minimal-coordinates
representation
(x,v,q,\omega) \in \mathbf{R}^{13}
(\theta_1, \dot{\theta}_1) \in \mathbf{R}^2
(\theta_2, \dot{\theta}_2) \in \mathbf{R}^2
(\theta_3, \dot{\theta}_3) \in \mathbf{R}^2
\mathbf{R}^{37}
maximal-coordinates
representation
maximal-coordinates
representation
(x_1,v_1,q_1,\omega_1) \in \mathbf{R}^{13}
maximal-coordinates
representation
(x_1,v_1,q_1,\omega_1) \in \mathbf{R}^{13}
(x_2,v_2,q_2,\omega_2) \in \mathbf{R}^{13}
maximal-coordinates
representation
(x_1,v_1,q_1,\omega_1) \in \mathbf{R}^{13}
(x_2,v_2,q_2,\omega_2) \in \mathbf{R}^{13}
(x_3,v_3,q_3,\omega_3) \in \mathbf{R}^{13}
maximal-coordinates
representation
(x_1,v_1,q_1,\omega_1) \in \mathbf{R}^{13}
(x_2,v_2,q_2,\omega_2) \in \mathbf{R}^{13}
(x_3,v_3,q_3,\omega_3) \in \mathbf{R}^{13}
(x_4,v_4,q_4,\omega_4) \in \mathbf{R}^{13}
maximal-coordinates
representation
(x_1,v_1,q_1,\omega_1) \in \mathbf{R}^{13}
\mathbf{R}^{169}
(x_2,v_2,q_2,\omega_2) \in \mathbf{R}^{13}
(x_3,v_3,q_3,\omega_3) \in \mathbf{R}^{13}
(x_4,v_4,q_4,\omega_4) \in \mathbf{R}^{13}
maximal-coordinates representation
Linear-Time Variational Integrators in Maximal Coordinates. J. Brudigam and Z. Manchester.
Linear-Time Contact and Friction Dynamics in Maximal Coordinates using Variational Integrators.
J. Brudigam and Z. Manchester.
variational integrator
variational integrator
Discrete mechanics and variational integrators. J. E. Marsden and M. West.
S = \int_{t_1}^{t_2}{\mathcal{L} dt}
variational integrator
Discrete mechanics and variational integrators. J. E. Marsden and M. West.
S = \int_{t_1}^{t_2}{\mathcal{L} dt}
Euler-Lagrange
F = m a
variational integrator
Discrete mechanics and variational integrators. J. E. Marsden and M. West.
S = \int_{t_1}^{t_2}{\mathcal{L} dt}
discretize
Euler-Lagrange
F = m a
p_+ = p + h (v + mg)
variational integrator
Discrete mechanics and variational integrators. J. E. Marsden and M. West.
S = \int_{t_1}^{t_2}{\mathcal{L} dt}
S_D = h \sum_{i=1}^{N} \mathcal{L}_i
discretize
discretize
Euler-Lagrange
F = m a
p_+ = p + h (v + mg)
variational integrator
m(p_+ -2p +p_-)/h - hmg = 0
Discrete mechanics and variational integrators. J. E. Marsden and M. West.
S = \int_{t_1}^{t_2}{\mathcal{L} dt}
discretize
discretize
Euler-Lagrange
Euler-Lagrange
F = m a
p_+ = p + h (v + mg)
S_D = h \sum_{i=1}^{N} \mathcal{L}_i
energy and momentum conservation
tennis-racket effect
astronaut
Simulation Tools for Model-Based Robotics: Comparison of Bullet, Havok, MuJoCo, ODE and PhysX.
T. Erez, Y. Tassa, and E. Todorov.
energy and momentum conservation
variational integrator
Euler:
J\dot{\omega} + \omega \times (I \omega) = \tau
contact model
floor penetration
drift
contact model
friction
On unilateral constraints, friction and plasticity. J. J. Moreau.
\text{minimize} \quad \text{kinetic energy} \\
\text{friction forces}
\text{subject to} \\
contact model
friction
On unilateral constraints, friction and plasticity. J. J. Moreau.
\text{minimize} \quad \text{kinetic energy} \\
\text{friction forces}
\text{subject to} \\
friction cones
approximation
exact
friction cones
Subtitle
MuJoCo linear
Dojo linear
MuJoCo nonlinear
Dojo nonlinear
contact model
impact
An Implicit Time-Stepping Scheme for Rigid Body Dynamics with Inelastic Collisions and Coloumb Friction. D. E. Stewart and J. C. Trinkle.
x
p
x
contact model
impact
An Implicit Time-Stepping Scheme for Rigid Body Dynamics with Inelastic Collisions and Coloumb Friction. D. E. Stewart and J. C. Trinkle.
\phi(x) \geq 0
x
p
x
contact model
impact
An Implicit Time-Stepping Scheme for Rigid Body Dynamics with Inelastic Collisions and Coloumb Friction. D. E. Stewart and J. C. Trinkle.
\phi(x) \geq 0
x
\gamma \geq 0
p
x
contact model
impact
An Implicit Time-Stepping Scheme for Rigid Body Dynamics with Inelastic Collisions and Coloumb Friction. D. E. Stewart and J. C. Trinkle.
\phi(x) \geq 0
x
\gamma \geq 0
p
\gamma \circ \phi(x) = 0
x
impact
Dojo
MuJoCo
interior-point solver
\text{minimize} \quad f(x)
x
\text{subject to}
\phi(x) \geq 0
interior-point solver
\text{minimize} \quad f(x)
x
\text{subject to}
\phi(x) \geq 0
\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))
x
interior-point solver
\text{minimize} \quad f(x)
x
\text{subject to}
\phi(x) \geq 0
\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))
x
x
f(x) = x \\
\phi(x) = x
interior-point solver
\text{minimize} \quad f(x)
x
\text{subject to}
\phi(x) \geq 0
\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))
x
x
f(x) = x \\
\phi(x) = x
interior-point solver
\text{minimize} \quad f(x)
x
\text{subject to}
\phi(x) \geq 0
\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))
x
x
f(x) = x \\
\phi(x) = x
interior-point solver
\text{minimize} \quad f(x)
x
\text{subject to}
\phi(x) \geq 0
\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))
x
x
x
f(x) -\kappa \:\log(\phi(x))
f(x) = x \\
\phi(x) = x
interior-point solver
\text{minimize} \quad f(x)
x
\text{subject to}
\phi(x) \geq 0
\kappa \rightarrow 0
\text{minimize} \quad f(x) -\kappa \:\log(\phi(x))
x
x
x
f(x) -\kappa \:\log(\phi(x))
f(x) = x \\
\phi(x) = x
interior-point solver
KKT conditions
\nabla f(x) + \nabla \phi(x)^T \gamma = 0 \\
\gamma \cdot \phi(x) = \kappa\\
\phi(x), \gamma \geq 0
\text{find} \quad x, \gamma
\text{subject to}
\phi
\gamma
p
\phi
\gamma
\gamma \cdot \phi(x) = \kappa
interior-point solver
KKT conditions
\nabla f(x) + \nabla \phi(x)^T \gamma = 0 \\
\gamma \cdot \phi(x) = \kappa \\
\phi(x), \gamma \ge 0
\text{find} \quad x, \gamma
\text{subject to}
\phi
\gamma
p
\phi
\gamma
\gamma \cdot \phi(x) = \kappa
0 \leftarrow \kappa
interior-point solver
\geq 0
interior-point solver
\geq 0
interior-point solver
using Plots
using Dojo
rectangle(w, h, x, y) = Shape(x .+ [0,w,w,0], y .+ [0,0,h,h])
light_blue = RGBA(0.4,0.4,1.0,0.8)
f(x) = x
ϕ(x) = x
L(x, ρ) = f(x) - ρ * log.(ϕ(x))
X = 0:0.0001:1
Y = -1:0.1:1
plt = plot(ylims=(-0.2,1.0), xlims=(-0.2,1.0), yticks=[0,1], xticks=[0,1], legend=:bottomright, size=(300,300))
plot!(plt, Y, f.(Y), linewidth=6.0, color=:black, label="f(x)")
plot!(rectangle(-0.2,2,0,-1), opacity=.6, color=:red, label="ϕ(x) < 0")
anim = @animate for i = 1:10
plot!(plt, X, [1; L.(X[2:end], 10*0.5^i)], linewidth=5.0, color=light_blue, label=false)# label="ρ = 3e-1")
end
gif(anim, fps=10, "/home/simon/Downloads/anim.gif")
PHI = Vector(1:-0.005:0)
plt = plot(ylims=(-0.01,1.0), xlims=(-0.01,1.0), yticks=[0,1], xticks=[0,1], legend=:topright, size=(300,300), )
anim = @animate for i = 1:440
# for (j,κ) ∈ enumerate([1e-1, 5e-2, 1e-2])
for (j,κ) ∈ enumerate([5e-2,])
PHI_i = PHI[1:min(i,length(PHI))]
(i == 1) && plot!(plt, PHI_i, κ ./ PHI_i, color=light_blue, linewidth=1+1.5j, label="κ = $κ")
plot!(plt, PHI_i, κ ./ PHI_i, color=light_blue, linewidth=1+1.5j, label=nothing)
end
end
gif(anim, fps=40, "/home/simon/Downloads/force_solo.gif")
plot!(plt, PHI, 1e-2 ./ PHI, color=light_blue, linewidth=5.0, label="κ = 1e-2")
plot!(plt, PHI, 1e-3 ./ PHI, color=light_blue, linewidth=5.0, label="κ = 1e-3")
gradients
gradients
gradients
gradients
gradients
gradients
implicit-function theorem
Lezioni di analisi infinitesimale. U. Dini.
r(w^*; \theta) = 0
gradients
implicit-function theorem
Lezioni di analisi infinitesimale. U. Dini.
gradients
implicit-function theorem
Lezioni di analisi infinitesimale. U. Dini.
solution
gradients
implicit-function theorem
Lezioni di analisi infinitesimale. U. Dini.
solution
residual
gradients
implicit-function theorem
Lezioni di analisi infinitesimale. U. Dini.
solution
data
residual
trajectory optimization
Dojo: success
MuJoCo: success
Dojo: success
MuJoCo: failure
gradients
\alpha = \frac{\text{dynamics evaluation time}}{\text{gradient evaluation time}}
examples
trajectory optimization
optimized with iterative LQR
reinforcement learning
Subtitle
augmented random search to train static linear policies
reinforcement learning
augmented random search v. augmented gradient search
system identification
ContactNets: Learning Discontinuous Contact Dynamics with Smooth, Implicit Representations. S. Pfrommer, M. Halm, and M. Posa.
system identification
ContactNets: Learning Discontinuous Contact Dynamics with Smooth, Implicit Representations. S. Pfrommer, M. Halm, and M. Posa.
learned
ground-truth
system identification
learning via quasi-Newton method
geometry
friction coefficient
ground-truth
learned
system identification
system identification
Dojo gradient
live demo
model-predictive control
model-predictive control
model-predictive control
model-predictive control
NeRF
fast policy learning
github.com/dojo-sim
Dojo: A differentiable simulator for robotics
By taylorhowell
