(in the age of foundation models)
Russ Tedrake
MIT Robotics Seminar
Feb 16, 2024
Russ, How are we going to solve robotics?
e.g. to deal with "multi-modal demonstrations"
Learning categorial distributions already worked well (e.g. AlphaGo)
Diffusion helped extend this to high-dimensional continuous trajectories
but our planners are weak (esp. planning through contact)
Big data
Big transfer
Small data
No transfer
Ego-Exo
robot teleop
(the "transfer learning bet")
Open-X
simulation rollouts
Shortest Paths in Graphs of Convex Sets.
Tobia Marcucci, Jack Umenberger, Pablo Parrilo, Russ Tedrake.
Available at: https://arxiv.org/abs/2101.11565
Motion Planning around Obstacles with Convex Optimization.
Tobia Marcucci, Mark Petersen, David von Wrangel, Russ Tedrake.
Available at: https://arxiv.org/abs/2205.04422
start
goal
Two aspects of the motion planning problem:
The Probabilistic Roadmap (PRM)
from Choset, Howie M., et al. Principles of robot motion: theory, algorithms, and implementation. MIT press, 2005.
Default playback at .25x
The Probabilistic Roadmap (PRM)
from Choset, Howie M., et al. Principles of robot motion: theory, algorithms, and implementation. MIT press, 2005.
Note: The blue regions are not obstacles.
start
goal
is the convex relaxation. (it's tight!)
Previous formulations were intractable; would have required \( 6.25 \times 10^6\) binaries.
minimum distance
minimum time
Transcription to a mixed-integer convex program, but with a very tight and efficient convex relaxation.
+ time-rescaling
Preprocessor makes easy optimizations fast!
Transitioning from basic research to real use cases
Dave Johnson (CEO): "wow -- gcs (left) is a LOT better! ... This is a pretty special upgrade which is going to become the gold standard for motion planning."
Tommy Cohn (w/ Seiji Shaw, Shruti Garg)
Bernhard Paus Graesdal
Going beyond collision-free motion planning...
C. Chi et al., “Diffusion Policy: Visuomotor Policy Learning via Action Diffusion.” Mar. 09, 2023
N. Doshi et al., “Manipulation of unknown objects via contact configuration regulation.” Jun. 01, 2022
High-level approach
T. Marcucci, J. Umenberger, P. A. Parrilo, and R. Tedrake, “Shortest Paths in Graphs of Convex Sets.” 2022.
Formulation
or
\(\longrightarrow\)
\(\longrightarrow\)
\(\longrightarrow\)
\(\longrightarrow\)
\(\longrightarrow\)
\(\longrightarrow\)
\(\longrightarrow\)
Start
Goal
start
goal
Two aspects of the motion planning problem:
Lots of work on "contact graphs", but it's a little more subtle than that...
These algorithms are not arbitrary.
GCS helped us see the deeper connections between motion planning and structured optimization (SDP relaxations, moment hierarchies, etc).
Stronger optimization enables simpler cost functions.
(No cost-function tuning required)
(for planning under uncertainty and precomputing all paths)
What if:
So far:
GCS version (top down)
Prelimary results by Savva Morozov
Philosophy:
For moving boxes, the key combinatorial object is the permutahedron.
\[ \min_{x[\cdot],u[\cdot]} \sum_{n=0}^N x_n^T Q x_n + u_n^T Ru_n \\ \text{s.t. } x_{n+1} = Ax_n + Bu_n \]
Sets \( X_n: (x_n, u_n) \)
Edge cost
Edge constraint
n=0
n=1
n=2
n=N
\( \cdots \)
e.g. for hybrid trajectory optimization
n=0
n=1
n=N
...
\[ \min_{x[\cdot],u[\cdot]} \sum_{n=0}^N x_n^T Q_i x_n + u_n^T R_iu_n \\ \text{s.t. } x_{n+1} = A_ix_n + B_iu_n \\ \text{iff } (x_n,u_n) \in D_i \]
by Tobia Marcucci in collaboration w/ Stephen Boyd
This is version 0.1 of a new framework.
There is much more to do...
from A Survey of Monte Carlo Tree Search Methods by Browne et at, 2012
MCTS
Give it a try:
pip install drake
sudo apt install drake
...
give a talk using all of the terms foundation models, generative AI, spectrahedron, and permutahedron, because it makes sense.
...
Shortest Paths in Graphs of Convex Sets.
Tobia Marcucci, Jack Umenberger, Pablo Parrilo, Russ Tedrake.
Available at: https://arxiv.org/abs/2101.11565
Motion Planning around Obstacles with Convex Optimization.
Tobia Marcucci, Mark Petersen, David von Wrangel, Russ Tedrake.
Available at: https://arxiv.org/abs/2205.04422