russtedrake PRO
Roboticist at MIT and TRI
-
Welcome and opening remarks: Russ (<10 min). [Note: Russ will present at 1pm]
-
GCS++
-
Real2Sim
-
scalable real2sim and steerable scene generation -- Russ (15 min), on behalf of Nicholas (who will be on an airplane). slides. prior work slides
-
-
Theory of Diffusion Policy / Visuomotor Control
-
Intro remarks – Russ (< 5 min)
-
long-context policies -- Abhinav (10 min) slides
-
sim+real cotraining => ambient -- Adam (10 min) slides; sim+real prior work slides
- post-training w/ diffusion policy + trajopt -- Shao (10 min) slides
-
Today: I'll share 3 storylines
2021
2025
Motion Planning (w/ Pablo Parrilo)
"Real2Sim" (w/ Phil Isola)
Theory of Visuomotor Control w/ Generative AI
(w/ Asu and Pablo)
A new approach to motion planning
Claims:
- Find better plans faster than sampling-based planners
- Avoid local minima from trajectory optimization
- Can guarantee paths are collision-free
- Naturally handles dynamic limits/constraints
- Scales to big problems (e.g. multiple arms)
- Important for Amazon: fast and consistent solve times
Graphs of convex sets (GCS) offers a new relaxation / modeling framework for joint discrete + continuous optimization
Traditional Shortest Path as a Linear Program (LP)
\(\varphi_{ij} = 1\) if the edge \((i,j)\) in shortest path, otherwise \(\varphi_{ij} = 0.\)
\(c_{ij} \) is the (constant) length of edge \((i,j).\)
\begin{aligned} \min_{\varphi} \quad & \sum_{(i,j) \in E} c_{ij} \varphi_{ij} \\
\mathrm{s.t.} \quad
& \sum_{j \in E_i^{out}} \varphi_{ij} + \delta_{ti} = \sum_{j \in E_i^{in}} \varphi_{ji} + \delta_{si}, && \forall i \in V, \\
& \varphi_{ij} \in \{0, 1\}, && \forall (i,j) \in E.
\end{aligned}
"flow constraints"
binary relaxation
path length
\begin{aligned} \min_{\varphi} \quad & \sum_{(i,j) \in E} c_{ij} \varphi_{ij} \\
\mathrm{s.t.} \quad
& \sum_{j \in E_i^{out}} \varphi_{ij} + \delta_{ti} = \sum_{j \in E_i^{in}} \varphi_{ji} + \delta_{si}, && \forall i \in V, \\
& \varphi_{ij} \ge 0, && \forall (i,j) \in E.
\end{aligned}
Graphs of Convex Sets
-
For each \(i \in V:\)
- Compact convex set \(X_i \subset \R^d\)
- A point \(x_i \in X_i \)
- Edge length given by a convex function \[ \ell(x_i, x_j) \]
Note: The blue regions are not obstacles.
GCS Trajectory Optimization
-
Welcome and opening remarks: Russ (<10 min). [Note: Russ will present at 1pm]
-
GCS++
-
Real2Sim
-
scalable real2sim and steerable scene generation -- Russ (15 min), on behalf of Nicholas (who will be on an airplane). slides. prior work slides
-
-
Theory of Diffusion Policy / Visuomotor Control
-
Intro remarks – Russ (< 5 min)
-
long-context policies -- Abhinav (10 min) slides
-
sim+real cotraining => ambient -- Adam (10 min) slides; sim+real prior work slides
- post-training w/ diffusion policy + trajopt -- Shao (10 min) slides
-
2025 Amazon Robotics RLG talks
By russtedrake
