• Welcome and opening remarks: Russ (<10 min).  [Note: Russ will present at 1pm]

  • GCS++

    • Intro remarks from Mathew Halm (5 min)

    • superfast iris, scs  -- Pete (10 min) slides

    • GCS planning on a manifold / bimanual -- Tommy (10 min) slides

    • global planning w/o iris regions -- Bernhard (10 min)  slides

    • solvers that exploit gcs structure -- Alex (15 min) slides

  • Real2Sim

    • scalable real2sim and steerable scene generation -- Russ (15 min), on behalf of Nicholas (who will be on an airplane).  slides.  prior work slides

  • Theory of Diffusion Policy / Visuomotor Control

Today: I'll share 3 storylines

2021

2025

Motion Planning (w/ Pablo Parrilo)

"Real2Sim" (w/ Phil Isola)

Theory of Visuomotor Control w/ Generative AI

(w/ Asu and Pablo)

A new approach to motion planning

Claims:

  • Find better plans faster than sampling-based planners
  • Avoid local minima from trajectory optimization
  • Can guarantee paths are collision-free
  • Naturally handles dynamic limits/constraints
  • Scales to big problems (e.g. multiple arms)
  • Important for Amazon: fast and consistent solve times

Graphs of convex sets (GCS) offers a new relaxation / modeling framework for joint discrete + continuous optimization

Traditional Shortest Path as a Linear Program (LP)

\(\varphi_{ij} = 1\) if the edge \((i,j)\) in shortest path, otherwise \(\varphi_{ij} = 0.\)

\(c_{ij} \) is the (constant) length of edge \((i,j).\)

\begin{aligned} \min_{\varphi} \quad & \sum_{(i,j) \in E} c_{ij} \varphi_{ij} \\ \mathrm{s.t.} \quad & \sum_{j \in E_i^{out}} \varphi_{ij} + \delta_{ti} = \sum_{j \in E_i^{in}} \varphi_{ji} + \delta_{si}, && \forall i \in V, \\ & \varphi_{ij} \in \{0, 1\}, && \forall (i,j) \in E. \end{aligned}

"flow constraints"

binary relaxation

path length

\begin{aligned} \min_{\varphi} \quad & \sum_{(i,j) \in E} c_{ij} \varphi_{ij} \\ \mathrm{s.t.} \quad & \sum_{j \in E_i^{out}} \varphi_{ij} + \delta_{ti} = \sum_{j \in E_i^{in}} \varphi_{ji} + \delta_{si}, && \forall i \in V, \\ & \varphi_{ij} \ge 0, && \forall (i,j) \in E. \end{aligned}

Graphs of Convex Sets

 

  • For each \(i \in V:\)
    • Compact convex set \(X_i \subset \R^d\)
    • A point \(x_i \in X_i \) 
  • Edge length given by a convex function \[ \ell(x_i, x_j) \]

Note: The blue regions are not obstacles.

GCS Trajectory Optimization

  • Welcome and opening remarks: Russ (<10 min).  [Note: Russ will present at 1pm]

  • GCS++

    • Intro remarks from Mathew Halm (5 min)

    • superfast iris, scs  -- Pete (10 min) slides

    • GCS planning on a manifold / bimanual -- Tommy (10 min) slides

    • global planning w/o iris regions -- Bernhard (10 min)  slides

    • solvers that exploit gcs structure -- Alex (15 min) slides

  • Real2Sim

    • scalable real2sim and steerable scene generation -- Russ (15 min), on behalf of Nicholas (who will be on an airplane).  slides.  prior work slides

  • Theory of Diffusion Policy / Visuomotor Control