Learning Control for Dexterous Robotic Manipulation

Russ Tedrake

 

CSAIL Alliances Annual Meeting

May 25, 2023

Follow live at https://slides.com/d/mGiwCvo/live
(or later at https://slides.com/russtedrake/cap_2023)

A golden age for robotics

​"What's still hard for AI" by Kai-Fu Lee:

  1. AI cannot create, conceptualize, or manage complex strategic planning.

  2. AI cannot accomplish complex work that requires precise hand-eye coordination.

  3. AI cannot deal with unknown and unstructured spaces, especially ones that it hasn’t observed.

  4. AI cannot, unlike humans, feel or interact with empathy and compassion; therefore, it is unlikely that humans would opt for interacting with an apathetic robot for traditional communication services.

Kai-Fu's key axes of development:

  • Manual dexterity
  • Social intelligence (empathy/compassion)

Q: Is it a hardware problem?

http://personalrobotics.stanford.edu/

Key advance:

Visuomotor Policies

Levine*, Finn*, Darrel, Abbeel, JMLR 2016 

Visuomotor policies

perception network

(often pre-trained)

policy network

other robot sensors

learned state representation

actions

 

How do we synthesize visuomotor policies??

OpenAI - Learning Dexterity

  • Reinforcement Learning (RL)
  • What's hot: "Behavior Cloning" (e.g. from human demonstrations)

"And then … BC methods started to get good. Really good. So good that our best manipulation system today mostly uses BC ..."

Senior Director of Robotics at Google DeepMind

Diffusion (generative) models

Image source: Ho et al. 2020 

Learns a distribution (score function) over actions

e.g. to deal with "multi-modal demonstrations"

ours

prior art

Deals well with "multimodal" demonstrations

Advanced contact simulation

Simulating diversity

Real 2 Sim (example: Common Sense Machines)

Advanced motion planning and (visuomotor) control

Key ingredients

  1. The linear programming formulation of the shortest path problem on a discrete graph.
     
  2. Convex formulations of continuous motion planning (without obstacle navigation), for example:

     
  3. New Graphs of Convex Sets (GCS) machinery
     
  4. New approximate convex decompositions of configuration space

Kinematic Trajectory Optimization

(for robot arms)

Graphs of Convex Sets

 

  • For each \(i \in V:\)
    • Compact convex set \(X_i \subset \R^d\)
    • A point \(x_i \in X_i \) 
  • Edge length given by a convex function \[ \ell(x_i, x_j) \]

Note: The blue regions are not obstacles.

          is the convex relaxation.  (it's tight!)

Previous formulations were intractable; would have required \( 6.25 \times 10^6\) binaries.

Default playback at .25x

Summary

  • Dexterous manipulation is still unsolved, but progress is fast
  • Visuomotor diffusion policies
    • via Behavior Cloning
    • via advanced simulation + planning and control

 

  • Much of our code is open-source:

 

pip install drake
sudo apt install drake

Drake is "production ready"

  • Extremely-high code quality / test coverage
  • Monthly releases
  • 3 to 6 month deprecation timelines
  • Aggressive license tracking
  • ...

Already built in production build system at Amazon Robotics and many others.  

Online classes (videos + lecture notes + code)

http://manipulation.mit.edu

http://underactuated.mit.edu

Learning Control for Dexterous Robotic Manipulation

By russtedrake

Learning Control for Dexterous Robotic Manipulation

CMU RI Seminar

  • 702