Learning Control for Dexterous Robotic Manipulation

Russ Tedrake

CSAIL's 60th

June 27, 2023

A golden age for robotics

https://code-as-policies.github.io/

Key advance:

Visuomotor Policies

Levine*, Finn*, Darrel, Abbeel, JMLR 2016 

Visuomotor policies

perception network

(often pre-trained)

other robot sensors

learned state representation

How do we train visuomotor policies??

OpenAI - Learning Dexterity

• Reinforcement Learning (RL)

• What's hot: "Behavior Cloning" (e.g. from human demonstrations)

"And then … BC methods started to get good. Really good. So good that our best manipulation system today mostly uses BC ..."

Senior Director of Robotics at Google DeepMind

Diffusion models (generative AI)

Image source: Ho et al. 2020 

Learns a distribution (score function) over actions

e.g. to deal with "multi-modal demonstrations"

ours

prior art

• Great for one skill
• Wanted: few shot generalization to new skills

• Big Questions:
  • How do we feed the data flywheel?
  • What are the scaling laws?

Advanced contact simulation

http://drake.mit.edu

Advanced motion planning and (visuomotor) control

Graphs of Convex Sets

Summary

• Dexterous manipulation is still unsolved, but progress is fast
• Visuomotor diffusion policies
  • via imitation learning from humans
  • via advanced simulation + planning and control

• Much of our code is open-source:

drake.mit.edu

Online classes (videos + lecture notes + code)