Behavior Cloning

Part 2

MIT 6.421

Fall 2026, Lecture 9

Russ Tedrake

The Robot Data Diet

(the "transfer learning bet")

The main cast

• Teleop on your robot / your task
  • First-person >> third-person
• Cross-task/embodiment teleop data
• Egocentric (w/ tracking, w/o tracking)
  • ​With novel devices (e.g. UMI, tactile gloves, ...)
• Third-person video
   
• Simulation data

Egocentric data collection

Demonstration + Sim Augmentation

https://lujieyang.github.io/physicsgen/

Data normalization

Relative actions

https://umi-gripper.github.io/

Data labeling / annotation

Metadata example from \(\pi_{0.7}\)

https://ambient-diffusion-policy.github.io/

https://sim-and-real-cotraining.github.io/