Overview of Robot Learning

for Manipulation

MIT 6.421

Fall 2026, Lecture 2

Russ Tedrake

Intro / Motivation

Walking into the building just before lecture 1 last week, I overheard:

The post(s) that caused the buzz

Not just direct robot control... also "real2sim", ...

Robots are just one more input/output modality...

omni model

text, images,

video, audio, ...

robot sensors

text, images,

video, audio, ...

robot actions

Did Astra "solve" robotics?

• Huge progress towards omni models trained on robot data

• I don't see "GPT learned how to watch youtube and write code, and got zero-shot robot performance for free"
  
  
• I see "Robotics is important enough that OpenAI (and Anthropic and ...) are pointing tons of robot people/data at their models, too"

[meta] Why and how to study robotics in 2026++

• The potential for impact is almost limitless
• The state-of-the-art is moving super fast (in every field)
• Hardware can't scale (quite) as fast as software
  • Low-pass filter on recursive self improvement
  • (but exponentials can still happen... e.g. robots building robots)
• Expert coders and vibe coders
  • How do you become an expert in 2026++?
  • Have to take the time to understand the details
  • Research labs and companies don't need many vibe coders

Goal for today's "overview of robot learning"

• Broad taxonomy of robot learning ideas
   
• Context for what you're seeing on social media
• Context for the lectures in this course
   
• We'll do deep dives on a handful of these topics
   
• Please make requests and ask questions

How do you formulate manipulation as a learning problem?

"Single-task" learning

"Single-task" learning

Approach #1: Reinforcement Learning (RL)

as you've seen in, e.g. MIT 6.390 (https://introml.mit.edu/)

image from old result:

OpenAI - Learning Dexterity

1. Make a simulator
2. Write reward function
3. Deep policy gradient
4. Deploy on hardware

Should you use this RL pipeline?

• Surprisingly effective for locomotion
• Limited results for manipulation
   
• Requires a simulator
  • Question for you: why?
• Requires reward function tuning (hacking)
  • Question for you: why?

Approach #2: Behavior Cloning (BC)

1. Collect demonstration data
   • e.g. teleoperation on real hardware ~200 times

note: BC is one type of imitation learning

Approach #2: Behavior Cloning (BC)

1. Collect demonstration data
   • e.g. teleoperation on real hardware ~200 times
      
2. Train with simple supervised-learning objective
   • ~next-token prediction
      
3. Deploy

note: BC is one type of imitation learning

Combined Approach: Imitation-guided RL

• For systems that you can simulate, can combine the two... 
  • e.g. single demonstration then RL in sim
     
• Potential virtues:
  • less data
  • more robust
  • superhuman performance

Discussion

Why is RL dominant for locomotion, but imitation learning more dominant for manipulation?

(And how will that evolve over the coming years...?)

Multitask learning

Multitask learning

1. Start with a large pretrained model (e.g. VLM or world model)
2. "midtrain" with robot data
   • BC objective
   • teaching the model one more output modality
3. often: task- or domain-specific post-training / fine-tuning
   • BC or RL or both

"Single-task" learning

• ~200 demonstrations
• \(\Rightarrow\) visuomotor policy

A path to dexterity & generality?

dexterity

generality

Architectures

Blind locomotion-style policies

Often very simple, e.g. 3 layer, 255 unit, multilayer perceptrons (MLPs)

Levine*, Finn*, Darrel, Abbeel, JMLR 2016 

Visuomotor policies

perception network

(often pre-trained)

other robot sensors

learned state representation

x history

VLAs, WAMs, ...

omni model

text, images,

video, audio, ...

robot sensors

text, images,

video, audio, ...

robot actions

Generalist Gen1+ forgoes the pretrained model

Thinking fast and slow

example: GR00T N1 architecture

Data Sources

The Robot Data Diet

(the "transfer learning bet")

The main cast

• Teleop on your robot / your task
  • First-person >> third-person
• Cross-task/embodiment robot data
• Egocentric (w/ tracking, w/o tracking)
  • ​With novel devices (e.g. UMI, tactile gloves, ...)
• Third-person video
   
• Simulation data

Midtraining data is being commoditized

Two separate roles for imitation data

1. Task-level knowledge
2. Dexterity-level knowledge

Evaluation

Is checkpoint A better than checkpoint B?

• "Violin plots" from Bayesian analysis assuming a uniform prior
  • "compact letter display" for separation

Rigorous evaluation

• Real-world hardware testing
  • A/B testing. Always blind, randomized trials.
  • Rich reporting (not just "pass/fail").
• Extensive simulation-based testing
  • Many more experiments, and they are repeatable
  • Many are skeptical of simulation; our investment has paid off

Multitask Scaling Laws

from the LBM 1.0 paper

more data from this task

more data from other tasks

fine-tuning

Failure analyses for our hardest (real) skills

ClearKitchenCounter

Single task

LBM finetuned

Advanced capabilities

Takeaways

Takeaways

• Imitation-guided RL for locomotion
• BC for manipulation, with RL coming fast
• Multitask BC for teaching super powerful omni models one more output modality
   
• Will dive more into architecture, evaluation, and some expanding topics in the main content lectures
   
• But first, we'll build some mastery of the lower-level components in the stack