"Diffusion" models

great tutorial: https://chenyang.co/diffusion.html

"Diffusion" models

another nice reference:

https://chenyang.co/diffusion.html

"Diffusion" models: deterministic interpretation

Image backbone: ResNet-18 (pretrained on ImageNet)
Total: 110M-150M Parameters
Training Time: 3-6 GPU Days ($150-$300)

Signs that things are working well...

• Often reactive (e.g. responds robustly to perturbations)
• Long-horizon tasks (multiple seconds, multiple steps)
• Discrete/branching logic.

a few more recent skills...

(we've now trained many hundreds)

We've switched to relative action parameterizations

interestingly, seems even more important in sim.

Standard training recipe

Some fairly typical training curves

Our standard training recipe (hardware eval)

• Run a fixed number of training steps (80k)
• Take the last checkpoint
  • occasionally (hardware) eval the last few checkpoints.

but now that we have sim eval that we trust...

Getting good data (at scale)

TRI's "CAM" data collect

Compared to many datasets, these are long-horizon (often 30 seconds - 1.5 minutes)

Often sequential/multistep

Getting good data ("batch dagger")

• Provide data that helps with robustness ("art, not magic")
  • Passive: arrangement, background, view, ...
  • Active: interactions, recovery, ...
• Initial data collect: ~100 demonstrations:
  • "If you fail... keep going!"
  • "Think about some likely failure cases, and demonstrate the recovery"
• Train the policy, test the rollouts
  • if we see obvious failures, collect more demonstrations showing recovery.
• Almost never more than 200 total demonstrations

Q: So how many demonstrations do I need to get to 99.99% success rate ($\Rightarrow$ a useful product)?

Things I want to understand better

Things that I'd like to understand better

• Why does val loss go up?
  • even val action mse goes up a little
• We're denoising past actions.  Why?
• Condition on previous actions (analogy to LQG)
• Transformer-based denoiser never worked as well as expected
• Faster inference
• ...
• then flow matching, equivariance, ... 

Enabling technologies

Haptic Teleop Interface

Excellent system identification / robot control

Visuotactile sensing

with TRI's Soft Bubble Gripper

Open source:

https://punyo.tech/

Predicting past actions (why?)

(that's almost what our code does, too)

nicely documented in lerobot 

but it's never really been tested for o > 2