Ambient Diffusion Update

Dec 22, 2025

Adam Wei

Agenda

1. Quick update on experiments

2. How should we write the paper?

Distribution Shifts in Robot Data

• Sim2real gaps
• Noisy/low-quality teleop
• Task-level mismatch
• Changes in low-level controller

• Embodiment gap
• Camera models, poses, etc
• Different environment, objects, etc

Sim &"Real" Cotraining

In-Distribution: 

50 demos in "target" environment

Sim & Real Cotraining

Real Only

Sim + Real

(reweighted)

56.5%

84.5%

Perf

(200 trials)

Bin Sorting

In-Distribution: 

50 demos with correct sorting logic

Out-of-Distribution: 

200 demos with incorrect sorting

2x

2x

Bin Sorting

Repeat:

1. Sample \(\sigma\) ~ Unif([0,1])
2. Sample (O, A, \sigma_{max}\)) ~ \(\mathcal{D}\) s.t. \(\sigma \in [0, \sigma_{max})\)
3. Optimize denoising loss

\(\sigma=0\)

\(\sigma=1\)

\(\sigma_{max}\)

\(\sigma>\sigma_{min}\)

\(\sigma_{max}\)

Bin Sorting

Good Data Only

Score

(Task + motion)

Correct logic

(Task level)

Cotrain

(\(\alpha^*=0.9\))

Completed

(Motion level)

Ambient-Omni

(\(\sigma_{max}=0.46\))

Cotrain

(task-conditioned, \(\alpha^*=0.5\))

Bin Sorting

Motion Planning

Motion Planning

Loss Function

Loss Function (for \(x_0\sim q_0\))

Denoising Loss vs Ambient Loss

Choosing \(\sigma_{min}\)

Motion Planning Results

GCS

Success Rate

(Task-level)

Avg. Jerk^2

(Motion-level)

Policies evaluated over 100 trials each

Loss Function

Loss Function (for \(x_0\sim q_0\))

Denoising Loss vs Ambient Loss

Choosing \(\sigma_{min}\)

Motion Plannin: 7-DoF Robot Arms

Generate good (expensive) and bad (cheap) motion planning data in 20,000 environments

Evalute in new scences. Goal is to generate good trajectories

Loss Function

Loss Function (for \(x_0\sim q_0\))

Denoising Loss vs Ambient Loss

Choosing \(\sigma_{min}\)

Cross Embodied Data

Loss Function

Loss Function (for \(x_0\sim q_0\))

Denoising Loss vs Ambient Loss

Choosing \(\sigma_{min}\)

Cross Embodied Data

Immediate Next Direction

• Sim2real gaps
• Noisy/low-quality teleop
• Task-level mismatch
• Changes in low-level controller

• Embodiment gap
• Camera models, poses, etc
• Different environment, objects, etc

Immediate Next Direction

Variant of Ambient Omni

Cool Task!!

Open-X Embodiment

Magic Soup++: 27 Datasets

Custom OXE: 48 Datasets

• 1.4M episodes
• 55M "datagrams"

Open-X Embodiment

Table Cleaning

Ambient policy on OOD objects. (2x speed)

Table Cleaning: Evaluation

Task completion =

   0.1 x [opened drawer]

+ 0.8 x [# obj. cleaned / # obj.]

+ 0.1 x [closed drawer]

Research Goals/Questions

1. Does ambient help?
2. Does ambient-omni help?
3. Reweighting is critical in cotraining, is it also needed for ambient?

Ex. Reweights vs Unweighted

Reweighted

(sample clean 50%)

Unweighted

(sample clean 0.06%)

Ambient

Even the best policy right now is mediocre...

Next Steps

1. Finish \(\sigma_{max}\) experiments
2. Policies are poor:
   1. Collect more data (50 demos -> 150 demos)
   2. Explore DiT + stronger vision encoder
   3. Some small fixes ;)
3. Second hardware task using best practises

How To Structure Paper

Option 1:

• We use ambient diffusion in robotics
• The results are good, the method is general

Option 2:

• Co-training is important in robotics; we are doing it wrong.
• Ambient is one approach that works well and illustrates this problem