Ambient-Omni: Locality
Oct 31, 2025
Adam Wei
Revisting Distribution Shifts
- Sim2real gaps
- Noisy/low-quality teleop
- Task-level mismatch
- Changes in low-level controller
- Embodiment gap
- Camera models, poses, etc
- Different environment, objects, etc
robot teleop
simulation
Open-X
Revisting Distribution Shifts
- Sim2real gaps
- Noisy/low-quality teleop
- Task-level mismatch
- Changes in low-level controller
- Embodiment gap
- Camera models, poses, etc
- Different environment, objects, etc
robot teleop
simulation
Open-X
Loss Function
Loss Function (for \(x_0\sim q_0\))
Denoising Loss vs Ambient Loss
Ambient Diffusion
Repeat:
- Sample \(\sigma\) ~ Unif([0,1])
- Sample (O, A, \(\sigma_{min}\)) ~ \(\mathcal{D}\) s.t. \(\sigma > \sigma_{min}\)
- Optimize denoising loss or ambient loss
\(\sigma=0\)
\(\sigma>\sigma_{min}\)
\(\sigma_{min}\)
*\(\sigma_{min} = 0\) for all clean samples
\(\sigma=1\)
Loss Function
Loss Function (for \(x_0\sim q_0\))
Denoising Loss vs Ambient Loss
Choosing \(\sigma_{min}\)
Task vs Motion Level
Ambient Diffusion: Most effective when corruption is motion level
\(\sigma=0\)
Bad Data
\(\sigma_{min}\)
\(\sigma=1\)
Good Data
Task level:
Task-level planning
Motion level:
Motion-level planning
Denoising Loss vs Ambient Loss
Ambient Omni: Locality
Ambient: "use low-quality data at high noise levels"
Ambient Omni: "use low-quality data at low and high noise levels"
- Leverages locality structure in many real world datasets
- Useful for robot data with "correct motions", but incorrect "logic"
Denoising Loss vs Ambient Loss
Locality
Which photo is the cat?
Denoising Loss vs Ambient Loss
Locality
Which photo is the cat?
Which photo is the cat?
Locality
Locality
Which photo is the cat?
Locality
Loss vs Receptive Field
Receptive Field vs \(\sigma\)
receptive field = \(f(\sigma)\)
Leveraging Locality
Intuition
- If \(p_\sigma \approx q_\sigma\) at receptive field \(f(\sigma)\), then \(q_\sigma\) can be used to learn \(p_\sigma\)
- The largest such \(\sigma\) is \(\sigma_{max}\)
receptive field = \(f(\sigma)\)
Ambient Omni
Repeat:
- Sample \(\sigma\) ~ Unif([0,1])
- Sample (O, A, \(\sigma_{min}, \sigma_{max}\)) ~ \(\mathcal{D}\) s.t. \(\sigma \in [0, \sigma_{max}) \cup (\sigma_{min}, 1]\)
- Optimize denoising loss or ambient loss
\(\sigma=0\)
\(\sigma>\sigma_{min}\)
\(\sigma_{min}\)
*\(\sigma_{min} = 0\) for all clean samples
\(\sigma=1\)
\(\sigma_{max}\)
\(\sigma>\sigma_{min}\)
\(\sigma_{max}\)
Implication for Robotics
\(\sigma=0\)
\(\sigma>\sigma_{min}\)
\(\sigma_{min}\)
\(\sigma=1\)
\(\sigma_{max}\)
\(\sigma>\sigma_{min}\)
\(\sigma_{max}\)
Task level
Motion level
- Data can be corrupt on the motion level and/or the task level
- Ambient handles motion level corruption
- Ambient Omni handles task level corruption as well
Example: Bin Sorting
Distribution shift: task level mismatch, motion level correctness
In-Distribution:Â
50 demos with correct sorting logic
Out-of-Distribution:Â
200 demos with arbitrary sorting
2x
2x
Example: Bin Sorting
Distribution shift: task level mismatch, motion level correctness
Contrived experiment... but it effectively illustrates the effect of \(\sigma_{max}\)
In-Distribution
Out-of-Distribution
2x
2x
Example: Bin Sorting
Contrived experiment... but it effectively illustrates the effect of \(\sigma_{max}\)
Repeat:
- Sample \(\sigma\) ~ Unif([0,1])
- Sample (O, A, \sigma_{max}\)) ~ \(\mathcal{D}\) s.t. \(\sigma \in [0, \sigma_{max})\)
- Optimize denoising loss
\(\sigma=0\)
\(\sigma=1\)
\(\sigma_{max}\)
\(\sigma>\sigma_{min}\)
\(\sigma_{max}\)
Results
Good Data Only
Score
(Task + motion)
Correct logic
(Task level)
Completed
(Motion level)
Ambient-Omni
(\(\sigma_{max}=0.46\))
61.01%
61.9%
98.6%
89.12%
92.8%
96.0%
Cotrain
(\(\alpha^*=0.9\))
60.8%
84.1%
72.3%
Results [OLD METRICS]
Good Data Only
Score
(Task + motion)
Correct logic
(Task level)
Cotrain
(\(\sigma_{max}=1\))
Completed
(Motion level)
Ambient-Omni
(\(\sigma_{max}=0.46\))
61.01%
44.5%
98.6%
56.98%
84.0%
60.4%
89.12%
82.0%
96.0%
Results
Good Data Only
Score
(Task + motion)
Correct logic
(Task level)
Cotrain
(\(\alpha^*=0.9\))
Completed
(Motion level)
Ambient-Omni
(\(\sigma_{max}=0.46\))
61.01%
61.9%
98.6%
60.8%
84.1%
72.3%
89.12%
92.8%
96.0%
Cotrain
(task-conditioned, \(\alpha^*=0.5\))
86.7%
88.5%
97.9%
Results [OLD METRICS]
Good Data Only
Score
(Task + motion)
Correct logic
(Task level)
Cotrain
(\(\sigma_{max}=1\))
Completed
(Motion level)
Ambient-Omni
(\(\sigma_{max}=0.46\))
61.01%
44.5%
98.6%
56.98%
84.0%
60.4%
89.12%
82.0%
96.0%
Cotrain
(task conditioned)
70.5%
82.0%
95.8%
Sweeping \(\sigma_{max}\): Overall Score
Motion Planning*
Task
Planning*
* not a binary distinction!!
Sweeping \(\sigma_{max}\): Overall Score
Sweeping \(\sigma_{max}\): Task-Level Metrics
Motion Planning
Task
Planning
Sweeping \(\sigma_{max}\): Motion-Level Metric
Motion Planning
Task
Planning
Sweeping \(\sigma_{max}\): Grasps/Block
Motion Planning
Task
Planning
Sweeping \(\sigma_{max}\): Time Per Grasp
Motion Planning
Task
Planning
Performance vs Num Good Data
Example: Bin Sorting
Distribution shift: task level mismatch, motion level correctness
In-Distribution
Out-of-Distribution
2x
2x
Open-X
2025/10/31 Pablo/Asu
By weiadam
