Ambient-Omni: Locality

Oct 31, 2025

Adam Wei

Revisting Distribution Shifts

  • Sim2real gaps
  • Noisy/low-quality teleop
  • Task-level mismatch
  • Changes in low-level controller
  • Embodiment gap
  • Camera models, poses, etc
  • Different environment, objects, etc

robot teleop

simulation

Open-X

Revisting Distribution Shifts

  • Sim2real gaps
  • Noisy/low-quality teleop
  • Task-level mismatch
  • Changes in low-level controller
  • Embodiment gap
  • Camera models, poses, etc
  • Different environment, objects, etc

robot teleop

simulation

Open-X

Loss Function

Loss Function (for \(x_0\sim q_0\))

Denoising Loss vs Ambient Loss

Ambient Diffusion

Repeat:

  1. Sample \(\sigma\) ~ Unif([0,1])
  2. Sample (O, A, \(\sigma_{min}\)) ~ \(\mathcal{D}\) s.t. \(\sigma > \sigma_{min}\)
  3. Optimize denoising loss or ambient loss

\(\sigma=0\)

\(\sigma>\sigma_{min}\)

\(\sigma_{min}\)

*\(\sigma_{min} = 0\) for all clean samples

\(\sigma=1\)

Loss Function

Loss Function (for \(x_0\sim q_0\))

Denoising Loss vs Ambient Loss

Choosing \(\sigma_{min}\)

Task vs Motion Level

Ambient Diffusion: Most effective when corruption is motion level

\(\sigma=0\)

Bad Data

\(\sigma_{min}\)

\(\sigma=1\)

Good Data

Task level:

Task-level planning

Motion level:

Motion-level planning

Denoising Loss vs Ambient Loss

Ambient Omni: Locality

Ambient: "use low-quality data at high noise levels"

Ambient Omni: "use low-quality data at low and high noise levels"

  • Leverages locality structure in many real world datasets
  • Useful for robot data with "correct motions", but incorrect "logic"

Denoising Loss vs Ambient Loss

Locality

Which photo is the cat?

Denoising Loss vs Ambient Loss

Locality

Which photo is the cat?

Which photo is the cat?

Locality

Locality

Which photo is the cat?

Locality

Loss vs Receptive Field

Receptive Field vs \(\sigma\)

receptive field = \(f(\sigma)\)

Leveraging Locality

Intuition

  • If \(p_\sigma \approx q_\sigma\) at receptive field \(f(\sigma)\), then \(q_\sigma\) can be used to learn \(p_\sigma\)
  • The largest such \(\sigma\) is \(\sigma_{max}\)

receptive field = \(f(\sigma)\)

Ambient Omni

Repeat:

  1. Sample \(\sigma\) ~ Unif([0,1])
  2. Sample (O, A, \(\sigma_{min}, \sigma_{max}\)) ~ \(\mathcal{D}\) s.t. \(\sigma \in [0, \sigma_{max}) \cup (\sigma_{min}, 1]\)
  3. Optimize denoising loss or ambient loss

\(\sigma=0\)

\(\sigma>\sigma_{min}\)

\(\sigma_{min}\)

*\(\sigma_{min} = 0\) for all clean samples

\(\sigma=1\)

\(\sigma_{max}\)

\(\sigma>\sigma_{min}\)

\(\sigma_{max}\)

Implication for Robotics

\(\sigma=0\)

\(\sigma>\sigma_{min}\)

\(\sigma_{min}\)

\(\sigma=1\)

\(\sigma_{max}\)

\(\sigma>\sigma_{min}\)

\(\sigma_{max}\)

Task level

Motion level

  • Data can be corrupt on the motion level and/or the task level
  • Ambient handles motion level corruption
  • Ambient Omni handles task level corruption as well

Example: Bin Sorting

Distribution shift: task level mismatch, motion level correctness

In-Distribution: 

50 demos with correct sorting logic

Out-of-Distribution: 

200 demos with arbitrary sorting

2x

2x

Example: Bin Sorting

Distribution shift: task level mismatch, motion level correctness

Contrived experiment... but it effectively illustrates the effect of \(\sigma_{max}\)

In-Distribution

Out-of-Distribution

2x

2x

Example: Bin Sorting

Contrived experiment... but it effectively illustrates the effect of \(\sigma_{max}\)

Repeat:

  1. Sample \(\sigma\) ~ Unif([0,1])
  2. Sample (O, A, \sigma_{max}\)) ~ \(\mathcal{D}\) s.t. \(\sigma \in [0, \sigma_{max})\)
  3. Optimize denoising loss

\(\sigma=0\)

\(\sigma=1\)

\(\sigma_{max}\)

\(\sigma>\sigma_{min}\)

\(\sigma_{max}\)

Results

Good Data Only

Score

(Task + motion)

Correct logic

(Task level)

Completed

(Motion level)

Ambient-Omni

(\(\sigma_{max}=0.46\))

61.01%

61.9%

98.6%

89.12%

92.8%

96.0%

Cotrain

(\(\alpha^*=0.9\))

60.8%

84.1%

72.3%

Results [OLD METRICS]

Good Data Only

Score

(Task + motion)

Correct logic

(Task level)

Cotrain

(\(\sigma_{max}=1\))

Completed

(Motion level)

Ambient-Omni

(\(\sigma_{max}=0.46\))

61.01%

44.5%

98.6%

56.98%

84.0%

60.4%

89.12%

82.0%

96.0%

Results

Good Data Only

Score

(Task + motion)

Correct logic

(Task level)

Cotrain

(\(\alpha^*=0.9\))

Completed

(Motion level)

Ambient-Omni

(\(\sigma_{max}=0.46\))

61.01%

61.9%

98.6%

60.8%

84.1%

72.3%

89.12%

92.8%

96.0%

Cotrain

(task-conditioned, \(\alpha^*=0.5\))

86.7%

88.5%

97.9%

Results [OLD METRICS]

Good Data Only

Score

(Task + motion)

Correct logic

(Task level)

Cotrain

(\(\sigma_{max}=1\))

Completed

(Motion level)

Ambient-Omni

(\(\sigma_{max}=0.46\))

61.01%

44.5%

98.6%

56.98%

84.0%

60.4%

89.12%

82.0%

96.0%

Cotrain

(task conditioned)

70.5%

82.0%

95.8%

Sweeping \(\sigma_{max}\): Overall Score

Motion Planning*

Task

Planning*

* not a binary distinction!!

Sweeping \(\sigma_{max}\): Overall Score

Sweeping \(\sigma_{max}\): Task-Level Metrics

Motion Planning

Task

Planning

Sweeping \(\sigma_{max}\): Motion-Level Metric

Motion Planning

Task

Planning

Sweeping \(\sigma_{max}\): Grasps/Block

Motion Planning

Task

Planning

Sweeping \(\sigma_{max}\): Time Per Grasp

Motion Planning

Task

Planning

Performance vs Num Good Data

Example: Bin Sorting

Distribution shift: task level mismatch, motion level correctness

In-Distribution

Out-of-Distribution

2x

2x

Open-X