Active Tactile Exploration

for Rigid Body State Estimation

Ethan K. Gordon, Bruke Baraki, Michael Posa

In Robotics, Models are difficult to build Online

  • Occlusions / Darkness

  • Clutter

  • Heterogeneous Materials

  • Broken Objects

State of the art: Tactile Model Learning

[1] Hu et al. "Active shape reconstruction using a novel visuotactile palm sensor", Biomimetic Intelligence and Robotics 2024

[2] Xu et al. "TANDEM3D: Active Tactile Exploration for 3D Object Recognition", ICRA 2023

Static Objects: "assume a sensor that can detect contact before causing movement" [2]

Utilizes discrete object priors.

Shared Idea: Spatially Sparse Data -> Active Learning

Active Tactile Exploration: Problem Statement

?

r[t]

What we Choose: \(u\)

  • Robot Trajectory \(r[t]\)

What we learn: \(\theta\)

  • Object State \(x[t]\)

  • Geometry, Inertia, Friction \(\theta\)

\lambda_m[t]
\hat{n}_m[t]
\Theta = (x[t], \theta)

What we MEASURE: \(m\)

  • Contact? \(c[t] \in \{0,1\}\)

  • Surface Normal \(\hat{n}_m[t]\)

  • Contact Force \(\lambda_m[t]\)

Information: How well do we know \(\Theta\)?

Learning:         \(\tilde{\Theta} = \arg\min_\Theta \mathcal{L}^\Theta(u, m)\)

Noise Floor

\(\Theta\)

\(\mathcal{L}\)

\(\tilde{\Theta}\)

\(\Theta\)

\(\mathcal{L}\)

\(\tilde{\Theta}\)

\(\frac{\partial\mathcal{L}}{\partial \tilde{\Theta}}= 0 \)

How sensitive is this to uncertainty?

Low Info

High Info

(Fisher) Information       \(\mathcal{I} := \nabla_\Theta^2 \mathcal{L}\)

Each Measurement Contributes to Information

\lambda_m[t]
\hat{n}_m[t]
c[t]
\frac{(\nabla_\Theta \phi)^2}{(1+e^\phi)^2}
\tilde{\Theta}
(\nabla_\Theta \hat{n})^2
\tilde{\Theta}
(\nabla_\Theta \lambda)^2
\tilde{\Theta}
\phi

Measurement:

\(\mathcal{I}\propto\)

Emergent Behavior:

"Get Close"

"Probe Corners"

(and edges if you don't know orientation)

"Push and Slide"

\(+\)

\(+\)

Preliminary Results

Green Sphere - Robot (simplified)

Red - Ground Truth

Blue - Our Guess

Preliminary Results

1. Force Action Z-Pinch

2. Learning....

3. Select New Action

X-Pinch Y-Pinch Z-Pinch

maximize Expected Info Gain

4. New Action: Y-Pinch

5. Learning...

X-Pinch Y-Pinch Z-Pinch

6. Select New Action

\sim 10^3
\sim 10^1
\sim 10^1
\sim 10^8
\sim 10^8
\sim 10^1

Green Sphere - Robot (simplified)

Red - Ground Truth

Blue - Our Guess

7. New Action: X-Pinch

Thank You!