Active Tactile Exploration
for Rigid Body State Estimation
Ethan K. Gordon, Bruke Baraki, Michael Posa
In Robotics, Models are difficult to build Online
-
Occlusions / Darkness
-
Clutter
-
Heterogeneous Materials
- Broken Objects
State of the art: Tactile Model Learning
[1] Hu et al. "Active shape reconstruction using a novel visuotactile palm sensor", Biomimetic Intelligence and Robotics 2024
[2] Xu et al. "TANDEM3D: Active Tactile Exploration for 3D Object Recognition", ICRA 2023
Static Objects: "assume a sensor that can detect contact before causing movement" [2]
Utilizes discrete object priors.
Shared Idea: Spatially Sparse Data -> Active Learning
Active Tactile Exploration: Problem Statement
?
r[t]
What we Choose: \(u\)
-
Robot Trajectory \(r[t]\)
What we learn: \(\theta\)
- Object State \(x[t]\)
-
Geometry, Inertia, Friction \(\theta\)
\lambda_m[t]
\hat{n}_m[t]
\Theta = (x[t], \theta)
What we MEASURE: \(m\)
-
Contact? \(c[t] \in \{0,1\}\)
-
Surface Normal \(\hat{n}_m[t]\)
-
Contact Force \(\lambda_m[t]\)
Information: How well do we know \(\Theta\)?
Learning: \(\tilde{\Theta} = \arg\min_\Theta \mathcal{L}^\Theta(u, m)\)
Noise Floor
\(\Theta\)
\(\mathcal{L}\)
\(\tilde{\Theta}\)
\(\Theta\)
\(\mathcal{L}\)
\(\tilde{\Theta}\)
\(\frac{\partial\mathcal{L}}{\partial \tilde{\Theta}}= 0 \)
How sensitive is this to uncertainty?
Low Info
High Info
(Fisher) Information \(\mathcal{I} := \nabla_\Theta^2 \mathcal{L}\)
Each Measurement Contributes to Information
\lambda_m[t]
\hat{n}_m[t]
c[t]
\frac{(\nabla_\Theta \phi)^2}{(1+e^\phi)^2}
\tilde{\Theta}
(\nabla_\Theta \hat{n})^2
\tilde{\Theta}
(\nabla_\Theta \lambda)^2
\tilde{\Theta}
\phi
Measurement:
\(\mathcal{I}\propto\)
Emergent Behavior:
"Get Close"
"Probe Corners"
(and edges if you don't know orientation)
"Push and Slide"
\(+\)
\(+\)
Preliminary Results
Green Sphere - Robot (simplified)
Red - Ground Truth
Blue - Our Guess
Preliminary Results
1. Force Action Z-Pinch
2. Learning....
3. Select New Action
| X-Pinch | Y-Pinch | Z-Pinch |
|---|---|---|
maximize Expected Info Gain
4. New Action: Y-Pinch
5. Learning...
| X-Pinch | Y-Pinch | Z-Pinch |
|---|---|---|
6. Select New Action
\sim 10^3
\sim 10^1
\sim 10^1
\sim 10^8
\sim 10^8
\sim 10^1
Green Sphere - Robot (simplified)
Red - Ground Truth
Blue - Our Guess
7. New Action: X-Pinch
Thank You!
Active Tactile Exploration for State Estimation (Showcase)
By Michael Posa
Active Tactile Exploration for State Estimation (Showcase)
June 12, 2025 Lab Showcase for Active Tactile Exploration Project
