What’s next for Vision-based Tactile Sensors?
Roberto Calandra
ViTac - 02 June 2023
Learning, Adaptive Systems, and Robotics (LASR) Lab
Benchmarking Touch
Roberto Calandra
ViTac - 02 June 2023
Learning, Adaptive Systems, and Robotics (LASR) Lab
Grand Vision
+ Applications
+ Community
Tactile Sensors in Robotics
[Wilson et al., 2019]
[Piacenza et al., 2020]
[ Fischel et al., 2012]
[Zhang et al., 2018]
[Church et al., 2019]
[Padmanabha et al., 2020]
The 3 Levels of Benchmarks
- Tasks
- Perceptual
-
Hardware
Benchmarks - Hardware
What are the properties of a given touch sensing hardware?
Problem: lack of standardization
- Spatial resolution
- Temporal resolution
- Multi-modality
- Robustness
- Form Factor
- Etc
Action Items:
- Build agreement on metrics and protocols for hardware
- Open-source design of benchmark evaluation setup
- Partner/establish 3rd party organization to performs evaluations
Benchmarks - Perceptual
What are the performance of a Sensor+Model to make sense of raw data?
- Material Classification
- Slip Detection
- Object reconstruction
- Any passive property that can be inferred from touch
Action Items:
- Build agreement on what are the important properties that we can measure with touch
- Build datasets/middleware for comparing models
(e.g., Skinware, PyTouch)
Benchmarks - Tasks
What are the performance of a Sensor+Controller to interact with the world?
- Grasping
- In-hand manipulation
- Assembling
- Any active task that can be benefit from touch
Action Items:
- Build agreement on what are the important tasks that we care about
- Design competitions in simulation and in real-world for comparing touch-based motor-control capabilities
(e.g., Tacto, Tactile Gym 2.0)
Why Benchmarks?
- Convince other communities/industry of the importance of touch
Outside of this room, people are not convinced that Touch is worth using! - Objectively measure progress
- Focus community towards important challenges
To Conclude
We are Hiring
- Post-docs
- PhD Students
- Robot Lab Manager
Overview
- To grow as a community and measure progress, we need to set goals in the form of benchmarks
-
3 Types of benchmarks:
- Hardware
- Perceptual
- Tasks
- We need to convincingly demonstrate the importance of touch to the wider robotic/industry community
Thank you!
Raw data
Models (passive)
Controllers (active)
Slack Channel
ViTac 2023
By Roberto Calandra
