What’s next for Vision-based Tactile Sensors?

Roberto Calandra

ViTac - 02 June 2023

Learning, Adaptive Systems, and Robotics (LASR) Lab

Benchmarking Touch

Roberto Calandra

ViTac - 02 June 2023

Learning, Adaptive Systems, and Robotics (LASR) Lab

Grand Vision

+ Applications

+ Community

Tactile Sensors in Robotics

[Wilson et al., 2019]

[Piacenza et al., 2020]

[ Fischel et al., 2012]

[Zhang et al., 2018]

[Church et al., 2019]

[Padmanabha et al., 2020]

The 3 Levels of Benchmarks

  • Tasks
  • Perceptual
  • Hardware

Benchmarks - Hardware

What are the properties of a given touch sensing hardware?

Problem: lack of standardization

  • Spatial resolution
  • Temporal resolution
  • Multi-modality
  • Robustness
  • Form Factor
  • Etc

Action Items:

  1. Build agreement on metrics and protocols for hardware
  2. Open-source design of benchmark evaluation setup
  3. Partner/establish 3rd party organization to performs evaluations

Benchmarks - Perceptual

What are the performance of a Sensor+Model to make sense of raw data?

  • Material Classification
  • Slip Detection
  • Object reconstruction
  • Any passive property that can be inferred from touch

Action Items:

  1. Build agreement on what are the important properties that we can measure with touch
  2. Build datasets/middleware for comparing models
    (e.g., Skinware, PyTouch)

Benchmarks - Tasks

What are the performance of a Sensor+Controller to interact with the world?

  • Grasping
  • In-hand manipulation
  • Assembling
  • Any active task that can be benefit from touch

Action Items:

  1. Build agreement on what are the important tasks that we care about
  2. Design competitions in simulation and in real-world for comparing touch-based motor-control capabilities
    (e.g., Tacto, Tactile Gym 2.0)

Why Benchmarks?

  • Convince other communities/industry of the importance of touch
    Outside of this room, people are not convinced that Touch is worth using!
  • Objectively measure progress
  • Focus community towards important challenges

To Conclude

We are Hiring

  • Post-docs
  • PhD Students
  • Robot Lab Manager

Overview

  • To grow as a community and measure progress, we need to set goals in the form of benchmarks
  • 3 Types of benchmarks:
    • Hardware
    • Perceptual
    • Tasks
  • We need to convincingly demonstrate the importance of touch to the wider robotic/industry community

Thank you!

Raw data

Models (passive)

Controllers (active)

Slack Channel

ViTac 2023

By Roberto Calandra

ViTac 2023

  • 322