Towards a Science of
Touch Processing

Roberto Calandra

Facebook AI Research

Workshop on Visuo-tactile Sensors for Robust Manipulation - 13 July 2020

The Importance of Touch

What do we need to make Touch Sensing ubiquitous in Robotics?

Touch as a Commodity

Audio

Touch

Vision

(~1890)

(~1990)

(2020s ?)

Making Touch Sensing Ubiquitous

Hardware Requirements

  • Readily available (e.g., mass-produced)
  • Inexpensive
  • Reliable
  • Form factor suitable for the tasks of interests
  • Rich in measurements w.r.t. the tasks of interests (e.g., high-resolution)
  • Open-source

DIGIT

Lambeta, M.; Chou, P.-W.; Tian, S.; Yang, B.; Maloon, B.; Most, V. R.; Stroud, D.; Santos, R.; Byagowi, A.; Kammerer, G.; Jayaraman, D. & Calandra, R.
DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
IEEE Robotics and Automation Letters (RA-L), 2020, 5, 3838-3845

Examples of DIGIT Measurements

Lambeta, M.; Chou, P.-W.; Tian, S.; Yang, B.; Maloon, B.; Most, V. R.; Stroud, D.; Santos, R.; Byagowi, A.; Kammerer, G.; Jayaraman, D. & Calandra, R.
DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
IEEE Robotics and Automation Letters (RA-L), 2020, 5, 3838-3845

Comparison

BioTac

DIGIT

~10,000 $

Cost

~15 $

Resolution

29
contact points

307,200
contact points

Mounted on multi-finger hands

Open-sourced

https://digit.ml

Marble Manipulation Trajectories

Making Touch Sensing Ubiquitous

Simulators Requirement

  • Fast, fast, fast... (>100Hz)
  • Open-source
  • General purpose
    (in the same way that a physics engine can simulate any rigid body robot)
  • Flexible
  • Accurate
    • In terms of output
      (Realistic rendering calibrated on real sensor)
    • In terms of physics
      (Not important for perceptual tasks. Very difficult to model non-rigid contacts accurately)

Making Touch Sensing Ubiquitous

Touch Processing Requirements

  • Standardized tools for processing touch readings
  • Touch Processing as an easy-to-use service for all robots
    (Just connect a tactile sensor and obtain high-level features useful for control)
  • Invariant to variation between sensors
  • (Likely to be Machine Learning based)
  • Open-source

Towards a Science of Touch Processing

  • What are good features for touch?
  • Do we need sensor standardization?
    • In Image Processing, images are represented as RGB [HxWx3] matrices
    • What representation do we want/need for touch?
    • What sensors do we even want/need for touch?
  • What are the useful structures in computational models for touch?
  • What are the useful metrics to characterize touch?
  • How can we quantify the human psychophysics of touch?
  • What are the different tasks that can benefit from touch?
  • What are meaningful benchmarks for touch processing?

Making Touch Sensing Ubiquitous

Benchmarks & Datasets Requirements

Benchmarks:

 

Datasets:

 
  • What are the tasks that we care about as a community?
  • The first great enabler of benchmarks will be accurate touch simulators
  • The second great enabler of benchmarks will be easily available hardware
  • Very few touch sensing datasets available nowadays
  • Data collection is currently limited by the reliability of hardware
  • Simulators, and available hardware will enable new and larger datasets
    (But we also need as a community to encourage and nurture this)

Making Touch Sensing Ubiquitous

Human Collaborators

Lambeta, M.; Chou, P.-W.; Tian, S.; Yang, B.; Maloon, B.; Most, V. R.; Stroud, D.; Santos, R.; Byagowi, A.; Kammerer, G.; Jayaraman, D. & Calandra, R.
DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
IEEE Robotics and Automation Letters (RA-L), 2020, 5, 3838-3845

Summary

  • Touch is a key sensor modality for robotics
  • With increased availability of hardware, the community will now need:
    • Simulators
    • Benchmarks and Datasets
    • Software to process touch readings
  • A science of Touch Processing will be crucial to standardize, understand and process tactile readings...
  • ...and ultimately make touch ubiquitous in robots

Towards Touch Processing as the field that studies Touch

References

  • Yuan, W.; Dong, S. & Adelson, E. H.
    GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force
    Sensors, 2017
  • Calandra, R.; Owens, A.; Jayaraman, D.; Yuan, W.; Lin, J.; Malik, J.; Adelson, E. H. & Levine, S.
    More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch
    IEEE Robotics and Automation Letters (RA-L), 2018, 3, 3300-3307
  • Allen, P. K.; Miller, A. T.; Oh, P. Y. & Leibowitz, B. S.
    Integration of vision, force and tactile sensing for grasping
    Int. J. Intelligent Machines, 1999, 4, 129-149
  • Chebotar, Y.; Hausman, K.; Su, Z.; Sukhatme, G. S. & Schaal, S.
    Self-supervised regrasping using spatio-temporal tactile features and reinforcement learning
    International Conference on Intelligent Robots and Systems (IROS), 2016
  • Schill, J.; Laaksonen, J.; Przybylski, M.; Kyrki, V.; Asfour, T. & Dillmann, R.
    Learning continuous grasp stability for a humanoid robot hand based on tactile sensing
    BioRob, 2012
  • Bekiroglu, Y.; Laaksonen, J.; Jorgensen, J. A.; Kyrki, V. & Kragic, D.
    Assessing grasp stability based on learning and haptic data
    Transactions on Robotics, 2011, 27
  • Sommer, N. & Billard, A.
    Multi-contact haptic exploration and grasping with tactile sensors
    Robotics and autonomous systems, 2016, 85, 48-61

From the lab of Dr. Ronald Johansson, Dept. of Physiology, University of Umea, Sweden

The Importance of Touch (in Humans)

Model-based Reinforcement Learning

Trajectories

Towards a Science of Touch Processing

By Roberto Calandra

Towards a Science of Touch Processing

[RSS Workshop on Visuo-tactile Sensors for Robust Manipulation: From Perception to Control]

  • 64
Loading comments...

More from Roberto Calandra