Roberto Calandra
UC Berkeley - CS280 - 29 Apr 2021
Facebook AI Research
From the lab of Dr. Ronald Johansson, Dept. of Physiology, University of Umea, Sweden
[Allen et al. 1999]
[Chebotar et al. 2016]
[Bekiroglu et al. 2011]
[Sommer and Billard 2016]
[Schill et al. 2012]
(~1890)
(~1990)
(2020s ?)
What do we need to make Touch Sensing practical and useful?
Important factors:
Many many sensors in the literature:
[Wilson et al., 2019]
[Piacenza et al., 2020]
[ Fischel et al., 2012]
[Zhang et al., 2018]
[Church et al., 2019]
Cannata, G.; Maggiali, M.; Metta, G. & Sandini, G.
An embedded artificial skin for humanoid robots
IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), 2008, 434-438
[Kamiyama, K.; Kajimoto, H.; Kawakami, N. & Tachi, S. Evaluation of a vision-based tactile sensor IEEE International Conference on Robotics and Automation (ICRA), 2004, 2, 1542-1547 ]
[Johnson, M. K. & Adelson, E. H. Retrographic sensing for the measurement of surface texture and shape Computer Vision and Pattern Recognition (CVPR), 2009, 1070-1077]
[Abad, A. C. & Ranasinghe, A. Visuotactile Sensors With Emphasis on GelSight Sensor: A Review IEEE Sensors Journal, 2020, 20, 7628-7638]
Credit:
[Yuan, W.; Dong, S. & Adelson, E. H. GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force Sensors, Multidisciplinary Digital Publishing Institute, 2017]
Lambeta, M.; Chou, P.-W.; Tian, S.; Yang, B.; Maloon, B.; Most, V. R.; Stroud, D.; Santos, R.; Byagowi, A.; Kammerer, G.; Jayaraman, D. & Calandra, R.
DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
IEEE Robotics and Automation Letters (RA-L), 2020, 5, 3838-3845
Lambeta, M.; Chou, P.-W.; Tian, S.; Yang, B.; Maloon, B.; Most, V. R.; Stroud, D.; Santos, R.; Byagowi, A.; Kammerer, G.; Jayaraman, D. & Calandra, R.
DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
IEEE Robotics and Automation Letters (RA-L), 2020, 5, 3838-3845
BioTac
DIGIT
~10,000 $
Cost
~15 $
Resolution
29
contact points
307,200
contact points
Mounted on multi-finger hands
Open-source
Being replicated in 15+ universities
1000x
Higher resolution
1000x
Cheaper
Lambeta, M.; Chou, P.-W.; Tian, S.; Yang, B.; Maloon, B.; Most, V. R.; Stroud, D.; Santos, R.; Byagowi, A.; Kammerer, G.; Jayaraman, D. & Calandra, R.
DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
IEEE Robotics and Automation Letters (RA-L), 2020, 5, 3838-3845
Lambeta, M.; Chou, P.-W.; Tian, S.; Yang, B.; Maloon, B.; Most, V. R.; Stroud, D.; Santos, R.; Byagowi, A.; Kammerer, G.; Jayaraman, D. & Calandra, R.
DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
IEEE Robotics and Automation Letters (RA-L), 2020, 5, 3838-3845
Reflective
Reflective
+
Markers
Transparent
+
Markers
Lambeta, M.; Chou, P.-W.; Tian, S.; Yang, B.; Maloon, B.; Most, V. R.; Stroud, D.; Santos, R.; Byagowi, A.; Kammerer, G.; Jayaraman, D. & Calandra, R.
DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
IEEE Robotics and Automation Letters (RA-L), 2020, 5, 3838-3845
Simulators play a crucial role in prototyping, debugging and benchmarking new advances in robotics
Wang, S.; Lambeta, M.; Chou, P.-W. & Calandra, R.
TACTO: A Fast, Flexible and Open-source Simulator for High-Resolution Vision-based Tactile Sensors
Under Review, 2020, Online: https://arxiv.org/abs/2012.08456
Wang, S.; Lambeta, M.; Chou, P.-W. & Calandra, R.
TACTO: A Fast, Flexible and Open-source Simulator for High-Resolution Vision-based Tactile Sensors
Under Review, 2020, Online: https://arxiv.org/abs/2012.08456
Wang, S.; Lambeta, M.; Chou, P.-W. & Calandra, R.
TACTO: A Fast, Flexible and Open-source Simulator for High-Resolution Vision-based Tactile Sensors
Under Review, 2020, Online: https://arxiv.org/abs/2012.08456
Predicting Good Grasps
Learning how to Grasp
Active Tactile Exploration
3D Reconstruction from Vision and Touch
Identify Objects from Touch
Calandra, R.; Owens, A.; Upadhyaya, M.; Yuan, W.; Lin, J.; Adelson, E. H. & Levine, S.
The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes?
Conference on Robot Learning (CORL), 2017, 314-323
Collected 6450 grasps from over 60 training objects over ~2 weeks.
Calandra, R.; Owens, A.; Jayaraman, D.; Yuan, W.; Lin, J.; Malik, J.; Adelson, E. H. & Levine, S.
More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch
IEEE Robotics and Automation Letters (RA-L), 2018, 3, 3300-3307
83.8% grasp success on 22 unseen objects
(using only vision yields 56.6% success rate)
Calandra, R.; Owens, A.; Jayaraman, D.; Yuan, W.; Lin, J.; Malik, J.; Adelson, E. H. & Levine, S.
More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch
IEEE Robotics and Automation Letters (RA-L), 2018, 3, 3300-3307
Fritsche, L.; Unverzagt, F.; Peters, J. & Calandra, R.
First-Person Tele-Operation of a Humanoid Robot
IEEE-RAS International Conference on Humanoid Robots (HUMANOIDS), 2015, 997-1002
Lambeta, M.; Chou, P.-W.; Tian, S.; Yang, B.; Maloon, B.; Most, V. R.; Stroud, D.; Santos, R.; Byagowi, A.; Kammerer, G.; Jayaraman, D. & Calandra, R.
DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
IEEE Robotics and Automation Letters (RA-L), 2020, 5, 3838-3845
Lambeta, M.; Chou, P.-W.; Tian, S.; Yang, B.; Maloon, B.; Most, V. R.; Stroud, D.; Santos, R.; Byagowi, A.; Kammerer, G.; Jayaraman, D. & Calandra, R.
DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
IEEE Robotics and Automation Letters (RA-L), 2020, 5, 3838-3845
Lambeta, M.; Chou, P.-W.; Tian, S.; Yang, B.; Maloon, B.; Most, V. R.; Stroud, D.; Santos, R.; Byagowi, A.; Kammerer, G.; Jayaraman, D. & Calandra, R.
DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
IEEE Robotics and Automation Letters (RA-L), 2020, 5, 3838-3845
Feedback at: https://tinyurl.com/cs280-touch
Wang, S.; Lambeta, M.; Chou, L. & Calandra, R.
TACTO: A Fast, Flexible and Open-source Simulator for High-Resolution Vision-based Tactile Sensors
Under Review, 2020, Online: https://arxiv.org/abs/2012.08456
How to scale to more complex, unstructured domains?
Robotics
Finance
Biological Sciences
Logistics /
Decision Making
Disaster Relief
Industrial Automation
Exploration
Medicine & Eldercare
From YouTube: https://www.youtube.com/watch?v=g0TaYhjpOfo