Lessons Learned from Designing and Evaluating a Robot-assisted Feeding System for Out-of-lab Use

Amal Nanavati, Ethan K. Gordon, Taylor Kessler Faulnker, Siddhartha Srinivasa, et al.

Work supported by: NSF (NRI; GRFP), DARPA, NIBIB, ONR, Amazon, and Cobot

ethan@ethankgordon.com

COMMUNITY-BASED PARTICIPATORY SYSTEM DESIGN

STUDY 1: MULTI-USER, IN-PUBLIC

LESSONS LEARNED AND FUTURE WORK

INTRODUCTION

If I can have a robot [working with me], it would be me feeding me, and that would be a huge deal.

-Tyler Schrenk, 1985-2023

Our goal is to develop an end-to-end robot feeding system that users can independently use to feed themselves meals of their choice outside the lab.

With CR1 and CR2, we designed a system following 5 guiding principles

• Portability. No internet required, powered from a wheelchair battery
• Safety. Many-layered: software watchdog, force sensor, accessible e-stops, etc.
• Customizability. App-based settings, retrofit to user's existing devices
• User Control. Behavior Tree (BT)-based task planning transparently communicates robot state for the user to direct and handles off-nominal fallback behavior

How does the system perform across different users in out-of-lab settings?

STUDY 2: SINGLE USER, 5-DAY, IN-HOME

Key Challenge: Off-Nominal Scenarios Will Arise

Key Insight: With customization and shared autonomy, users can independently overcome those scenarios.

Integrated Tech: face detection for transfer, online learning for novel food acquisition, cartesian end-effector control for intuitive movements.

How does the system perform across the diverse contexts that arise when eating in the home?

• Spatial Context. CR2 cannot sit up continuously, and so alternates between bed and wheelchair
• Social Context. CR2 has 3 caregivers who typically feed him
• Temporal Context. Mornings and daytime are busy with work, evenings are relaxed
• Activity Context. Deployment goals, e.g., eating while watching TV or working
• Food Context. CR2 is flexible; Ramen, pizza, chicken, fruits/vegetables, etc.

Therapist-assessed independence increased from "dependent" to "supervision" (Medicare Section GG).

Robot outperforms in sense of independence, is comparable in sense of control, without compromising sense of safety.

[it would help] others who can't use a self-feeding system like me - P3

Wide variability of ratings, from A+ (P4) to F (P3)

1. Variable autonomy helped with off-nominals, but customization reduced effort
2. Customization requires intuitive control over parameters and transparency into their downstream impacts.
3. Depending on level of impairment and other contexts, robot benefits outweigh cognitive workload even when autonomy fails.

Future Work

1. Bite Acquisition: expanded food variety and utensil types (this study only used a fork)
2. Bite Transfer: motion variety and in-mouth motions
3. User Comfort and Safety: compliant control
4. User Control and Customizability: user-directed debugging and planning scenes
5. Commercial Viability: Reduce system cost and co-design setup / maintenance procedures to integrate into existing care routines.