Empowering Physically Assistive Robots with Multimodal Active Learning

Ethan K. Gordon

PhD 2023, University of Washington; Postdoc, University of Pennsylvania

Work supported by: NSF (IIS; DMS; NRI; CHS; GRFP; FRR), NIH, Amazon, and RAI

ethan@ethankgordon.com

PREVIOUS WORK: ROBOT-ASSISTED FEEDING

Computing \(g=\nabla \log p(m_t | x_T)\)

ONGOING WORK: ACTIVE TACTILE EXPLORATION

Next Step

Translate to contact-rich physical activities of daily living (ADLs) with visual occlusions.

Dressing, Bathing, etc.

More Info:

ethankgordon.com

FUTURE DIRECTIONS: SAFE MULTI-FUNCTION ASSISTIVE SYSTEMS

RESEARCH AGENDA

Empowerment is the promise of Physically Assistive Robots (PARs)

If I can have a robot [working with me], it would be me feeding me, and that would be a huge deal.

-Community Researcher with a Spinal Cord Injury

Goal: Work with community members to develop the technologies necessary to achieve that promise.
User Discussions inform New Technologies support System Design
Methods:
- Community-Based Participatory Research: Involve stakeholders in system co-design, pre-build observations, and user studies
- Active Learning: Users wanted robots that can explore for information and exploit that understanding
- Reasoning Through Contact: Leverage haptic sensing and physical contact models to enhance safety and data efficiency

Leverage expert heuristics and data-driven priors for policy-space reduction.

At deployment time: share post hoc haptic info with a vision-based contextual bandit to learn the best policy for new, never-before-seen food items.

Reached user-defined 80% success threshold within 8 Attempts

Emergent Policies:

Wiggling

Titing

High-Pressure

Scooping

Bring on community researchers (CR) as co-designers to create a system robust enough to be deployed in public and the home.

CR: Tyler

CR: Jonathan

Portability: powered from wheelchair battery, no internet
Safety: Watchdog monitoring force sensor at 120Hz and stopping for any anomaly
Customization: Enable full control of the state machine within an accessible app

Tractably learn the shape and pose of arbitrary, dynamic rigid bodies using only tactile data.

Leverage the mathematical structure of contact dynamics to learn and quantify information through time and take actions that maximize expected info gain (EIG).