Lifelong Adaptation

Ethan K. Gordon

June 26, 2026

Dagstuhl Seminar Breakout Session

Wednesday 11am

Definition: What is active learning?

Tentative Summary (Can be Adapted Online)

Contention: Online adaptation is necessary for assistive devices.

Debate: Autonomous Active Learning vs. Manual Customization

Brainstorm: Methods (Interfaces and Algorithms) for Online Adaptation

Discussion: Refined Challenge/Question + Key Points

Active Exploration

Choosing actions to take in the environment with the goal of gaining useful information.

Online Learning

Using data collected at test time to permanently update:

  • Policy
  • Model
  • Reward Distribution, etc.

What is Active Learning?

+

Example: OL w/o AE

Any Passive Observation

Many Greedy RL / Bandit Algorithms

 

Assume we have a prior amenable to online learning (see yesterday's session)

Example: AE w/o OL

Policy with History \(\pi_\theta(a|s_{[t-H,t]})\)

 

Choose actions to create a "good" context, but don't update \(\pi\)

Contention: Online Adaptation is Necessary

Why not collect enough data for the model / policy / etc. to be perfect in advance?

Technical Considerations:

  • Covariate Shift: who we see in the lab is biased, the general population can be (and likely is) very different!
  • People change overtime: what if you train for everybody right now but tomorrow everybody is different?

Users Want It (to paraphrase from Monday):

  • Christina: I want a robot that can adapt to the changing capabilities of the person. There are no averages.
  • Charlotte: Focus on different designs for different symptoms on the same robot.

Contention: Online Adaptation is Necessary

Much stronger argument for Active Exploration specifically:

  • Test-Time Epistemic Uncertainty is unavoidable, example

True Preference Distribution:

70% Outside

30% Inside

(training learns this)

Aleatoric uncertainty at training time:

inherent to the system. No training will tell us about any specific user.

Epistemic uncertainty at test time:

we can collect info about the user to reduce/eliminate it.

Contention: Online Adaptation is Necessary

Other examples in feeding:

  • what the person wants to eat / what food is available
  • the configuration of food on the plate
  • how the person whats the food to be prepared in the moment (bite size, collection of heterogenous food on the utensil)
  • how fast a person wants to eat
  • what position the robot should be relative to the person both during feeding and during breaks
  • style of approach (conditioned on the food and the person and the day)
  • style of food transfer (conditioned on the food an the person and the day)

Contention: Online Adaptation is Necessary

Any Objections? Nuances? Thoughts?

Debate: Active Learning vs. Manual Customization

Position 1: Active Learning

The robot should autonomously adapt to user preferences based on objective or subjective feedback. This can include deciding when to directly ask the user or caregiver for help when uncertainty gets too high.

Position 2: Manual Customization

Online adaptation should be completely orchestrated by the user via an extensive customization interface for each facet of the task. This can include e.g. asking the robot for assistance in understanding what each customization option does.

Brainstorm: Methods and Interfaces

Autonomous Adaptation

Contextual Bandit (limited RL) Algorithms

Optimal Experimental Design (e.g. Fisher Information)

Asking questions in natural language

Customization Interfaces

Phone App: Literal Sliders and Knobs

Direct Teleoperation with cache/saving

Kinesthetic placement / motion

 

Discussion: Let's do our homework.

Refined Challenge / Question?

5 Key Points for the Abstract