Lifelong Adaptation
Ethan K. Gordon
June 26, 2026
Dagstuhl Seminar Breakout Session
Wednesday 11am
Definition: What is active learning?
Tentative Summary (Can be Adapted Online)
Contention: Online adaptation is necessary for assistive devices.
Debate: Autonomous Active Learning vs. Manual Customization
Brainstorm: Methods (Interfaces and Algorithms) for Online Adaptation
Discussion: Refined Challenge/Question + Key Points
Active Exploration
Choosing actions to take in the environment with the goal of gaining useful information.
Online Learning
Using data collected at test time to permanently update:
- Policy
- Model
- Reward Distribution, etc.
What is Active Learning?
+
Example: OL w/o AE
Any Passive Observation
Many Greedy RL / Bandit Algorithms
Assume we have a prior amenable to online learning (see yesterday's session)
Example: AE w/o OL
Policy with History \(\pi_\theta(a|s_{[t-H,t]})\)
Choose actions to create a "good" context, but don't update \(\pi\)
Contention: Online Adaptation is Necessary
Why not collect enough data for the model / policy / etc. to be perfect in advance?
Technical Considerations:
- Covariate Shift: who we see in the lab is biased, the general population can be (and likely is) very different!
- People change overtime: what if you train for everybody right now but tomorrow everybody is different?
Users Want It (to paraphrase from Monday):
- Christina: I want a robot that can adapt to the changing capabilities of the person. There are no averages.
- Charlotte: Focus on different designs for different symptoms on the same robot.
Contention: Online Adaptation is Necessary
Much stronger argument for Active Exploration specifically:
- Test-Time Epistemic Uncertainty is unavoidable, example
True Preference Distribution:
70% Outside
30% Inside
(training learns this)
Aleatoric uncertainty at training time:
inherent to the system. No training will tell us about any specific user.
Epistemic uncertainty at test time:
we can collect info about the user to reduce/eliminate it.
Contention: Online Adaptation is Necessary
Other examples in feeding:
- what the person wants to eat / what food is available
- the configuration of food on the plate
- how the person whats the food to be prepared in the moment (bite size, collection of heterogenous food on the utensil)
- how fast a person wants to eat
- what position the robot should be relative to the person both during feeding and during breaks
- style of approach (conditioned on the food and the person and the day)
- style of food transfer (conditioned on the food an the person and the day)
Contention: Online Adaptation is Necessary
Any Objections? Nuances? Thoughts?
Debate: Active Learning vs. Manual Customization
Position 1: Active Learning
The robot should autonomously adapt to user preferences based on objective or subjective feedback. This can include deciding when to directly ask the user or caregiver for help when uncertainty gets too high.
Position 2: Manual Customization
Online adaptation should be completely orchestrated by the user via an extensive customization interface for each facet of the task. This can include e.g. asking the robot for assistance in understanding what each customization option does.
Brainstorm: Methods and Interfaces
Autonomous Adaptation
Contextual Bandit (limited RL) Algorithms
Optimal Experimental Design (e.g. Fisher Information)
Asking questions in natural language
Customization Interfaces
Phone App: Literal Sliders and Knobs
Direct Teleoperation with cache/saving
Kinesthetic placement / motion
Discussion: Let's do our homework.
Refined Challenge / Question?
5 Key Points for the Abstract
Breakout: Lifelong Active Learning
By Michael Posa
Breakout: Lifelong Active Learning
Dagstuhl Seminar Breakout, Quick Slides
