Artificial intelligence
In-context robot learning
Definition
In-context robot learning is robot adaptation in which demonstrations, examples, or interaction feedback supplied at deployment condition new behavior without a task-specific parameter update. The context acts as a prompt that tells a trained policy what to do in the current setting.
Also known as: Robot in-context learning, Robotic in-context learning
Updated
Adapt through the prompt
An in-context robot learner receives a context, such as a short video demonstration, an example trajectory, or recent action outcomes, together with the current observation. It infers a task or strategy from that context and generates actions using its existing parameters. The SimpleICL preprint studies visual demonstrations as prompts and describes robot in-context learning as an emerging, still under-defined paradigm.
This can let one trained policy attempt a new object arrangement or manipulation instruction without another gradient-based training run. The context may show the desired goal, useful spatial relations, or the order of sub-tasks.
Not every demonstration specifies the same thing
A visual example contains several signals at once: object identity, motion, affordances, spatial relations, and the final state. The SimpleICL authors call this prompt ambiguity because the policy may follow a different part of the example than the operator intended. An evaluation should vary these signals independently instead of treating one successful replay as evidence of task understanding.
In-context learning differs from imitation learning as a training category. Imitation learning fits a policy from demonstrations, while the in-context case uses examples during inference. It also differs from test-time adaptation methods that update model parameters at deployment.
Context has practical limits
Long or high-resolution demonstrations consume input capacity and inference time. A policy may depend on superficial visual matches, fail when the camera or embodiment changes, or reproduce an unsafe part of an example. It also needs a controller that can translate the inferred intent into feasible robot motion.
The SimpleICL results are author-reported simulation and real-world experiments for one framework. They support the feasibility of its setup, not a general claim that prompt-only adaptation is reliable across robots or contact-rich tasks.
Sources
Related terms
Imitation learning
Imitation learning learns behavior from examples supplied by a demonstrator. In robotics, demonstrations can teach a policy how to perform a task without requiring every action or objective to be programmed by hand.
Robot-free demonstration
A robot-free demonstration records a human example without driving the target robot during data collection. Tracking devices, instrumented grippers, or video are used to recover observations and actions that can later train a robot policy.
Test-time adaptation
Test-time adaptation adjusts a trained model using data encountered during evaluation or deployment. Unlike ordinary fixed-model inference, it updates model parameters or statistics in response to the target data.
Vision-language-action model
A vision-language-action model is an AI model that uses visual observations and language instructions to produce actions for a robot. It connects what a robot sees and what it is asked to do with outputs that a robot controller can execute.