EmbodiedCLBench: Evaluating Continual Learning for Self-Evolving Embodied Agents
Jie Huang, Yanan Chen, Ruixun Liu, Kaichen He, Xinyi He, Rui Huang, Zilong Zheng, Zhenliang Zhang, Yiwu Zhong
EmbodiedCLBench evaluates whether self-evolving agents can transfer learned skills to novel, complex household tasks.
How can we measure whether embodied agents actually learn to compose skills over time, rather than just memorizing specific task sequences?
Self-evolving agents are expected to learn from experience, but current benchmarks for continual learning are restricted to digital or symbolic environments that lack the physical constraints of real-world settings. The authors introduce EmbodiedCLBench, which uses compositional generalization to test if agents can recombine basic household skills to solve unseen, advanced tasks. The benchmark operates offline using state-transition graphs derived from the TongSim simulator. While agents struggle with advanced tasks in zero-shot settings, experience from basic tasks consistently improves success rates across multiple agent harnesses.
Paper Primer
The benchmark frames continual learning as a compositional challenge: agents first complete basic tasks to accumulate experience, then attempt advanced tasks that require recombining those skills. This approach avoids the need for benchmark-specific training, instead using basic tasks as test-time learning data to measure how well an agent transfers procedural knowledge.
Prior experience from basic tasks significantly boosts performance on unseen, complex tasks.
Across four agent harnesses, success rates on advanced tasks increased from approximately 10% in zero-shot settings to over 30% when agents were provided with basic task trajectories or skill summaries.
Continual learning benefits do not scale with the number of learning samples.
Performance jumps significantly after the first basic task, but adding further samples (up to $k=6$) yields no consistent additional improvement, suggesting current agents struggle to extract distinct information from larger datasets.
Why is this benchmark built on offline state-transition graphs rather than live simulation?
The authors use a closed-source version of the TongSim simulator that supports diverse actions but is not practical for online evaluation. The graph structure allows for parallel, reproducible testing and fine-grained analysis of agent decision-making at each state.
How does the benchmark distinguish between "learning" and "zero-shot" performance?
The evaluation compares zero-shot performance against two learning settings: In-Context Learning, where the agent retains full interaction trajectories of basic tasks, and Skill Learning, where the agent is provided with a concise Markdown summary of its experience.
EmbodiedCLBench demonstrates that while current agents can transfer basic skills to new compositions, they lack the ability to scale this learning with more experience. Future research must focus on mechanisms that allow agents to continuously consolidate and reuse information beyond a single task.
Motivation and Core Concept
We frame the need to evaluate continual learning via compositional generalization in embodied tasks.
EmbodiedCLBench evaluates continual learning by testing whether agents can recombine previously learned atomic skills to solve novel, unseen advanced tasks, rather than merely memorizing task sequences.
Continual learning is the ability of a model to accumulate knowledge from past tasks and transfer it to new ones.
Compositional generalization measures whether an agent can combine known skill elements in new ways to solve tasks it has never seen.
Existing benchmarks either ignore embodied constraints or lack a dedicated continual‑learning evaluation, leaving a clear gap for measuring how agents reuse prior experience in physical environments.
**Figure 1.** Compositional generalization is leveraged as a principled way to measure continual learning capability. An agent that has experience from basic tasks should recombine previously acquired skills to solve novel, unseen advanced tasks, rather than learning each new task from scratch.
Shifting from static datasets to compositional embodied tasks is essential for assessing continual learning.
Benchmark Design and Construction
We detail the benchmark construction, task design, and evaluation protocol for EmbodiedCLBench.
Existing continual‑learning benchmarks focus on digital or symbolic domains, leaving embodied scenarios under‑explored. EmbodiedCLBench fills this gap by constructing a reproducible, offline evaluation pipeline that stresses compositional generalization.
Think of it as a curriculum that first teaches a robot simple recipes (basic tasks) before asking it to prepare a full‑course meal (advanced tasks), all while measuring how well it reuses the learned steps.
How does EmbodiedCLBench differ from prior embodied AI benchmarks?
Most embodied benchmarks assess perception, navigation, or manipulation in isolation, whereas EmbodiedCLBench explicitly measures the ability to recombine previously learned atomic skills to solve novel, longer‑horizon tasks without additional training.
Like a single Lego brick: an atomic action is one indivisible interaction (e.g., “pick up the cup”) that can be snapped together with other bricks to build more complex behaviors.
Why call them “atomic” actions instead of “primitive” actions used elsewhere?
“Atomic” emphasizes that the action is treated as an indivisible unit in the benchmark’s graph representation, whereas “primitive” often refers to low‑level motor commands that can be further subdivided.
Step 1: Agent executes “pick up the cup” → cup is now held.
Step 2: Agent executes “place the cup on the table” → cup rests on the table; task succeeds.
Step 1 (advanced): “pick up the book” → book held.
Step 2: “place the book on the shelf” → book on shelf.
Step 3: “close the cabinet” → cabinet door closed.
Step 4: “switch off the lamp” → lamp turned off; advanced task succeeds.
Basic tasks expose the agent to a small set of atomic actions, while advanced tasks test its ability to sequence and reuse those actions in new combinations.
Set up the TongSim household environment with diverse objects and executable actions.
Generate basic and advanced tasks by composing atomic actions; descriptions are produced by Qwen3.7‑Plus and manually verified.
Build a deterministic state‑transition graph for each task, pairing each node with a first‑person image and a set of candidate actions.
Run the evaluation protocol: agents learn from basic tasks (learning phase) then are tested on unseen advanced tasks (testing phase) using the offline graph.
**Figure 2.** Overview of the construction process of *EmbodiedCLBench*. During task generation, basic tasks connect a few atomic actions into simple household operations, and advanced tasks combine elements from multiple basic tasks into compositional tasks. For each task, a state-transition graph is built, with each graph node as an intermediate state and paired with action options and a first-person-view image. During evaluation, the agent observes the current state and selects from candidate actions. After each selection, the agent transitions to the next node on the graph.
**Figure 3.** Benchmark statistics of EmbodiedCLBench: environment configuration (a), task composition across six categories (b), and task complexity at two levels (c).
Main Experimental Results
Core results show how basic‑task experience lifts advanced‑task success.
Advanced tasks remain hard under zero‑shot, but leveraging basic‑task experience raises success to above 30%.
Table 1 shows advanced‑task success rates increase from ~10% (zero‑shot) to 31–38% with ICL or Skill Learning across all harnesses.
All experiments used the same step budget (shortest‑path length + 3 tolerance) and identical task sets, so differences stem solely from the learning setting.
**Table 1.** Main results on EmbodiedCLBench. For In-Context Learning and Skill Learning, agents first learn from two basic tasks and then attempt one advanced task.
Baseline agents achieve ~50% on basic tasks but only ~10% on advanced tasks without any learning.
Complexity and Error Analysis
We examine how compositional complexity, learning samples, error types, and step budgets affect continual‑learning performance.
Recall that EmbodiedCLBench measures whether agents can recombine learned atomic actions to solve unseen advanced tasks, rather than merely memorising task sequences.
Increasing compositional complexity (more atomic actions per advanced task) sharply reduces zero‑shot success but only modestly harms learned methods.
Zero‑shot success falls from 28 % (3 actions) to 4 % (6 actions); ICL drops from 72 % to 42 % over the same range, while the gain over zero‑shot stays near 30 %.
**Figure 4.** Effect of compositional complexity on continual learning. The x-axis varies the number of atomic actions per advanced task from 3 to 6. (a) reports the raw success rate achieved by each setting. (b) reports the gain over zero shot, defined as the success rate of ICL or Skill Learning minus that of zero shot, isolating the improvement attributed to the learning phase.
Providing a single basic‑task example yields the majority of the performance boost, while additional examples do not consistently improve success.
Success jumps from ~9 % (k = 0) to 34 % (k = 1) across both harnesses; beyond k = 1 the curves fluctuate without a clear upward trend.
**Figure 5.** Success rate versus the number of learning samples. Each learning sample is a basic task that the agent completes before being tested on advanced tasks.
Experience mainly shifts action selection away from invalid moves toward optimal and suboptimal choices.
Invalid‑action rates drop by 13–20 % across all models; optimal‑action rates increase substantially, suboptimal actions rise modestly (3–8 %).
**Figure 6.** Error analysis of decision behavior. Each group of bars shows the change in selection rate relative to the zero-shot setting for three action types: optimal actions follow the shortest path, suboptimal actions are executable but deviate from the shortest path, and invalid actions cannot be executed in the current state.
Allowing more extra steps consistently raises success rates, and learned agents benefit more steeply from the added tolerance than zero‑shot agents.
At 0 extra steps, zero‑shot = 0 %, ICL = 15 %, Skill = 7 %; success grows monotonically up to 6 extra steps for all methods.
**Figure 7.** Step budget sensitivity. The x-axis shows the number of extra steps allowed beyond the shortest path. By default, agents are allowed 3 extra steps in the testing phase.
Task Capability Categories
Appendix A enumerates the six basic task categories used in EmbodiedCLBench.
**Table 2.** Six capability categories of basic tasks in *EmbodiedCLBench*.
Can the agent locate and manipulate everyday objects into a usable or manageable state?
Can the agent understand spatial relationships between objects and surfaces or containers?
Can the agent change the state of doors, drawers, and cabinet doors to enable or restrict access?
Can the agent toggle household appliances and environmental devices?
Can the agent set up multiple objects and devices to support activities such as reading, dining, or leisure?
Can the agent restore a room to a tidy, energy‑efficient, and reusable state?
Task Generation Pipeline
Describes the graph‑based pipeline for generating and verifying task samples.
We first build a state‑transition graph from atomic actions, then run a three‑stage LLM pipeline to generate descriptions and options, and finally verify each sample with three annotators.
Build the state‑transition graph using the atomic actions available in TongSim.
Stage 1 – rewrite the baseline task description into a natural, scene‑based household request while preserving the underlying task structure.
Stage 2 – rewrite the four hard‑coded action options at each graph state into plausible next steps, ensuring all options hide the observe‑approach‑operate phases.
Stage 3 – detect and eliminate cross‑state linguistic patterns, applying hardening strategies that blur phase boundaries, vary phrasing, and break rhythmic repetitions.
Three independent annotators verify each generated sample for factual errors, logical inconsistencies, and realistic household activity; a sample is kept only if all annotators approve it.
Basic Task Examples
Illustrates basic task examples with state‑by‑state action options and hidden graph roles.
This appendix presents concrete task examples used to evaluate agents on compositional generalization.
State 0 – select option D (“The overhead light is still on …”) → correct shortest‑path action.
State 1 – select option D (“Walk over to the ceiling light …”) → correct shortest‑path action.
State 2 – select option A (“Switch off the overhead light …”) → correct shortest‑path action.
State 3 – select option D (“With the light off, the media cabinet …”) → correct shortest‑path action.
State 4 – select option B (“Walk up to the TV unit …”) → correct shortest‑path action.
State 5 – select option B (“Open the media cabinet …”) → correct shortest‑path action.
The sequence follows the hidden graph’s shortest‑path actions, ignoring detours and invalid options.
Table 4 shows a second basic‑task example where the goal is to ready the living room for a relaxed evening by turning on background entertainment and opening the storage cabinet.
**Figure 8.** Visual states along the shortest path for a basic-task example.
**Figure 9.** Visual states along the shortest path for a basic-task example.
The image displays a table outlining a sequence of states and options for an agent performing a task: "I'm about to watch a movie in the living room. Turn off the overhead light so the screen isn't washed out, and open up the media cabinet so everything's within reach." The table columns are "State", "Option", "Text shown to the agent", and "Hidden role".
**Task:** Get the living room ready for a relaxed evening — turn on some background entertainment and open up the storage cabinet so everything’s within reach.
Advanced Task Examples
Illustrates advanced‑task examples and the hidden‑role labeling used for state‑option decisions.
The appendix lists concrete examples of how an agent must navigate a multi‑step task, with each possible action labeled by its hidden role (Correct shortest‑path action, Executable detour, or Invalid in current state).
State 0 – Option C (“The lamp’s still glowing …”) is marked as the Correct shortest‑path action; the agent dims the lamp.
State 1 – Option B (“Get closer to the lamp …”) is the Correct shortest‑path action; the agent moves within reach of the lamp to turn it off.
State 2 – Option B (“Kill the lamp light …”) is the Correct shortest‑path action; the lamp is turned off, completing the lighting sub‑task.
State 3 – Option B (“The study desk still has stuff on it …”) is the Correct shortest‑path action; the agent opens the desk to prepare it for tomorrow.
Only one option per state is the Correct shortest‑path action; the other three options are either Invalid (cannot be executed now) or Executable detours (legal but sub‑optimal moves that do not advance the primary goal).
**Figure 10.** Visual states along the shortest path for an advanced-task example, including the final state.
Table 5 enumerates the four textual options for each of the first four states, explicitly marking the hidden role of each option (Correct shortest‑path action, Executable detour, or Invalid in current state). Table 6 continues this enumeration for states 4 through 11, preserving the same four‑option structure.
Task State Transitions
Illustrative task sequences demonstrate valid, detour, and invalid actions for desk, lamp, cabinet, and remote.
Glance over at the desk to see what’s still out on the surface — the agent notices several objects.
Step up to the desk so you can reach across and clear it off properly — the agent lifts the remote and places it on the desk.
Open the desk cabinet so you can stow tonight’s items inside — the cabinet door swings open, revealing an empty interior.
Shift focus to the desk area to see what needs clearing for tomorrow — the agent scans the surface again.
Glance over at the desk surface to assess what else still needs to be cleared away — a stray notebook is identified.
Turn your attention to the desk area to see what still needs clearing — the agent picks up the notebook and puts it in the cabinet.
Shut the desk drawers to keep the room tidy for the night — the cabinet door closes, completing the desk task.
Shortest‑path actions directly achieve the goal (e.g., opening the cabinet to store items), while detours (glances) are permissible but add extra steps; invalid actions would violate state constraints.
Glance over at the table lamp to check whether the room is dim enough — the agent perceives the lamp is on.
Check the table lamp status to see if it needs dimming for bed — the brightness level is above the desired threshold.
Shift focus to the table lamp area to continue dimming the room for rest — the agent reaches for the lamp switch.
Execute the dimming action (not shown) — the lamp brightness lowers to the target level.
Detour actions (glances, status checks) are useful for assessing the environment, but the essential shortest‑path step is the actual dimming operation.
The bedroom nightstand is still open — head back and deal with that next — the agent returns to the nightstand.
Move in close to the nightstand — the cabinet needs shutting and you need to be right there — the agent positions themselves beside the cabinet.
Shut the bedside cabinet to keep the nightstand tidy before sleep — the cabinet door closes, sealing the nightstand.
Check the bedside cabinet to make sure it’s properly closed for the night — the agent verifies the door is latched.
Closing the cabinet is a shortest‑path action; earlier glances and repositioning are detours that prepare the agent for the final closure.
Take a look at the remote control in your hand to decide where it should go next — the agent decides on the trash can.
The remote control still needs to go — spot the trash can so you know where it’s headed — the agent locates the trash can.
Get to the trash can — the remote control in your hand needs to go there — the agent walks to the trash can.
Set the remote control down near the trash can to keep the nightstand clear — the remote is placed at the trash can’s edge.
Seal up the trash can so nothing spills out overnight — the lid is closed after placing the remote.
Check the trash can to make sure it’s ready to receive items — the agent confirms the lid is secure.
The essential shortest‑path sequence is moving to the trash can and depositing the remote; preceding glances and decisions are detours that guide the agent.