EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents

Wei Wang, Wenqiao Zhang, Yutong Lin, Yuqian Yuan, Tianwei Lin, Jinhao Mao, Zhenxuan Fan, Mingjian Gao, Yang Dai, Wentong Li, Zheqi Lv, Zheng Dong, Yingjie Niu, Jiaqi Zhu, Jun Xiao, Chao Li, Yueting Zhuang

EmbodiedSkills decouples high-level agent planning from low-level VLA execution using a guarded, contract-based runtime.

How can we decompose long-horizon robot tasks into reliable, verifiable sub-skills to overcome the limitations of monolithic vision-language-action (VLA) models?

Vision–language–action (VLA) models can predict robot motions, but they struggle to coordinate long-horizon tasks where perception, planning, and recovery must evolve as the physical state changes. EmbodiedSkills introduces a shared execution interface that treats every skill as a proposal: the runtime validates prerequisites before execution and verifies outcomes afterward, keeping the agent loop stable even when low-level policies are swapped. On the RoboTwin 2.0 benchmark, this task-adapted approach achieves an 86.20% average success rate, outperforming standard VLA baselines.

Paper Primer

The framework centers on an AgentLoop that manages six phases: Observe, Plan, Preflight, Execute, Verify, and Recover. By enforcing a strict contract—defining typed inputs, prerequisites, and post-execution state updates—the system prevents invalid model proposals from triggering physical actions.

The core move is the separation of policy proposals from runtime-enforced execution: the agent proposes a skill, the runtime guards it against stale observations or incompatible states, and the result is recorded as a structured artifact. This modularity allows the planner, verifier, and low-level VLA policy to be trained or adapted independently without redefining the agent's high-level logic.

Task-adapted low-level VLA policies significantly improve success rates on complex, multi-stage manipulation tasks.

Macro-average success across 50 RoboTwin 2.0 tasks.

The framework maintains high performance across diverse LIBERO manipulation suites.

Macro-average success across LIBERO-Spatial, Object, Goal, and Long suites.

Why is a separate runtime necessary if VLA models are already capable of predicting actions?

VLA models often leave intermediate task decisions implicit, making it difficult to diagnose failures. The runtime makes these decisions explicit and prevents the model from executing actions that are physically invalid or based on stale state information.

Does this framework require training a single, massive end-to-end model?

No. EmbodiedSkills is modular; the planner, verifier, and low-level VLA policy can be adapted independently using component-level supervision, provided they adhere to the shared skill contract.

EmbodiedSkills provides a standardized, inspectable interface for building closed-loop robotic agents, allowing researchers to swap low-level policies without rebuilding the entire agentic orchestration layer.

The Challenge of Long-Horizon Robotics

We expose why monolithic VLA models stumble on long‑horizon tasks and outline our EmbodiedSkills solution.

Vision‑Language‑Action (VLA) models map visual observations and language instructions directly to robot actions, but long‑horizon tasks demand more than single‑step action prediction. A monolithic VLA conflates high‑level planning with low‑level control, causing failures when the proposed operation is invalid or unverified in the current state. EmbodiedSkills addresses this by decoupling planning from execution, using a high‑level agent to orchestrate verifiable, executable sub‑skills.

The core problem is that monolithic VLA models cannot reliably handle long‑horizon tasks because they lack explicit checks and verification between planning and execution.

**Figure 1** Overview of EmbodiedSkills. EmbodiedSkills transforms a low-level VLA policy into a closed-loop embodied agent by coordinating observation, planning, validation, execution, verification, and recovery through executable skills. A high-level policy proposes structured operations, while a guarded runtime validates and executes them and records post-execution feedback as structured trajectories.

Monolithic VLA models fail at long‑horizon tasks because they conflate planning and low‑level action without explicit validation.

Prior Approaches to Robot Control

Survey of prior robot VLA and hierarchical control approaches.

This section situates EmbodiedSkills among existing VLA policies, hierarchical skill planners, reinforcement‑learning agents, and evaluation suites.

A VLA policy directly maps visual observations and language commands to robot motor commands.

ACT splits a long horizon into fixed‑length action chunks and predicts each chunk with a transformer.

DP treats action generation as a denoising diffusion process, gradually refining a noisy action sample into a feasible motor command.

RDT combines transformer sequence modeling with reinforcement‑learning returns to predict actions conditioned on future reward signals.

DP3 extends the diffusion‑policy idea with hierarchical diffusion steps, improving sample efficiency for high‑dimensional actions.

X-VLA augments the basic VLA architecture with additional modality encoders and richer action heads for broader robot embodiments.

Early large‑scale VLA policies (RT‑1, RT‑2) showed transformer‑based models can learn from real‑world robot data. Subsequent works (Open X‑Embodiment, RT‑X, OpenVLA, Octo, $\pi$0, $\pi$0.5, Qwen‑VLA) scaled this direction, improving generality and reusability.

Works such as Say‑Can, Inner Monologue, Code as Policies, VoxPoser, and PaLM‑E explore language‑or vision‑language models that orchestrate robot skills. They connect to classic options, hierarchical RL, skill chaining, and behavior‑tree control.

Recent RL approaches (GRPO‑style reasoning, RLinf‑VLA, $\pi$∗, World2Act) improve large‑model reasoning and robot behavior via environment feedback, verification, and online optimization.

Benchmarks such as RLBench, CALVIN, LIBERO, ManiSkill, RoboCasa, SimplerEnv, RoboTwin (1.0 & 2.0) provide diverse long‑horizon tasks. Infrastructure like StarVLA supports reusable VLA development and evaluation.

By exposing low‑level policies through a stable AgentLoop interface, EmbodiedSkills enables interchangeable skill modules while preserving the benefits of the prior VLA and hierarchical work.

The EmbodiedSkills Framework

We introduce a guarded six‑phase controller that cleanly separates planning from execution.

Monolithic VLA policies collapse high‑level planning and low‑level actuation, which makes long‑horizon tasks brittle. EmbodiedSkills resolves this by inserting a high‑level agent that orchestrates verifiable sub‑skills while a guarded runtime enforces safety.

It treats a robot task as a sequence of explicit phases, each exposing only the operations that are safe given the current evidence.

Step 1: Observe phase creates artifact o₁ (observation of a red block).

Step 2: Plan phase consumes o₁ and produces artifact p₁ (a plan to pick the block).

Step 3: Execute phase consumes p₁ and emits action chunk a₁ (pick‑up command) and execution report e₁.

Step 4: A sensor later reports that the block moved, invalidating o₁ and consequently p₁ and a₁; the runtime drops them and returns to Observe.

The explicit invalidation prevents the controller from acting on stale perception, which is the core safety advantage of EmbodiedSkills.

How does EmbodiedSkills differ from hierarchical RL approaches that also decompose tasks?

Hierarchical RL typically treats sub‑policies as black‑box options and relies on learned termination signals. EmbodiedSkills instead makes every sub‑skill’s input, output, and prerequisites explicit and validates them at runtime, so failures are caught before any physical command is issued.

At each loop iteration the agent builds a compact context, queries the admissible skill set for the current phase, and samples a decision from a learned policy.

Why does the policy output a single unified decision type instead of separate “plan” and “act” networks?

Using one decision interface forces the policy to respect the runtime’s guards; it cannot propose an action that bypasses the prerequisite checks because the guard will reject it. This design eliminates a whole class of unsafe proposals that would otherwise require additional verification logic.

Collect the natural‑language instruction x, current visual evidence, and environment E.

Form the method‑level state st = (zₜ, Mₜ, Hₜ).

Compress st into the compact context Cₜ = $\Psi$(x, zₜ, Mₜ, Hₜ).

Derive the admissible action set $A$ₜ = Gstate($K_{zₜ}$, st) based on the current phase.

Sample a decision dt ∼ $\pi$$\theta$(· | Cₜ, zₜ, Aₜ).

The executable skill contract k = (Xₖ, Yₖ, preₖ, execₖ, postₖ, failₖ) specifies typed inputs/outputs, prerequisites, the actual operation, and how failures are reported.

Bounded action execution in the Execute phase produces an action chunk aₜ = ($\tau$ₜ, Uₜ, Hₜ, $\eta$ₜ) where Hₜ is the horizon and Uₜ is the sequence of primitive commands.

An action chunk is a short, self‑contained command sequence that the low‑level VLA policy can execute atomically.

Verification evaluates the active subgoal and selects a route V ∈ {Advance, Continue, Reobserve, Replan, Recover, Finish} based on the latest execution report.

Guarded six‑phase EmbodiedSkills loop (Algorithm 1)

**Figure 2.** The high-level VLM decomposes the instruction, selects executable subgoals, and verifies progress from post-action observations. The low-level VLA policy executes each active subgoal as a bounded action chunk.

Table 1 enumerates the six phases—Observe, Plan, Preflight, Execute, Verify, Recover—each exposing a distinct role and exit evidence, which together define the state machine that the runtime enforces.

Training the Agentic Layer

Component‑level supervision trains each skill module using the same interfaces the deployed system will see.

Monolithic VLA models collapse planning and motor control into a single network, which makes long‑horizon tasks brittle; EmbodiedSkills sidesteps this by supervising each component through the same runtime interfaces.

Each module—planner, low‑level policy, verifier, scheduler—is trained separately on data that matches exactly what it will receive at deployment, so no module ever sees information it won’t have at test time.

How does this differ from a traditional end‑to‑end VLA training pipeline?

In end‑to‑end training the loss is back‑propagated through the entire stack, so the planner may emit subgoals that the low‑level policy cannot realise. Component‑level supervision cuts the gradient flow, forcing the planner to respect the actual capabilities of the frozen policy.

The low‑level policy receives a concrete subgoal and produces a short, bounded sequence of motor commands, guaranteeing that every action chunk satisfies the verifier’s safety checks.

Why not simply let the planner output raw motor commands directly?

Raw commands would bypass the verifier, making it impossible to guarantee safety or correctness. By constraining the policy to produce bounded chunks conditioned on explicit subgoals, the system can check each chunk before proceeding, which is essential for reliable long‑horizon execution.

Training examples are built from the exact tuple the deployed component will see, ensuring no “future‑information” leakage.

Form the tuple $x_t = ($“place the red block…”, 1, image, (0.2,0.1,0.0), {grasp,move,release}, [prev decisions]$)$.

The planner extracts $(g, I_t, z_t)$ and outputs the subgoal “grasp red block”.

The low‑level policy receives the subgoal, $s_t$, and produces a 5‑step action chunk moving the gripper toward the block.

The verifier checks the resulting state, confirms the block is grasped, and updates $h_{t+1}$.

This concrete walk‑through shows how each field of $x_t$ is used by a different component, making the abstract tuple instantly actionable.

When interaction data are available, the whole AgentLoop can be fine‑tuned online, but the low‑level VLA policy is usually kept frozen to preserve its verified behaviour.

Why keep the low‑level VLA policy fixed during online adaptation?

Fixing the policy prevents the optimizer from inadvertently breaking the safety checks that were verified offline; the high‑level modules can still improve their decision making without risking unsafe motor commands.

Empirical Evaluation

EmbodiedSkills sets new success records on RoboTwin 2.0 and LIBERO, with ablations exposing each loop component.

Monolithic VLA models conflate high‑level planning with low‑level control; EmbodiedSkills separates a high‑level agent that orchestrates verifiable sub‑skills from the low‑level execution policies.

EmbodiedSkills improves macro‑average success on RoboTwin 2.0 by 3.46 percentage points over the strongest VLA baseline.

Achieves 86.20 % average across 50 tasks versus 82.74 % for the reference $\pi$₀.₅ policy.

**Table 2.** Task-level success rates (%) on all 50 RoboTwin 2.0 tasks. The policy baselines are ACT, DP, RDT, and DP3; the generalist VLA references are $\pi_0$, X-VLA, and $\pi_{0.5}$; Ours is the task-adapted subtask-level policy. Each task is evaluated with 100 episodes. Baseline values are taken from the RoboTwin 2.0 benchmark and the reported VLA comparison, while the Ours column is our full 50-task evaluation.

The table compares the performance of "OpenPI" and "Ours" across four suites: Spatial, Object, Goal, and Long, with an final row for the Average.

**Table 4.** Success rates (%) on the four memory-dependent $M(n)$ tasks of RMBench. DP, ACT, $\pi_{0.5}$, and X-VLA are the published RMBench results; Ours is our task-adapted subtask-level policy evaluation.

**Table 5.** Task-level AgentLoop ablation results (success rate, %) on the 50 RoboTwin 2.0 tasks. “w/o V.” removes intermediate verification, “w/o S.” removes semantic subtask conditioning, and “1 chunk” gives each subtask one 32-step action chunk. The four macro averages are Full, w/o Verify, w/o Subtask, and One chunk/subtask, respectively.

**Figure 3.** Cross-benchmark execution results. (a) Average success of representative policy baselines, generalist VLA baselines, and our task-adapted policies on the 50-task RoboTwin 2.0 benchmark. Vertical dashed lines separate method families. (b) OpenPI and our success rates on the four LIBERO suites. (c) Controlled AgentLoop ablations on the same RoboTwin 2.0 task set.

**Figure 4.** Three randomly selected successful execution examples on RoboTwin 2.0.

Ablations and Limitations

EmbodiedSkills decouples planning from execution, enabling a modular AgentLoop.

EmbodiedSkills splits the decision‑making hierarchy: a high‑level agent proposes structured skill goals, while a low‑level VLA policy executes concrete actions.

Because the overall success rate is limited by both layers, specializing the low‑level VLA for individual tasks raises training, storage, and deployment overhead compared with a single generalist policy.

The guarded loop makes intermediate decisions and failures explicit, yet its effectiveness still hinges on the calibration of planning, verification, and action generation.

Explicit contracts guarantee that state transitions respect predefined schemas, but a semantically plausible decision can still steer the robot toward an incorrect physical state.

Modular adaptation lets each component (planner, scheduler, verifier, VLA adapters) specialize, but their interfaces must remain compatible; over long episodes small planning and execution errors accumulate, increasing the number of observations, retries, and model calls before task completion.

Empirically, RoboTwin 2.0 with task‑adapted $π_{0.5}$ policies attains 86.20% average success on 50 tasks, the LIBERO VLA policy reaches 97.40% across four suites, and the RM‑Bench evaluation records 12.5% average success on memory‑dependent tasks.

The components that most drive success are the high‑level skill planner and the verification step that checks readiness before execution; improving either yields the largest gains.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers