Fewer Tokens, Better Action: GPT-6 Astra Robot Agents with 14% Higher Success Rate but 65% Fewer Tokens
Ruiyang Si, Jianxin Bi, Shunyu Yang, Rui Ni, Wenbo Huang, Qiang Wang, Shulong Jiang, Duomin Wang, Xiuyu Li, Haiwen Feng, Zhen Dong, Daquan Zhou
PyRUA-Lean uses interactive Python code execution to compose robot primitives, increasing task success while slashing token overhead.
Can robot agents achieve higher task success with significantly fewer tokens by replacing iterative tool-calling with programmatic primitive composition?
Vision-language model (VLM) agents for robotics often rely on tool-calling interfaces that force a new model invocation for every step, leading to redundant observations and high token costs. PyRUA-Lean replaces this with an interactive Python execution interface where the agent composes robot primitives into code cells, enabling conditional logic, local retries, and selective visual feedback. Across 700 simulated tasks, this approach increases overall success by 14% while reducing input token consumption by 65% on jointly solved instances.
Paper Primer
The core mechanism is a persistent Python runtime that allows the agent to chain multiple robot primitives—such as motion, perception, and vision-language-action (VLA) policies—within a single code cell. By moving intermediate coordination and geometric computation into the runtime, the agent avoids repeated model calls and only returns requested images to the VLM context: code is the orchestrator, and the VLM is the programmer.
PyRUA-Lean significantly improves task success rates compared to standard tool-calling baselines.
Across 700 simulated instances from LIBERO-PRO, RoboTwin 2.0, and RoboCasa365, the overall success rate rose from 63.1% to 71.7%.
The interface drastically reduces inference overhead on tasks solved by both agents.
On jointly solved instances, PyRUA-Lean required 49% fewer LLM calls and 65% fewer input tokens.
Why does this approach outperform simple "on-demand" image retrieval?
The authors found that merely returning images on request is insufficient; success requires integrating selective feedback with programmatic action execution and replanning, which tool-calling interfaces cannot natively support.
Does this method rely on the VLA policy to achieve its efficiency gains?
No; ablation studies show that PyRUA-Lean maintains higher success and lower token usage even when VLA policies or operating guides are removed, as the agent can compose classical primitives into effective routines.
For embodied agents, moving from a "tool-calling" paradigm to an "interactive code-execution" paradigm allows for feedback-driven control flow that is both more reliable and significantly cheaper to run.
Introduction and Motivation
We expose the inefficiency of repeated VLM calls and propose a code interface to cut token waste.
Vision Language Model (VLM) agents repeatedly invoke the model for each robot action, accumulating token overhead and redundant observations.
In a tool‑calling interface, every primitive’s result triggers a new VLM call, and intermediate camera images flood the context, inflating inference cost.
Repeated VLM invocations make the agent spend most of its budget on bookkeeping rather than advancing the task.
**Figure 8.** An instance only code solved: LIBERO-PRO, “turn on the stove and put the moka pot on it” (10 swap, task 2, seed 3). Both agents turned the stove on and dropped the pot. Tool calling asked Pi0 to grasp the fallen pot four times and ended the episode; the code agent grasped it itself, fitted a plane to the points of its lid to find how it leans, turned it upright, and set its base, not the gripper, over the burner. *Top*: the recorded camera views after the LLM calls named. *Bottom*: the code agents cells, excerpts exactly as the model wrote them; “...” marks lines left out. The plane fit in call 23 raised an error (NumPy’s linear algebra refuses half-precision arrays); call 24 cast the points and ran it again.
**Figure 9.** An instance only code solved: RoboCasa365 atomic, “Open the cabinet door.” (seed 4). Both agents located the door’s hinge. Tool calling pulled the handle a few centimeters per LLM call and was still pulling when its 40 calls ran out; the code agent took the hinge as the centre of the circle through three positions of the handle, two of which it copied from earlier outputs, and pulled along that arc in one cell, stopping if a move fell short or the grip slipped. The door opened, and a VLA call in call 34 completed the task. *Top*: the recorded camera views; *bottom*: the code agent’s cells, excerpts exactly as the model wrote them.
The core inefficiency stems from invoking a VLM at every action step, which PyRUA‑Lean eliminates by composing primitives in code and returning only needed observations.
Methodology: PyRUA-Lean
We compare tool‑calling and code‑based robot agents, then introduce PyRUA‑Lean’s core mechanisms.
Prior VLM agents repeatedly invoke the model for each primitive, inflating compute and token usage. By contrast, code‑based agents can batch primitives, but their design space is less explored for robot manipulation.
A VLM selects a robot primitive, calls it, receives the result, and repeats this loop for every action step.
Instead of calling the model for every primitive, the VLM emits a Python cell that composes multiple primitives and runs them locally, returning only the feedback the agent explicitly requests.
How does PyRUA‑Lean differ from the standard tool‑calling loop?
Tool‑calling sends a single primitive per model turn, forcing the VLM to re‑encode the full history each time. PyRUA‑Lean lets the VLM emit a whole Python cell that runs many primitives locally, returning only the explicitly requested observations, thus collapsing many turns into one.
The agent writes Python code that stitches together primitive calls, helper functions, and computed values, enabling complex task logic without leaving the runtime.
robo.`move_to`(`bowl_pos`) → returns `approach_success`=True.
if `approach_success`: robo.grasp(bowl) → returns `grasp_success`=True.
if `grasp_success`: `target_z` = bowl.height + 0.05; robo.place(`target_z`) → returns `place_success`=True.
By nesting the conditionals inside one cell, the agent avoids three separate VLM calls; only the final success flag is sent back.
Why isn’t this just ordinary Python scripting?
The key difference is that the code is generated on‑the‑fly by a VLM and executed within a constrained runtime that returns only the explicitly requested observations. Ordinary scripting would not involve the model’s reasoning loop or the selective feedback mechanism.
The agent decides which intermediate results (images, state prints) are sent back to the VLM, suppressing unnecessary data.
What would happen if every primitive’s image were returned?
The prompt would quickly balloon with high‑resolution images, exhausting the model’s context window and forcing costly summarization or truncation, which degrades performance and increases inference cost.
**Figure 2.** PyRUA-Lean combines primitive composition with selective feedback. (a) The tool-calling agent invokes robot primitives through tool calls. Operations that depend on preceding results generally require another VLM turn, and motion calls automatically return images and state. (b) PyRUA-Lean instead generates Python cells that compose robot primitives with helper functions, conditional checks, and local retries. Intermediate execution stays within the runtime, while explicitly requested images and state messages are recorded and returned at cell end for replanning. (c) A recorded placement example illustrates both mechanisms: one PyRUA-Lean cell replaces baseline steps 14–17, reducing four LLM turns to one. The cell computes a placement target from scene geometry (compose), conditionally executes lowering and release (compose), and requests an image only if the task remains unfinished (select). This successful execution returns five printed lines and no image, demonstrating how programmatic composition and selective feedback reduce repeated model interaction. Appendix A follows the whole episode.
Together, these mechanisms let PyRUA‑Lean execute complex manipulation tasks with far fewer LLM invocations while preserving the ability to react to failures.
Experimental Setup
PyRUA‑Lean dramatically cuts token usage while boosting task success on composite benchmarks.
PyRUA‑Lean raises success rate by 11.0 points on the RC365 composite benchmark versus tool calling.
Table 2 shows 94.0 % success for PyRUA‑Lean compared to 83.0 % for the baseline.
We evaluate 40 tasks from LIBERO‑PRO, 50 dual‑arm tasks from RoboTwin 2.0, and 50 tasks from RC365’s Target50 set, each with five random seeds, yielding 1 400 episodes per agent.
Both agents use GPT‑6 Astra with identical robot stacks and frozen $VLA$ policies ($\pi$0.5, LingBot‑VLA, RLDX‑1), and share the same segmentation backbone (SAM 3).
Success is measured under a 40‑call (or 100‑call for RC365 composite) budget per episode, with a two‑hour wall‑time limit; we also report cumulative token counts and estimated monetary cost.
Main Results
PyRUA‑Lean boosts task success while cutting inference cost across all benchmarks.
We now present the quantitative impact of PyRUA‑Lean compared to the tool‑calling baseline.
PyRUA‑Lean raises overall task success from 63.1 % to 71.7 % while halving inference cost.
Table 2 reports an 8.6 pp absolute gain (≈ 14 % relative) and a drop in estimated per‑episode cost from \$1.63 to \$0.74.
**Figure 1.** PyRUA-Lean matches or exceeds tool calling’s success rate on each sub-suite and costs less on every benchmark. (a) Success rate on all 700 instances, tagged by task property (semantic: perturbed layouts, objects, goals); RoboTwin 2.0 split by skill from task names. (b, c) Input tokens and GPT-6 Astra list-price dollars per solved episode, on instances both agents solved; Avg: mean over all of them. RC365: RoboCasa365.
Token‑budget efficiency is further illustrated by varying the allowed token budget per episode.
**Figure 3** Success rate if every episode were stopped once its token consumption reached the budget on the horizontal axis. Dashed lines mark the budget at which each agent first reaches tool calling's final success rate, with its value on the axis; the large number is their ratio. RC365: RoboCasa365.
Qualitative examples demonstrate capabilities that the baseline cannot achieve.
**Figure 4** Only code solved this LIBERO-PRO instance (“turn on the stove and put the moka pot on it”): both agents dropped the pot, and the code agent fitted a plane to the lid, turned the pot upright and set its base over the burner (Appendix D).
Overall, the results confirm the three hypotheses: higher success, lower inference cost, and robustness of the programmatic composition approach.
Ablations and Efficiency
We quantify how each component contributes to efficiency and success.
**Figure 5** Token-savings decomposition on jointly solved instances. Tool-calling-to-PyRUA-Lean ratios quantify reductions in LLM calls (solid) and tokens per call (light); their product gives the total token-reduction factor (right). Ratios below 1x indicate increased usage by PyRUA-Lean.
Removing the VLA policy drops success rates by 4–11 % across environments, while stripping operating guides hurts success by up to 8 % and inflates LLM calls and token usage. The code‑based agent consistently outperforms the tool‑calling baseline, even when components are ablated, confirming that programmatic primitive composition is the primary driver of the observed efficiency gains.
Qualitative Episode Analysis
This appendix details the token savings and success gains of the code agent versus tool‑calling.
The episode illustrated in Figure 2 shows a stark contrast: the tool‑calling agent reads the guides, localizes objects, and moves one step per LLM call, receiving three camera images after each move. It reaches the goal at call 17 after processing 33 images. In contrast, the code‑agent composes primitives programmatically, localizes both objects in its first cell, grasps the bowl in the second, checks the grasp in the fourth, and completes the task by call 6 with only five images.
Because the code agent executes multiple primitive actions per model turn, it achieves a 14% higher success rate while using 65% fewer tokens than the tool‑calling baseline. The token reduction stems from the fact that each call supplies three camera images, whereas the code agent needs only a single image per call.
Initial state, object segmentation, and first two LLM calls.
Lifting the bowl after it is grasped.
Carrying the bowl toward the plate and computing the placement offset.
Final placement correction, lowering, and release.
Supplementary Data
Appendix details the code agent’s behavior and extra episode analyses.
Table 5 expands the five‑task per‑benchmark analysis shown in Figure 7, reporting LLM calls and token consumption for each whole episode of the Tool and Code agents.
**Figure 7** Token consumption so far of the episodes in Table 5, five per benchmark, LLM call by LLM call; circles mark the solving call. On RoboTwin 2.0 and RoboCasa365's atomic tasks the median episodes of both agents are nearly identical from task to task, so their lines overlap.
Table 6 quantifies what each code‑agent cell does across all 700 episodes, contrasting it with a single tool‑calling LLM call.
**Figure 6** The episode of Figure 2, whole. *Top*: each box is one LLM call; the number is how many tools (tool calling) or primitives (code) it called, dark boxes returned camera images, and the outlined box solved the task. *Bottom*: token consumption so far; circles mark the solving call, reached after 816k tokens with tool calling and 156k with code. *Left*: the agent-view camera at the start and when the task was solved.
Four recurring patterns explain why a code cell packs more work than a tool call.
Guarded chains appear in 19 % of cells (45 % on LIBERO‑PRO); a cell runs several primitives sequentially, each depending on the previous result, whereas tool calling would require a separate LLM call for every link.
Listing 1 – Guarded chain example from LIBERO‑PRO (spatial swap, task 0, seed 0)
Retries occur in 8 % of cells (18 % on RC365 atomic); when a VLA policy fails to achieve the goal, the cell re‑invokes it until success, whereas each retry in tool calling incurs a full round‑trip with three camera images.
Listing 2 – Retry pattern from RC365 atomic (CloseToasterOvenDoor, seed 1)
Perception‑of‑its‑own shows up in 30 % of cells; a cell reads raw camera images and point clouds into NumPy arrays, computes masks, principal axes, and grasps autonomously, while tool calling receives only pre‑computed tool outputs.
Listing 3 – Perception example from RoboTwin 2.0 (hand‑over block, seed 100003)
Skills of its own appear in 2 % of cells (7 % call a previously defined helper); the agent writes reusable functions such as a guarded descent and reuses them later, eliminating the need for a separate tool call.
Listing 4 – Helper‑function skill from RoboTwin 2.0 (place object stand, seed 100002)
Looking‑when‑needed accounts for 20 % of cells; 84 % of cells request a camera image, but they do so conditionally, often only after a failed check, unlike tool calling which always fetches three images per move.
Single‑step cells (35 % overall, 54 % on RoboTwin 2.0) behave like a tool call: they perform one primitive and optionally request an image, which explains why the Code agent saves LLM calls on that benchmark.
Aggregating the statistics, a code cell runs on average 2.1 acting primitives (45 % run two or more) versus 0.65 primitives per tool‑calling LLM call, illustrating the efficiency gap.
Success‑rate numbers show the code agent solves 96 task instances while tool calling solves only 36; in many tool‑calling failures the VLA policy stalls, whereas the code agent can recover by recomputing grasps or geometry.
Cost accounting removes memory‑ and recipe‑related prompts, counts only LLM calls, and applies GPT‑6 Astra pricing ($10 M⁻^{1} input tokens, $1 cached input, $12.50 cache write, \$50 output). No call exceeds the 272 k‑token threshold that would double input cost.
Median first‑prompt sizes differ: tool calling uses 20 k tokens on LIBERO‑PRO versus 22 k for code; on RoboTwin 2.0 the numbers are 15 k vs 21 k, and on RC365 benchmarks 17 k vs 24 k.
Overall token‑ratio results across benchmarks are 3.1× (LIBERO‑PRO), 1.8× (RoboTwin 2.0), 1.6× (RC365 composite), and 1.1× (RC365 atomic), confirming the consistent advantage of programmatic primitive composition.
**Table 7.** The task instances of the main comparison. Each instance runs once with each agent.
**Table 8.** Tool calling with camera images after every move (the LIBERO-PRO baseline of Table 3) and only on demand, against code, on all 200 task instances of LIBERO-PRO. *On the 131 instances all three solved (means).
**Table 5.** The episodes of Figure 7: LLM calls and token consumption of each whole episode, with tool calling (Tool) and with code (Code); shaded: how many times fewer tokens code consumed.
**Table 6.** What the code agent’s cells do, over all 700 episodes of the main comparison (each cut at its budget), against what an LLM call of tool calling does on the same task instances. Acting primitives are moves, grasps, gripper commands and VLA calls. RC365: RoboCasa365.