Has GPT-6-Astra Achieved Embodied Intelligence?
SenseNova, Zhongang Cai, Ziang Cao, Tailai Chen, Han Chong Chua, Linjun Dai, Hanming Deng, Hokin Deng, Haiwen Diao, Chenjing Ding, Weichen Fan, Xiao Fu, Zerui Gong, Chenyang Gu, Zexin He, Zhi Hou, Bo Li, Qinyue Li, Quanzhou Li, Xiaowan Li, Zhichao Liao, Dahua Lin, Zilong Lin, Bo Liu, Chunxiao Liu, Pengfei Liu, Tianqi Liu, Ziwei Liu, Chen Change Loy, Lewei Lu, Jiapeng Luo, Yuexiao Ma, Liang Pan, Xingang Pan, Oscar Qian, Ziyi Shang, Yuyao Shao, Dawei Su, Dacheng Tao, Qingyi Tao, Quan Wang, Ruisi Wang, Xiaogang Wang, Ziyi Wang, Beichen Wen, Donghao Wu, Jiaxu Wu, Silei Wu, Xiaoming Wu, Zhonghua Wu, Hongcan Xiao, Weiye Xu, Atsushi Yamashita, Lei Yang, Zhe Yang, Baiqiao Yin, Wanqi Yin, Hanfei Zhao, Tianbo Zhong, Long Zhuo
GPT-6-Astra achieves high competence in robotic manipulation but remains limited by inference costs and physical-world constraints.
Can a general-purpose model like GPT-6-Astra achieve high-competence embodied control across diverse robotic platforms without task-specific training?
General-purpose models struggle to translate their reasoning capabilities into reliable robotic control, often failing to recover from physical errors or manage long-horizon tasks. The authors evaluate GPT-6-Astra across 12 benchmarks using a "System 2" reasoning harness that reviews and refines low-level controller actions, while allowing the model to learn from training demonstrations in-context. While the model approaches saturation on several tabletop benchmarks, it remains far from practical intelligence due to high inference costs, difficulty verifying physical success, and an inability to trade execution time for success in real-world settings.
Paper Primer
The paper investigates the gap between "embodied competence"—the ability to perform specific tasks in controlled settings—and "embodied intelligence," which requires reliable, efficient, and autonomous deployment. The core mechanism is a closed-loop harness where a high-level reasoner (GPT-6-Astra) reviews and modifies actions proposed by a low-level controller (System 1), effectively acting as a supervisor that can reject or refine movements before they are executed.
In-context learning (ICL) and extended interaction budgets significantly boost performance in simulation.
On the LIBERO-Ultra-Long benchmark, GPT-6-Astra nearly doubles the best published success rate, and on RoboDojo, success rates roughly triple when the interaction budget is increased 10x. Success rates approach saturation on LIBERO and SIMPLE benchmarks, reaching 81.9% on RoboDojo and 87.3% on DexJoCo.
Closed-loop collaboration between the reasoner and controller is essential for performance but creates a massive inference cost.
Closed-loop collaboration improves success rates by up to 20 percentage points over open-loop execution, but combining this with ICL consumes 19.3 million tokens per episode. Increasing the review horizon (the number of actions reviewed per model call) reduces token consumption by 63.5% while maintaining performance gains.
Why does the model perform so well in simulation but struggle in real-world deployment?
In simulation, the model can "buy" success by spending extra time and interaction steps to recover from failures. In the real world, physical constraints like safety stops, actuator overheating, and the fact that time cannot be paused during deliberation make this strategy unsustainable.
What is the "representation gap" the authors identify in robot control?
The authors observe that direct joint-angle control often underperforms compared to semantically aligned end-effector commands, suggesting that multimodal models reason more effectively when the action space matches their high-level understanding of the task.
Researchers should shift focus from benchmark saturation to developing agents that can verify physical state, recover from irreversible failures, and operate within real-time constraints without relying on expensive, task-specific demonstrations.
GPT-6-Astra and Embodied Intelligence
We frame the gap: moving from task‑specific models to a single general‑purpose embodied agent.
Current robotics research relies on a plethora of task‑specific models, each tuned to a narrow manipulation or navigation problem. This fragmentation limits scalability and hampers progress toward truly general embodied intelligence.
GPT‑6‑Astra is a single large‑scale language model that can be prompted with demonstrations to control a wide variety of robots without any additional fine‑tuning.
We evaluate GPT‑6‑Astra under a unified protocol that follows two simple principles. First, we allow in‑context learning (ICL) from training‑set demonstrations, constructing a knowledge package that the model can retrieve and adapt at test time. Second, we increase the execution budget to up to 10× the standard step limit, revealing that many tasks benefit from additional interaction steps.
Despite impressive scores—81.9 % on RoboDojo and 87.3 % on DexJoCo—the authors identify six core limitations: (i) ICL transfers poorly beyond the demonstrated tasks; (ii) inference cost and execution time far exceed practical deployment budgets; (iii) the model struggles to verify physical success; (iv) it often cannot pinpoint the cause of a failure; (v) irreversible failures are rarely recovered; and (vi) maintaining progress over long horizons remains difficult.
To unleash GPT‑6‑Astra’s potential, the authors ablate the harness design. Closed‑loop collaboration between “System 2” (GPT‑6‑Astra) and a low‑level “System 1” controller (e.g., $\pi$0.5) yields higher success rates than either component alone, but each round of System 2 supervision incurs substantial token cost. Aligning the action interface with the model’s reasoning—using end‑effector commands rather than raw joint angles—and adaptively chunking action sequences further improve performance while reducing unnecessary model calls.
Real‑world experiments on eight robot configurations confirm that GPT‑6‑Astra works across diverse embodiments, excelling at tabletop grasping but faltering on deformable‑object manipulation and long‑horizon tasks. The model exhibits active‑vision behavior, repositioning cameras to acquire needed observations, and can cooperate with a high‑frequency “System 0” locomotion controller to achieve humanoid manipulation that requires rapid low‑level control.
Deploying GPT‑6‑Astra in the physical world uncovers additional challenges: longer execution budgets no longer translate to higher success because safety constraints, mechanical limits, and real‑time requirements cap the number of interaction steps. Moreover, collecting successful trajectory demonstrations for ICL is costly and often infeasible outside simulation.
Looking forward, the authors propose three principles for the next generation of embodied benchmarks: (1) tasks should be derived from genuine human needs and evaluated on the utility of the outcome; (2) no in‑domain demonstrations should be provided, forcing agents to learn during deployment; and (3) the world should continue to evolve while the agent deliberates, making reasoning time and its physical consequences part of the evaluation.
The shift from specialized models to general‑purpose embodied agents defines the next frontier of robotics research.
Unified Protocol and In-Context Learning
Unified protocol evaluates GPT‑6‑Astra with frozen weights, ICL, and extended budgets.
Standard robotic pipelines require per‑task fine‑tuning, which limits scalability. Our large‑scale study instead adopts a unified protocol that keeps the model weights frozen and relies on in‑context learning and an extended interaction budget to recover from mistakes.
The protocol lets GPT‑6‑Astra absorb task demonstrations once, store them in a knowledge package, and then solve test episodes without ever updating its parameters.
How does ICL differ from conventional fine‑tuning of the model?
Fine‑tuning changes the model’s weights for each task, requiring a new training run. ICL leaves the weights untouched; it only writes a static knowledge package that the model reads at inference time, turning a one‑off learning phase into a reusable toolbox.
Step 1: Read the two demonstrations and create a JSON file listing the start pose, grasp pose, and target pose.
Step 2: Freeze the JSON as part of $K$.
Step 3: In a test episode, the model loads $K$, selects the first waypoint as the initial action, observes the resulting state, and proceeds to the next waypoint.
Step 4: If the grasp fails, the model re‑reads $K$ to choose an alternative grasp from the second demonstration.
The crucial benefit is that the same static package can be consulted repeatedly, eliminating the need for per‑task weight updates.
A knowledge package is a compact bundle of task‑specific artefacts that GPT‑6‑Astra can query during inference.
In what way is a knowledge package different from simply prompting the model with a long text?
A knowledge package separates raw data (e.g., trajectories, images) from executable code and verification logic. The model does not have to parse a monolithic prompt; instead it loads structured artefacts, which enables efficient reuse and systematic checks that a plain prompt cannot provide.
Step 1: The model reads the goal “stack block A on block B”.
Step 2: It loads the trajectory and executes the first waypoint using the helper script.
Step 3: After placement, the check module reads the depth image and confirms the height exceeds a threshold, signalling success.
Step 4: If the check fails, the model selects an alternative trajectory from the motion references and retries.
The package’s modular layout lets the model swap components (e.g., a different trajectory) without rebuilding the entire prompt, streamlining adaptation and recovery.
Select one of the twelve embodied benchmarks and its associated task category.
Run the ICL preparation phase: ingest the benchmark’s training demonstrations and construct a frozen knowledge package $K$.
Execute test episodes under the standard budget (1×) using fresh sessions that receive the task instruction, $K$, and the observation/action interfaces.
Repeat the same episodes under the extended budget (10×), preserving the interaction history from the 1× run.
Record success rates at each budget cutoff (1×, 2×, …, 10×) for comparison.
Aggregate results across benchmarks to assess the impact of ICL and budget extension.
**Figure 1.** Probing the capability upper bound of GPT-6-Astra across 12 benchmarks under a unified protocol that 1) allows for in-context learning (ICL) on the training set and 2) extends the step limit (budget) up to 10x. RoboLab has no training set and thus no ICL; BEHAVIOR is not extended to 10x due to its large official budget. SOTA cut-off date: October 12, 2026. Details in Tab. 2.
**Figure 2** In-context learning protocol. For each task, GPT-6-Astra studies the training data and writes a knowledge package $K$, which is frozen before testing. Each test episode starts a fresh session with $K$ and runs under the standard ($1\times$) or extended ($10\times$) budget. Model weights remain fixed throughout, and $K$ remains unchanged during testing, and no information passes between episodes.
**Figure 3** Contents of an ICL knowledge package. GPT-6-Astra decides what to retain, yet packages share six recurring components; file types are typical and content varies by benchmark. At test time, $K$ is read alongside current observations to retrieve, adapt and check actions.
**Figure 6.** Residual failures under ICL with the extended budget. Each pair shows the state before a key event and its outcome; purple arrows mark the relevant objects. (a) Verifying physical success: apparent or self-reported completion diverges from the actual task state. (b) Recovering from consequential actions: an early mishap, such as a spill or a dropped part, cannot be undone and derails later subtasks. (c) Preserving progress over long horizons: progress that later steps depend on, such as an open lid or a placed object, is lost and blocks the task.
Harness Design and Execution Budgets
We test which harness designs boost task success and how they affect token cost.
To understand how the interface between GPT‑6‑Astra’s reasoning and the robot’s sensors, actuators, and feedback affects performance, we run a series of controlled ablations.
The harness is the software layer that routes observations into the model, receives its action proposals, and feeds back execution outcomes.
How does this Harness differ from a naïve command‑only interface?
A naïve interface would send a single command and never observe the outcome. The Harness, by contrast, continuously feeds back execution results, allowing GPT‑6‑Astra to revise or reject actions, which is essential for handling failures and sub‑task progress.
Extending the review horizon to $L=4$ cuts token consumption by 63.5 % while raising success rate by 2.22 percentage points.
Closed‑loop $L=4$ uses 2.796 M tokens versus 7.664 M for per‑action review, a 63.5 % reduction, and achieves 25.83 % vs 23.61 % success.
Adaptive chunking raises success by 4.44 percentage points over the baseline while keeping token cost modest.
Baseline success 15.56 % vs adaptive chunk 20.00 %; token cost rises from 2.560 M to 1.146 M, a net reduction.
**Figure 5.** (a) Average per-task token usage and (b) average per-task time for each benchmark, stacked from bottom to top as ICL preparation, the 1x run, and the additional cost of extending to 10x. Here, 1x and 10x denote maximum action-step budgets, not token or time budgets. The relatively small additional token and time costs primarily reflect early termination of extended runs upon success or terminal failure, before the full 10x budget is exhausted. We observe that the vast majority of tokens consumed are cached input tokens (approximately 98% in our runs). Accordingly, we use a simple rule of thumb for cost estimation: 1M tokens ≈ 1 USD.
**Figure 7.** Token cost versus performance on RoboDojo for different System 1–System 2 collaboration strategies and action interfaces. Each point represents an experimental configuration, with token cost measured in millions and performance measured by the unweighted mean of category success rate. Dashed lines intersect at the Agentic GPT-6-Astra baseline reference point, defining four shaded quadrants.
Real-World Robot Experiments
Zero‑shot control on real robots reveals task‑dependent success and vision limits.
Recall that GPT‑6‑Astra adapts to new robotic tasks via in‑context learning without any task‑specific fine‑tuning.
GPT‑6‑Astra receives only a human‑written description of the task and the robot’s hardware, then directly issues commands to the robot without any prior demonstrations.
How does zero‑shot real‑world control differ from the simulation ICL protocol?
In simulation the protocol supplies a successful demonstration and a step budget; in the real‑world variant the model gets only a textual description (no demonstration) and a wall‑clock time budget, so it must reason from scratch and cannot rely on free extra steps.
Rigid pick‑and‑place succeeds in 17 of 18 real‑world trials, while deformable or contact‑sensitive tasks lag far behind.
Across six platforms the model achieves 94 % success on rigid pick‑and‑place, 50 % on cloth folding (2/4), 0 % on plug insertion (0/11), and 39 % on long‑horizon navigation (9/23).
**Figure 8.** Zero-shot real-world control, from simple to complex embodiments. Frames from released videos in temporal order. Chips give successes over all counted trials. (c) GPT-6-Astra re-plans after contact displaces the receiving bowl. (e) Dexterous grasping with the Wuji hands of the Tianji M6S, given human-written grasp scripts (the successful bottle run; the toy grasp succeeds in 3/3). The full gallery is in Fig. 13, and all tasks per platform in App. E.
**Figure 9** What GPT-6-Astra can do depends on what it can see. Flower insertion, main view and wrist views. (a) Single arm (Franka 3): the gripper holds the stem vertically, so its wrist camera cannot see the vase mouth, and the insertion fails. (b) Dual-arm Piper: the wrist view of the holding arm shows only the stem; GPT-6-Astra moves the free arm until its wrist camera sees the vase mouth (boxed frames: no vase, approaching the vase, vase mouth in view), and the insertion succeeds.
**Figure 12. GPT-6-Astra in the household proof of concept.** All runs are zero-shot, with contact-based grasping and no demonstrations. (a) While moving a book during tidying, the robot displaces desk items, knocks objects onto the floor and pushes the monitor out of place with its elbow. (b) Having opened the refrigerator to store the milk, the robot turns to carrying fruit to the table, and the door stays open from 300 s to 765 s of world time. (c) Making breakfast, the robot holds the refrigerator door open with one arm, but from where it stands the door blocks its other arm from the bacon inside; when it releases the handle to reach in, the door swings shut. (d) The running tap is not closed, and with the drain closed the sink fills until water spills down the cabinet onto the floor. (e) GPT-6-Astra watches an egg through the robot’s head camera and decides to turn the burner off once it has set; deciding takes 8 s of world time with the burner still on, and by the time the burner is turned off, about 90 s after the egg has set, the egg has charred. Purple outlines highlight relevant objects; timestamps denote elapsed world time. External camera views are used for visualization; the agent sees only through the robot’s onboard cameras.
**Figure 13.** Full gallery of zero-shot real-world tasks, grouped by outcome. Three frames per task in temporal order; chips give successes over all counted trials. (A) Tasks in which every counted trial succeeded; for the dual-arm Piper and the Galaxea R1 Pro tabletop tasks, counts are those of the reported rollouts (App. E.5 and E.7). A1 and A2, and A12 and A13, show two rollouts of the same task. (B) Tasks with moderate completion, ordered by success rate: some trials fail, or, as in B1, every counted trial succeeds but the result is imperfect (the folded cloth remains untidy). (C) Tasks in which no trial succeeded.
Extended Simulation Benchmarks (Long++, RoboDojo, RoboLab, Bench2Dex)
Extended benchmarks show GPT‑6‑Astra + ICL dominates across Long++, Ultra‑Long, RoboDojo, RoboLab, and Bench2Dex.
GPT‑6‑Astra + ICL attains near‑perfect success at extended budgets, reaching 99.6 % success rate across the nine Long++ and Ultra‑Long scenarios.
Macro‑averaged over the nine scenarios, the method achieves 99.6 % $SR$ and 99.8 % $AP$ at the $10\times$ step limit, surpassing all baselines.
**Table.** Comparison of GPT-6-Astra + ICL against published baselines (LiLo-VLA, $\pi_{0.5}$, OpenVLA-OFT) across Long++ and Ultra-Long benchmarks, reporting Success Rate (SR) / Average Precision (AP) percentages, token usage, and time requirements.
**Table.** Baseline scores and model sizes follow the default-instruction results on the official RoboLab leaderboard [161] (accessed 15 October 2026). Each listed baseline reports 1,200 evaluation episodes (120 tasks with 10 episodes per task). VLA: vision-language-action model; WAM: world action model. B denotes billions of parameters; – indicates that model size is not reported here.
RoboDojo evaluation covers 42 tasks grouped into Generalization, Precision, Long‑Horizon, Memory, and Open categories. Each task is run for five episodes (210 episodes total) with $1\times$ and $10\times$ step limits, using dual‑arm ARX X5 observations and joint‑level control. The method demonstrates robust performance across diverse task families, confirming its generality.
RoboLab benchmarks 120 language‑conditioned tasks spanning visual, relational, and procedural competencies. GPT‑6‑Astra achieves a 57.1 % success rate, outpacing the strongest baseline (FLUX 3 Action at 42.9 %) by 14.2 percentage points, demonstrating its advantage in high‑fidelity simulation.
Bench2Dex comprises 26 manipulation tasks evaluated on 12 arm‑hand embodiments under full randomization. GPT‑6‑Astra receives multi‑camera RGB and joint observations, selects absolute joint targets at 20 Hz, and is assessed with $1\times$ and $10\times$ budgets. The results show consistent gains over prior methods, confirming scalability across embodiments.
Towards a Next-Generation Benchmark
We reveal that GPT‑6‑Astra spends most time deliberating and that step budgets can be expanded tenfold.
GPT‑6‑Astra spends about 90 % of each run deliberating, leaving little time for physical actions.
Section 4.3 reports that 90 % of runtime is spent on inference rather than actuation.
We propose three principles for next‑generation embodied benchmarks: (1) tasks must be derived from real human needs, (2) requests should be judged by utility over outcomes rather than a fixed goal predicate, and (3) the world must keep running while the agent thinks.
Finding 1 shows that GPT‑6‑Astra focuses on the object in hand and loses awareness of the rest of the home, causing it to leave doors open or knock over items while pursuing the immediate sub‑task.
Finding 2 reveals that without demonstrations the model cannot discover how unfamiliar fixtures move or how its own body must be positioned, leading to repeated failed attempts such as a closed refrigerator door blocking the other arm.
Finding 3 demonstrates that decisions are made on stale observations; the agent turns off a burner only after the egg has already burned because the world continues to evolve during its deliberation.
Open challenges include building judges that can reliably score physical outcomes, ensuring simulation fidelity for long‑horizon tasks, handling the cost of a continuously running world, and reducing the expense of hour‑long deployments.
An embodied benchmark measures how well a robot performs tasks that require sensing, actuation, and physical interaction in a realistic environment.
**Figure 4.** Success rate versus step budget. The budget is expressed as a multiple of each benchmark's official step limit; dots show observed success rates and pink curves show fitted trends. We evaluate budgets up to 10x the official limit (1x for BEHAVIOR). The gray line marks the upper bound, which assumes that every episode that exhausts the evaluated budget without succeeding has the potential to succeed with an unlimited budget.
**Figure 10** Two specifications of a breakfast task. Left: the BEHAVIOR-1K activity `cook_bacon` [8], with its complete goal. The goal is tested against the current state at every step, and the episode succeeds at the first step at which it holds. Nothing requires the stove to be turned off, and since cooked holds once an object's maximum temperature has passed a threshold, it remains true for bacon that has burned. Right: what a judge for the request “make breakfast” needs to express. Any acceptable refinement earns credit, graded by what is served and by when, with a value that decays once the resident has left [56]; items count as served only if they never burned. Predicates are defined on physical quantities known only to the judge. The :never and :whenever clauses are the absence [38] and bounded-response [84] specification patterns, expressible in metric [85] and signal [108] temporal logic and checked over the whole trajectory on the world clock, which keeps running after the agent declares that it is done. All constants are illustrative.
**Figure 11** Building and using the household environment. Publicly available assets (1) and a floor plan (2) are composed into an interactive home. Coding agents place everyday items at real-world scale on named supports under human review (3), and every part is bound to physics: articulation, liquids, cloth and contact (4). A task states only a need (5): asked to make breakfast, the agent must work out the fridge, air fryer and toaster by itself, and is to be judged both on the request (Intent) and on obligations no one stated, such as closing the fridge (Duty). The agent sees only through the robot's onboard cameras, and its deliberation advances world time (6).
Additional Benchmarks (DexJoCo, Navigation) and Harness Ablation Study
Ablation of the Harness and benchmark results on DexJoCo, navigation, and other tasks.
We assess the impact of the Harness by reporting results on several embodied benchmarks and by ablating its components.
**Table.** Performance comparison across various robotic tasks, comparing the $\pi_{0.5}$ baseline with GPT-6-Astra at $1\times$ and $10\times$ scales, measuring success rate (%), token usage (millions), and time (minutes).
**Table.** Success rates, token usage, and time requirements for various tasks comparing $\pi_{0.5}$, GPT-6-Astra 1$\times$, and GPT-6-Astra 10$\times$.
**Table.** Performance comparison of different policies across various metrics including SR, ISR, CSR, and CGT.
**Protocol and metrics.** Each episode starts in a fresh session with a frozen ICL workspace prepared using training scenes that do not overlap with the test scenes. Test episodes do not share interaction history. Observations contain a single forward-facing RGB image at $512 \times 512$ resolution. Forward actions move 0.25 m. R2R uses $15^\circ$ left or right turns, while RxR uses $30^\circ$ turns and permits $30^\circ$ upward or downward camera tilts. Ground-truth routes, goal coordinates, and evaluation scores are withheld. STOP terminates the episode and succeeds within 3 m geodesic distance of the goal. The $1\times$ and $10\times$ limits are 500 and 5,000 actions. Extended evaluation is applied to initially unsuccessful episodes. Reported extended results retain prior successes and the better per-episode value of each metric. Metrics are averaged over each 100-episode subset. NE is the final geodesic distance to the goal. OS is the oracle success rate, measuring whether the trajectory ever enters the 3 m success region. SR measures successful completion, and SPL [2] weights success by path efficiency. nDTW [69] measures alignment with the reference trajectory using normalized dynamic time warping.