GreenWAM: A World Foundation Model for Robotics
Gennady Sidorov, Valerii Kornilov, Denis Grankin, Polina Fedotova, Aleksey Postnikov
GreenWAM unifies 20 robot families into a single world foundation model for prediction and control.
How can we unify world modeling and robot policy learning into a single foundation model that handles diverse robot embodiments?
Robot datasets are fragmented across different bodies, joint layouts, and recording conventions, making it difficult to build a single model that generalizes across diverse hardware. GreenWAM solves this by mapping all robot states into a shared anatomical action space of 79 slots, training one generator to predict both video and motion, and splitting off an action expert for policy adaptation. As a world model, it achieves top-tier accuracy on WorldArena 1.0; as a policy, it reaches the highest zero-shot success on LIBERO-plus and outperforms existing methods on real humanoid hardware.
Paper Primer
The core challenge is the "correspondence problem": different robots have different joint layouts and grippers, meaning the same tensor dimension represents entirely different physical motions. GreenWAM replaces embodiment-specific projections with a shared anatomical codec that maps every robot's joints to a common set of slots (e.g., legs, torso, arms, fingers), ensuring that a specific slot always corresponds to the same physical joint motion regardless of the platform.
GreenWAM is a reasoner-generator architecture that treats world modeling and policy control as two sides of the same coin. The model is trained on seven modes—including video generation, forward dynamics, and inverse dynamics—using a rectified-flow objective. To create a policy, the authors split an action expert off the generator, allowing the expert to specialize in control while the generator retains its visual-dynamics knowledge.
GreenWAM outperforms existing world-action models and video generators on WorldArena 1.0.
The model achieves the highest trajectory, depth, and 3D accuracy among all compared methods on the benchmark's 1,000 test episodes. It reaches an aggregate score of 63.31, surpassing Motubrain (63.16) and FlowWAM-FiveAges (62.99).
Robot specialization significantly improves both world prediction and control performance.
Controlled experiments show that adapting from the GreenWAM foundation rather than its action-trained parent raises trajectory accuracy by 3.0 points and lowers video prediction error by 23%. Policy success rates increase by 5.4% overall, or 8.4% when utilizing the dedicated action expert.
Why is a shared anatomical action space superior to the embodiment-specific projections used in previous models?
Embodiment-specific projections leave the physical correspondence between different robots implicit or absent. By using a shared codec, GreenWAM ensures that a specific anatomical slot (like the left elbow) means the same joint motion on every robot, allowing one model to control 20 different families without robot-specific action parameters.
What is the advantage of splitting an action expert off the generator during policy adaptation?
Splitting the expert allows the policy to acquire separate parameters for control while maintaining access to the generator's visual-dynamics knowledge via joint attention. This architecture raises policy success by 3.3% compared to using a shared tower.
GreenWAM demonstrates that a single foundation model can serve as both a high-fidelity world simulator and a robust control policy across diverse robot embodiments, provided the action space is physically aligned rather than just projected.
Introduction and Motivation
We expose the gap between world modeling and policy learning and introduce GreenWAM to bridge it.
Robots require two complementary capabilities: a world model that predicts how their actions will alter the scene, and a policy that chooses those actions. In practice, models are built and evaluated for only one of these roles, and each robot records its actions in a format specific to its hardware.
Because existing approaches treat world prediction and control as separate problems and rely on robot‑specific encodings, we lack a unified system that can both forecast scene dynamics and generate actions for any robot body.
**Figure 1.** From robot experience to prediction and control. (a–b) Video, measured states and text from 20 robot families form aligned subtask examples. (c–d) A shared 79-slot anatomical action space connects robot motion to a joint video–action model built on Cosmos 3 Edge. (e) Forward prediction of GreenWAM for Unitree G1 and AgiBot G2A on held-out recordings (Section 5.1). (f) As a policy, the foundation adds a coupled action expert and is distilled for humanoid control. (g) GreenWAM against the best of three references on every axis (grey dashed outline). Left: policy, against $\pi_{0.5}$ [62], X-VLA [103] and FastWAM [95]; the $\pi_{0.5}$ CALVIN value is the supervised fine-tuned checkpoint reported by $\pi_{RL}$ [17], RoboDojo scores come from the official leaderboard [67]. Right: WorldArena 1.0 against Motubrain [56], FlowWAM-FiveAges [21] and UNIS [46]: three categories of the motion-capped score (Section 5.2; for the other models an upper bound) and three fidelity metrics. Every axis starts where its bar chart in Figures 9 and 11 starts, but not below half of its best value, and ends at the best value on that axis.
GreenWAM maps diverse robot state formats into a shared anatomical action space, enabling a single foundation model to predict scene changes and select actions across different robot bodies.
Related Work
We position GreenWAM among prior robot world‑action models and highlight key differences.
Table 1 summarizes the closest technical reports by the capabilities they expose—actions→video generation, video→actions inverse dynamics, joint video‑action policy prediction, long‑rollout memory, and real‑robot experiments. A check mark indicates the report presents the model in that role; the table does not rank quality.
Robotics specialization of world foundations began with Cosmos 3, which unifies visual generation, forward and inverse dynamics, and world–action modeling but limits itself to six embodiments and a gripper‑only action representation. GreenWAM expands this by retraining the foundation on measured joint and Cartesian trajectories from twenty robot families, aligning each subtask with video and text. Genie Envisioner similarly connects a shared world foundation to prediction, control and simulation, while DreamDojo and GigaWorld‑0 explore human video with latent proxy actions and generated robot experience respectively.
Cross‑embodiment policies face a correspondence problem. Prior systems such as Open X‑Embodiment, Octo, CrossFormer, HPT, X‑VLA, UniAct, RDT‑1B, and Green‑VLA each propose different mechanisms for mapping diverse proprioceptive and visual inputs into a shared representation. Cosmos 3 and GE‑Sim 2.0 use action maps projected into every calibrated camera view, which works for robots with fixed cameras but struggles with moving head or finger joints. GreenWAM instead passes measured joint and Cartesian states through a single unified projector covering 79 anatomical slots, enabling consistent control across humanoids, manipulators and fixed‑base arms.
From world prediction to policy, early approaches either generate a video plan and extract actions, or pre‑train on video then fine‑tune for joint prediction. UVA, UWM, DreamZero, LingBot‑VA and GE‑Act 2.0 exemplify these strategies. GreenWAM retains joint video‑action prediction and introduces a dedicated action expert copied from the generator at the split, preserving the parent function before further optimization. Temporal context is handled by four fixed‑position anchors encoded separately, allowing continuations that exceed 20 seconds on the WorldArena benchmark.
**Figure 2.** Robot data and action types of Cosmos 3 [58] and GreenWAM: shares of robot hours (top; AgiBot World supplies 81% of the robot hours of Cosmos 3, the largest GreenWAM family 15% of ours) and action representations (bottom).
GreenWAM Overview
GreenWAM unifies robot states into a common anatomical space and trains one generator for video and action prediction.
Robot datasets use heterogeneous joint orderings and scalings, making it hard to reuse a single model across embodiments.
GreenWAM translates each robot’s native joint readings into a common set of anatomical slots—like a translator that converts robot‑specific joint names into a shared anatomical dictionary—then feeds the unified tokens to a single generator that can predict video frames, robot motions, or both.
Encode each joint angle into a 2‑dimensional vector (e.g., linear projection) → shoulder → (0.52, 0.86), elbow → (0.78, 0.62), wrist → (0.17, 0.03).
Concatenate the three 2‑D vectors and pad zeros for the missing two slots, yielding a 5‑slot anatomical vector: [0.52, 0.86, 0.78, 0.62, 0.17].
Feed this vector to the generator; the generator outputs a video token sequence and an action token sequence.
The action expert splits off the action branch, producing predicted joint deltas ($\Delta$shoulder = 5°, $\Delta$elbow = ‑3°, $\Delta$wrist = 2°).
Decode the deltas back to native joint angles, updating the robot state to (35°, 42°, 12°).
The example shows how a fixed‑size anatomical vector can represent robots with different numbers of joints, enabling the same generator to handle both simple and complex embodiments.
Robotic datasets come in many formats; each robot family has its own joint ordering and scaling. GreenWAM solves this by inserting a lightweight codec per layout that projects the native joint vector into the 79‑dimensional anatomical space before the shared input projection.
During generation, filled tokens represent observed conditions, while dashed tokens serve as noised targets for the rectified‑flow objective. When predicting robot states, the model outputs deltas relative to the initial state q₀ (Equation 2) and a final decoder maps these deltas back to the robot’s native joint representation.
How does GreenWAM differ from a conventional world‑action model that directly predicts actions from raw joint inputs?
GreenWAM first translates raw joints into a shared anatomical space via a codec, then uses a single generator for both video and action prediction. A conventional model would either train a separate predictor per robot or operate on heterogeneous inputs without a unified representation, losing the efficiency and generality that GreenWAM provides.
Shared Anatomical Action Space
Robot datasets use incompatible joint layouts; a shared 79‑slot space aligns them.
Robot datasets disagree on joint order, units, sign conventions, and even which joints exist. GreenWAM resolves this pain by first establishing a common anatomical action space before any projection.
A fixed 79‑slot vector where slot $j$ always means the same joint motion (e.g., left‑shoulder roll) on every robot, regardless of that robot’s native naming or scaling.
Codec selects $\pi_\ell(1)=1$, $\pi_\ell(2)=2$, $\pi_\ell(3)=3$; all other slots get $m_{\ell j}=0$.
Sign flips are $\rho_{\ell 1}=+1$, $\rho_{\ell 2}=-1$ (the robot defines elbow pitch opposite to the canonical direction), $\rho_{\ell 3}=+1$.
Zero offsets are $o_{\ell 1}=0.0$, $o_{\ell 2}=0.1$, $o_{\ell 3}=0.0$.
Raw slots: $q_1=+1\cdot(x_1-0.0)$, $q_2=-1\cdot(x_2-0.1)$, $q_3=+1\cdot(x_3-0.0)$, $q_4=0$, $q_5=0$.
Assuming per‑layout statistics $\mu_{\ell j}=0$, $s_{\ell j}=1$ for $j=1\!-\!3$, the normalized slots are $\bar q_j = q_j$ for the three occupied slots and $0$ for the empty ones.
The example shows how a single codec can reconcile opposite sign conventions and missing joints while producing a uniform 5‑slot vector for the shared projector.
How does this shared‑slot approach differ from per‑embodiment projectors like those in Cosmos 3 Edge?
Per‑embodiment projectors learn a separate mapping for each robot family, so the same joint may occupy different latent dimensions across robots. The shared slot approach forces every robot to write into the same dimension, guaranteeing that “left‑shoulder roll” is identical on all platforms and allowing a single projector to be trained jointly.
**Figure 4 Shared 79-slot anatomical action space.** Top: the codec of one layout selects its native channels, puts them in slot order and flips signs and shifts zeros where needed, so that a slot means the same joint motion on every robot, for example the same positive direction of the left shoulder roll; three of the fourteen Worker channels are shown. A shared input projection maps the slot vector to state tokens. Bottom: slots filled by eight source layouts; empty slots are masked in the loss.
Joint Video-Action Modeling
Jointly train a single generator to predict video frames and robot actions across seven conditioning modes.
A robot clip contains both visual frames and joint states, yet we often ask four different questions: predict future video, predict future robot view, infer the trajectory that produced a video, and decide the next action. Training separate models for each question wastes data and parameters. Instead we train one generator that can answer all of them by switching which streams are treated as clean conditions and which are noised targets.
We treat video frames and anatomical state vectors as interchangeable streams; by designating any subset as observed (clean) and the rest as targets, a single transformer can learn to predict either modality from the other.
How does this differ from ordinary multi‑task learning where each task has its own loss?
Here the tasks are not independent; they share the same noise level $\sigma$ and the same velocity prediction $\hat v_{\theta}$. The loss couples them through the rectified‑flow objective, so learning on video streams directly improves state prediction and vice‑versa, unlike separate task‑specific heads that only share a backbone.
Compute the noised target: $x_{\sigma}= (1-0.5)\times10 + 0.5\times12 = 5 + 6 = 11$.
Compute the true velocity: $v = 12 - 10 = 2$.
Feed $x_{\sigma}=11$, the clean condition (none for video‑only mode), and $\sigma=0.5$ to the generator; suppose it predicts $\hat v_{\theta}=1.8$.
Mask $M_{s}=1$ (the stream is fully valid) and weight $\lambda_{s}=1$. The squared error is $(1.8-2)^{2}=0.04$; normalized loss = $0.04/1 = 0.04$.
For the state‑only mode, the same $\sigma$ and $\varepsilon$ are used, producing the identical loss term, demonstrating how a single noise level ties the two predictions together.
This tiny example shows that a single $\sigma$ forces the model to treat video and state streams as two views of the same underlying dynamics, rather than unrelated tasks.
**Figure 3.** GreenWAM architecture, shown for the world-action (WAM) mode on one held-out Green humanoid episode. The task instruction, the multi-camera canvas and the robot’s native joint state at $t=0$ enter a frozen reasoner, a trained generator for the video tokens and, in the adapted policy, an action expert for the state and action tokens, all sharing multimodal attention (the Cosmos 3 Edge mixture of transformers); the layout’s codec maps the native joints to the 79 anatomical slots before the shared input projection (Figure 4). In GreenWAM the state and action tokens pass through the generator; the action expert is split off it for policy adaptation (Section 6.1). Filled tokens are clean conditions and dashed tokens are noised targets; which streams are conditions changes with the mode (Figure 29). Predicted state targets are decoded relative to $q_0$ (Equation 2) and mapped back to native joint states. Video: ground truth (top) and the GreenWAM prediction (bottom). Frozen and trained refer to the robotics stage.
**Figure 29.** Clean conditions and noised targets in the seven modes, with each mode's share of training examples. Text (caption or instruction) is a clean condition in every mode. Forward dynamics, with and without history, conditions on $q_0$ and the clean future targets $\delta_k$ of Equation (2); inverse dynamics predicts absolute states including $q_0$. WAM predicts future frames and state offsets over about 1.07s at the effective frame rate of the source. Autoregressive modes predict the next whole-subtask block from four RGB anchors of the history.
Multi-View Observations and Temporal Context
We align multi‑camera views into a unified canvas and condition generation on four fixed visual anchors.
Training on heterogeneous robot rigs demands a uniform visual representation despite differing camera counts and layouts. Without a common canvas, the model would see wildly varying aspect ratios and spatial arrangements, breaking the shared anatomical action space.
Table 2 lists the seven training modes and their sampling probabilities; the multi‑view pipeline must serve all of them.
We fuse the head camera (top two‑thirds) and the two wrist cameras (bottom left‑right) into a single 640×640 canvas, then condition each generated video block on four RGB anchors placed at fixed fractions of the available history.
Resize head view to 426×640, each wrist view to 107×320, then place the head view at the top (426 px high) and the two wrist views side‑by‑side below (each 107 px high).
Anchor 1 encodes frame 0 (the very first head frame) via the VAE, producing latent $z_0$.
Anchor 2 encodes frame 40 (≈ 1/3 of the history) producing latent $z_{1/3}$.
Anchor 3 encodes frame 80 (≈ 2/3 of the history) producing latent $z_{2/3}$.
Anchor 4 encodes frame 119 (the most recent frame) producing latent $z_{\text{last}}$.
The generator receives the ordered tuple $(z_0, z_{1/3}, z_{2/3}, z_{\text{last}})$ and predicts the next video block.
Fixing anchor positions forces the model to learn a consistent temporal progression, so it can generalize from short history windows to longer ones without re‑learning where to look.
**Figure 5.** Training canvases of five robot families. From left: Unitree G1 and AIRBOT MMK2 with dexterous hands, RealMan AIDAL, AgileX Cobot Magic and AgiBot G2 with grippers; all recorded on real robots except AgiBot G2, which comes from simulation. The head camera occupies the top two thirds of the canvas, and the left and right wrist cameras share the bottom row (dashed lines); each view is resized to its panel without cropping. All five use the 640x640 canvas, the most frequent of the five canvas sizes.
**Figure 6. Robot data.** (a) The corpus: 20 robot families, 1 688.5 h after admission, 74% real, all two-armed, in three embodiment groups; each group is shown by a head-camera frame whose width is proportional to its hours. (b) Episodes are cut into subtask clips (atoms) at annotated subtask boundaries within length limits (Equation (10)) (1.44 million clips, 1 688.5 h); video-quality filtering removes idle, duplicated, cut and undecodable clips (1.43 million clips, 1 674.7 h). (c) One clip (RoboCOIN, Unitree G1 with dexterous hands, real), shown at the boundaries of its action phases. Its visual description and derived command are quoted in part, verbatim; its recorded state trajectory shows the three joints that move most of the left arm and of the left hand, each scaled to its own range. Arrows name the training modes that use each output, with their shares of examples (Table 2); the state trajectory serves all three action modes, as a condition of forward dynamics and as the target of world-action prediction and inverse dynamics, so the shares overlap.
**Figure 26** Hours per robot family, real-world and simulated. Combined names such as Piper / Cobot Magic denote datasets of one robot body grouped into one family (Section A). End labels are rounded. Hour shares differ from the sampling probabilities used in training (Section B.4).
Data Curation and Training Mixture
We curate diverse robot clips and define a weighted sampling scheme to train across modalities.
Curating robot video data is noisy: annotations miss boundaries, clips contain idle or duplicated frames, and many recordings lack usable captions. Without a disciplined curation pipeline the model would see far more failed or irrelevant examples than useful ones, biasing learning toward easy modalities.
We sample training examples proportionally to the square‑root of how many admissible sub‑tasks each robot cluster provides, ensuring rare robots and under‑represented modalities receive enough updates.
Compute raw weights: A = 2, B = 1, C = 1.
Normalize to probabilities: $p_A=2/4=0.5$, $p_B=0.25$, $p_C=0.25$.
Sample a mode uniformly (e.g., forward dynamics).
Draw a cluster according to $p$; suppose A is chosen.
Select a random admissible clip from cluster A and a compatible window; assemble the example.
The square‑root weighting gives cluster A twice the chance of being sampled compared to a naïve count‑proportional scheme, yet still leaves room for clusters B and C to appear regularly.
How does this sampling differ from simply sampling clips uniformly across all robots?
Uniform sampling would over‑represent the abundant two‑arm platform clips and under‑expose the humanoid dexterous hands, causing the model to specialize on the former. The square‑root weighting deliberately boosts the rare clusters, producing a more balanced representation of anatomical action space across robot morphologies.
**Figure 7** Captioner and command pass. The inputs are the head-camera clip of Figure 6c with its duration, subtask label, embodiment, effector and domain. The captioner (Qwen3.8-27B [64]) writes nine fields in one dialogue, one field per turn; the attempt record and the consistency field (red) are checks and never enter a training prompt. Blue dots mark the fields that form the visual description (fields from subjects through context, action phases and the temporal caption); green dots mark the fields the command pass reads (context, action phases and the temporal caption). Content gates test forbidden phrases, the effector against the registry, and contact and completion claims against the attempt record; a failing field is rewritten at most twice, and the whole dialogue is attempted at most four times. The command pass turns the action phases into one command per phase, keeping all phases in order; shown are the opening words of the three commands derived for the input clip. Bottom: the visual description conditions text-to-video and image-to-video, the command conditions forward dynamics and world-action prediction, and inverse dynamics receives a fixed instruction.
GreenWAM Training
Describes the unified training objective and scale for GreenWAM.
Training a single model on seven heterogeneous streams requires a loss that treats video and state equally while respecting the differing dimensionalities of body‑part groups.
The objective penalises the squared error of predicted velocities, normalising each body‑part group so that a larger group does not dominate the loss.
Compute errors: $e = (0.5, -0.2, 0.1)$.
Square errors: $e^{2} = (0.25, 0.04, 0.01)$.
Group‑wise sums: arms → $0.25+0.04=0.29$, legs → $0.01$.
Group‑wise averages: arms → $0.29/2=0.145$, legs → $0.01/1=0.01$.
Average across groups: $(0.145 + 0.01)/2 = 0.0775$ gives $\mathcal{L}_{\text{state}}$ for this step.
The normalisation prevents the arm group (with more coordinates) from overwhelming the loss, keeping the optimisation balanced across heterogeneous body parts.
How does this rectified‑flow objective differ from a standard mean‑squared error applied directly to all coordinates?
Standard MSE would sum squared errors over all coordinates, so a group with more joints would dominate the gradient. The rectified‑flow objective first averages within each group and then across groups, guaranteeing equal influence regardless of group size.
Training proceeds on 64 H100 GPUs with a global batch of 256 examples; parameters are sharded across the eight GPUs of each node and replicated across nodes, while attention checkpoints are applied selectively.
Sample a minibatch of 256 examples and apply the shared noise level $\sigma$ according to the Cosmos 3 Edge schedule.
Run the generator to predict velocities for each noised target stream.
Compute $\mathcal{L}_{\text{video}}$ and $\mathcal{L}_{\text{state}}$ using the masks and group normalisation described above.
Scale the summed loss by $\lambda$ = 10 and back‑propagate through the frozen reasoner and VAE.
Update only the generator parameters with Adam; checkpoint the optimizer state every few thousand steps.
Repeat until 36 500 optimizer steps (≈ 9.34 M sampled examples) are completed.
**Figure 8 Action following.** One start frame of Unitree G1 (head camera) predicted forward with the recorded state offsets scaled by 0, 0.5, 1 and 1.5; each column shows the halfway point (top) and the end (bottom) of the prediction. With zero offsets the bread stays on the table; with the other three scales it ends on the plate, and at the halfway point the larger the scale, the further the arm has moved.
To assess whether the world model respects commanded actions, we keep the first frame of each held‑out recording and multiply the recorded joint displacement by 0, 0.5, 1, and 1.5. Across all six robot families the generated motion grows monotonically with the command, and trajectory accuracy is later measured on the WorldArena benchmark.