Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience
Tianqi Liu, Hanfei Zhao, Qifeng Liang, Hui En Pang, Jiarui Xu, Sayang Mu, Penghao Qiu, YingSheng Zhang, Jun Wang, Ziang Cao, Mingze Sun, Yukang Cao, Fangzhou Hong, Zhaoxi Chen, Ziwei Liu
Ropedia W0 unifies egocentric loco-manipulation across humans and robots using a shared kinematic interface.
How can we build a single, unified world model that predicts future egocentric video for both humans and robots, despite their vastly different physical embodiments?
World models struggle to generalize across diverse robot embodiments because they rely on embodiment-specific action spaces that fail to capture the complex interplay between locomotion and dexterous manipulation. Ropedia W0 solves this by using a shared kinematic interface that projects camera poses and manipulator skeletons into a common image-space format, allowing the model to learn from both human and robot data without needing custom control heads for each robot. On five cross-embodiment benchmarks, this unified approach achieves state-of-the-art predictive performance, with scaling analyses showing that larger human pre-training data directly improves transfer to robot embodiments.
Paper Primer
The core mechanism hinges on a unified kinematic control interface: it maps camera motion to dense Plücker-ray maps and manipulator motion to projected skeletons, effectively treating different embodiments as variations of the same visual-geometric task. This is like a universal translator for movement: it strips away the specific joint-space details of a robot or human and renders them as a shared "skeleton" that the video diffusion backbone can understand regardless of the underlying hardware.
Ropedia W0 achieves superior cross-embodiment generalization and data efficiency compared to existing world models.
On the Xp-Teleop benchmark, the A14B mid-trained model's zero-shot performance exceeds the full-data performance of all baseline models. The model reaches 19.83 PSNR (5B) and 21.98 PSNR (A14B) on cross-embodiment prediction without any target-specific post-training, significantly outperforming fine-tuned baselines.
Human egocentric data acts as a scalable foundation for robot world modeling.
Scaling analyses demonstrate that increasing human pre-training data improves prediction accuracy on both held-out human videos and robot benchmarks. With only 1% of post-training data, the 5B model outperforms the "official base" model trained on 100% of the data by 4.8–7.1 dB in PSNR.
Why use an explicit kinematic interface instead of just learning latent actions from video?
Latent actions inferred from video often entangle motion with other visual changes, making them difficult to generalize across embodiments. The explicit kinematic interface preserves embodiment-specific geometry while providing a consistent, spatially aligned signal that is easier for the model to map across different robot types.
What is the scope of this model — does it only work for manipulation?
No, it is specifically designed for "loco-manipulation," meaning it jointly models viewpoint changes (locomotion) and dexterous hand-object interactions. It is evaluated on tasks ranging from gripper-based manipulation to long-horizon humanoid teleoperation involving both walking and fine motor skills.
Ropedia W0 demonstrates that human egocentric video is a powerful, scalable source of supervision for embodied AI, provided the data is unified through a shared kinematic representation rather than embodiment-specific action heads.
Introduction and Motivation
Introducing Ropedia W0, a unified model bridging human and robot action spaces for loco‑manipulation.
World models must predict how actions reshape the environment, yet existing approaches struggle to jointly capture locomotion‑induced viewpoint changes and dexterous hand interactions across heterogeneous embodiments.
We need a single representation that can describe both where a camera moves (locomotion) and how a body or robot manipulates objects (hand poses), so a model can learn from any source—human video or robot demonstrations.
**Figure 1.** **Ropedia W0: a unified egocentric world model for cross-embodiment loco-manipulation.** Trained on Xp-Data (14 human and 13 robot public datasets plus in-house recordings; ~10k h human and ~3k h robot video), a single shared interface of camera poses and projected kinematic skeletons (center) lets one model predict future egocentric video across humans, gripper robots, and dexterous humanoids (left and right). Prediction quality on both human and robot evaluations improves as the amount of human-video pre-training is scaled up (bottom).
The essential gap is the mismatch between human and robot action spaces, which our unified kinematic interface eliminates.
The Xp-Data Corpus
We build Xp-Data, a unified cross‑embodiment dataset via a five‑stage curation pipeline.
Prior cross‑embodiment datasets lack metric camera poses and provide noisy hand keypoints, preventing a single world model from learning both locomotion and manipulation.
Xp-Data is a curated collection where every human or robot recording is annotated with synchronized RGB, metric camera trajectories, unified skeleton keypoints, and textual captions, all using the same schema.
Select frames 1–8 → resample to 81 frames by linear interpolation (each original frame repeats 10 times, last frame repeats 11 times).
Project the 3D hand keypoints into the image using the intrinsics, producing 2D coordinates for each interpolated frame.
Attach the same caption “picking up a cup” to all 81 frames, since the underlying action spans the original 4 s.
Store RGB, 2D keypoints, intrinsics, and caption as a single training clip.
This toy example shows how heterogeneous raw recordings are normalized to a common 81‑frame, 640 × 480 format, preserving both visual and kinematic information.
Video filtering removes clips with severe blur, poor illumination, or no visible hands, ensuring that downstream geometry and kinematic annotation remain reliable.
Multimodal annotation adds metric camera trajectories (via SLAM or DA3), hand skeletons (WiLoR + MANO for humans, URDF forward kinematics for robots), and hierarchical captions generated by a language model.
Quality checks verify cross‑modal consistency: camera trajectories are re‑projected for depth consistency, hand keypoints are cross‑validated with MediaPipe and MMPose, and captions are vetted by Gemini, Qwen, and human reviewers.
Training‑clip construction slices each verified sub‑task into a standardized 81‑frame clip (640 × 480, 16 fps), merging per‑frame captions into a single window‑level description.
An episode (robot) or clip (human) is defined as a contiguous source recording; a window is an 81‑frame segment extracted from that recording.
**Figure 2.** **Xp-Data curation pipeline.** Heterogeneous human and robot recordings pass through five stages. (1) *Data sources*: 14 public human egocentric datasets, 6,000 h of in-house four-camera recordings, and 13 public robot datasets. (2) *Video filtering* removes low-quality or task-irrelevant clips with a vision-language model. (3) *Multimodal annotation* produces geometry (metric camera poses via SLAM and DA3 [69]), kinematics (human hands via WiLoR [91] and MANO [96], robot keypoints via URDF forward kinematics, manual annotation, and correction), and semantics (hierarchical task, subtask, and atomic-action captions). (4) *Quality check* verifies completeness, correctness, and cross-modal consistency, pairing automated checks with a caption cascade (Gemini [25] annotation, Qwen [6] verification, and human review). (5) *Training-clip construction* cuts quality-verified subtasks into standardized 81-frame, 640 × 480, 16 fps multimodal clips, yielding the unified cross-embodiment corpus.
**Figure 3 | Overview of Xp-Data.** Top: caption terms. Middle: environment and task distributions for robot (inner ring) and human (outer ring) data. Bottom: unified skeleton annotations across human hands, dexterous robot hands, and parallel grippers.
Ropedia W0 Architecture
We describe the unified kinematic interface and its diffusion transformer architecture for cross‑embodiment loco‑manipulation.
Cross‑embodiment loco‑manipulation suffers because each embodiment expresses motion in its own joint space, making it hard for a single model to learn from both human and robot data. The trick is to map all motions into a common image‑space control signal, then let a diffusion transformer consume that signal.
The interface translates camera trajectories and articulated skeletons into aligned image‑space maps, so humans, grippers, and humanoid robots share the same visual conditioning while still preserving their unique geometry.
Compute $d_{1}(u)$ for each pixel using $R_{1}$ and $K_{1}=I$; the direction vectors rotate $30^\circ$ relative to the initial frame.
Form $c_{\text{cam}}^{1}(u)$ by concatenating $o_{1}\times d_{1}(u)$ (with $o_{1}$ at the origin) and $d_{1}(u)$, yielding a $4\times4$ Plücker‑ray map.
Apply forward kinematics $F^{(e)}$ to $(0^\circ,45^\circ)$, obtaining a 3‑D hand keypoint at $(0.5,0.5,1.0)$. Transform into the camera frame with $R_{c\leftarrow b}$ (identity here) and project with $\pi$, giving pixel $(2,2)$.
Render the keypoint into $c_{\text{skel}}^{1}$ as a bright pixel at $(2,2)$. Repeat for $t=2,3$ with the updated joint angles, producing a moving keypoint trajectory.
Result: a three‑frame pair of aligned condition maps where the camera ray map rotates consistently and the skeleton pixel follows the arm motion, illustrating how heterogeneous motions share a common image‑space representation.
The example shows that even though the robot has a different joint dimensionality, both camera and skeleton conditions occupy the same $4\times4$ grid, enabling a single transformer to process them jointly.
How does this interface differ from simply feeding raw joint angles and camera poses to the model?
Raw joint angles live in a high‑dimensional, embodiment‑specific space, and camera poses are 6‑DoF vectors. By projecting both into dense image‑space maps we obtain a fixed‑size, spatially aligned conditioning tensor that the transformer can attend to directly, avoiding the need for separate encoders or hand‑crafted alignment steps.
The conditioning architecture injects the camera and skeleton latents into the diffusion transformer by adding their projected tokens to the video token stream at every diffusion step. Optional wrist‑pose and wide‑FOV branches are added in the same additive fashion, but their projection matrices are zero‑initialized and gated off when annotations are missing.
Pre‑train on large‑scale egocentric human videos using the unified kinematic interface.
Sample batches that contain either human or robot data with equal probability.
For robot samples, compute $c_{\text{cam}}$ and $c_{\text{skel}}$ from the robot’s URDF and joint states, then feed them through the same conditioning pipeline.
Optimize the diffusion loss jointly over image reconstruction and conditioning prediction.
Periodically evaluate on the Xp‑Teleop benchmark to monitor cross‑embodiment transfer.
Distillation proceeds in two phases. First, a block‑causal four‑step generator is obtained via Causal Forcing. Then LongLive extends this generator to arbitrary video length by matching its latent predictions to those of a frozen bidirectional denoiser, normalizing the gradient by the real denoiser’s correction magnitude.
**Figure 4.** **Overview of Ropedia W0.** Left: Human pose annotations and robot forward kinematics provide kinematic structure and wrist poses; manual robot-keypoint annotation and correction align robot skeletons with the observed images. Middle: Camera trajectories are encoded as Plücker-ray maps, while manipulation is represented by skeleton maps, optionally augmented with wrist-pose and wide-FOV renderings. Right: A video diffusion transformer predicts future frames from an initial image, a text prompt, and these control signals. Pixel-aligned conditions are fused through additive projections, while wide-FOV features are injected via cross-attention.
Table 1 compares existing humanoid teleoperation datasets, highlighting that Xp‑Teleop uniquely provides synchronized long‑horizon loco‑manipulation with the full set of modalities (RGB, depth, IMU, whole‑body joints, calibration, and keypoints), which is essential for training the unified kinematic interface.
Benchmark and Evaluation
Defines the Xp‑Teleop benchmark and the evaluation protocol across cross‑embodiment datasets.
We evaluate on 100 Xp‑Teleop windows (50 held‑out scenes, 50 unseen scenes) to assess long‑horizon prediction.
Section 6.1 specifies the split of 100 evaluation windows for Xp‑Teleop.
It is a humanoid teleoperation dataset that couples whole‑body locomotion with dexterous hand manipulation in egocentric video, offering synchronized multimodal streams for each episode.
**Figure 5.** **Overview of Xp-Teleop.** About one hour of dexterous-humanoid teleoperation spanning six everyday household activity groups, from tidying and food preparation to laundry and cleaning (a). Each take is a long-horizon episode that chains locomotion with manipulation, such as putting objects into a box or folding a jacket on the bed (b). Takes are recorded as a single synchronized multimodal stream (egocentric RGB-D, 6-DoF camera pose, projected hand/robot skeletons and masks, 29-joint robot state, time-aligned language segments, per-hand tactile, and head IMU; c), and their 543 annotated segments are dominated by loco-manipulation verbs such as reach, walk, and grasp (d).
Xp‑Teleop is essential because it supplies the long‑horizon, whole‑body data needed to evaluate whether a world model can predict coordinated locomotion and manipulation over extended episodes.
Performance and Ablations
Ropedia W0 sets new performance records on egocentric video and loco‑manipulation benchmarks.
Ropedia W0 (A14B, midtrain) closes the seen–unseen PSNR gap to just 0.06 dB on the $\Psi$₀ benchmark, the smallest gap among all evaluated models.
Table 4 reports a 0.06 dB gap for the midtrain variant, outperforming the next‑best baseline by a clear margin.
Across the three benchmark families—egocentric video (Table 2), DreamDojo transfer (Table 3), and world‑model evaluation (Table 4)—Ropedia W0 consistently ranks first or second on every metric, demonstrating both cross‑embodiment generalization and data‑efficient adaptation.
**Table 2.** Results on a 200-clip EgoSuite and OpenAoe evaluation set (100 clips each). Zero-shot egocentric human-video prediction, reported as (a) camera- and hand-action following (trajectory errors ATE/RPE/RRE, hand position error HandL2, and PCK-based hand-keypoint accuracy with detection rate Det) and (b) video prediction and generation quality (PSNR/SSIM/LPIPS, VBench consistency/dynamics/aesthetics, and FVD). ATE/RPE are in cm, RRE in degrees, HandL2 in pixels, and PSNR in dB; PCK/PCKm/Det and Cons./Dyn/Aes are reported as percentages. Best bold, second-best underlined; full protocol in Sec. C.1.
**Table.** Quantitative comparison of video generation models on $\Psi_0$, $\omega$-HOME, and Xp-Teleop benchmarks.
**Table 4.** World-model evaluation on $\Psi_0$ [117], $\omega$-HOME [65], and Xp-Teleop under three post-training settings: no post-training (zero-shot transfer), few-shot, and full-data, reporting PSNR, SSIM, and LPIPS. For Ropedia W0 we report both the pretrain (human-only) and midtrain (human + robot) stages at the 5B and A14B scales. Best bold, second-best underlined.
Downstream Applications
Ropedia W0 powers policy evaluation, planning, world‑action modeling, and live teleoperation.
Ropedia W0 achieves 92 % task progress in simulation and 55 % on the real Unitree G1 robot, far surpassing random selection (53 % / 26 %) and the best fixed‑checkpoint baseline (85 % / 45 %).
Figure 9c,d show the planning bar (red) at 92 % (simulation) and 55 % (real) compared to the random and fixed‑checkpoint baselines.
**Figure 9 | Policy evaluation and world-model-based planning.** (a,b) Predicted vs. ground-truth task-progress scores in simulation and on the real robot (dashed: perfect agreement). (c,d) Task-progress scores for individual policies and world-model-based candidate selection; *Random* selects one candidate rollout uniformly at random, so its expected score equals the mean over candidates.
Distillation and Scaling
Ablation analysis of key components and visual evidence of Ropedia W0 performance.
This appendix reports ablation experiments that isolate the contribution of each component of Ropedia W0, and presents visualizations that illustrate the model’s behavior across the evaluated tasks.
Causal Distillation transfers knowledge from a teacher model to a student by enforcing that the student’s predictions respect the causal ordering of actions observed in the teacher’s trajectories.
**Figure 10.** **World-action modeling.** (a) Real-robot executions of the policy initialized from Ropedia W0, shown as four chronological pairs of recorded keyframes with different objects, containers, and placements. (b) Aggregate real-robot task progress for Ropedia W0 initialization, FastWAM [131], and the $\Psi_0$ [117] policy.
**Figure 11.** Live teleoperation with Ropedia W0. On-site recordings show the headset user and the monitor displaying model-generated feedback. Five chronological keyframes show interactions with human (top) and robot (bottom) embodiments.
**Figure S1.** Data curation funnel for Xp-Data. Robot and human egocentric recordings pass through source screening, motion annotation, and quality control; gray bands mark data removed at each step, and the surviving streams form the unified corpus of ~3,000 robot hours and ~10,000 human hours.
**Figure S2.** Motion annotations on the egocentric human data. Projected 21-point hand skeletons are overlaid on each frame, with the two hands drawn in different colors; the inset shows the camera trajectory of the surrounding window.
**Figure S3.** Example frames from the robot data in Xp-Data. The grid spans dexterous five-finger hands, three-finger hands, and parallel grippers on single-arm, dual-arm, and humanoid platforms, across household, retail, industrial, and tabletop scenes.
**Figure S4.** Motion annotations on the robot data. Image-aligned skeletons combine embodiment-native FK projections with manual keypoint annotation and correction (Sec. 3), covering dexterous hands and grippers; the inset shows the camera trajectory of the surrounding window.
Model Scaling and Backbones
We detail the backbones, distillation pipeline, and evaluation protocols.
Our experiments use two Wan2.2 backbones. The dense variant, Wan2.2‑TI2V‑5B, contains roughly $5\,\text{B}$ parameters, while the expert mixture, Wan2.2‑I2V‑A14B, activates a high‑noise and a low‑noise expert for an effective $14\,\text{B}$ parameter count.
The Wan2.2 family provides a video diffusion transformer that can be swapped between a single dense model and a two‑expert mixture, letting the same architecture operate efficiently across very different noise regimes.
Distillation proceeds by first extracting a 4‑step autoregressive generator using Causal Forcing, then extending it to streaming via LongLive. Asymmetric DMD trains the causal generator with a frozen bidirectional real‑score teacher and an online fake‑score model, both initialized from the bidirectional camera‑skeleton teacher.
On a single GPU (640×480), the 4‑step student produces a 125‑frame clip in about $2\,\text{s}$ of autoregressive generation and $6\,\text{s}$ total including VAE decoding (≈21 fps). The 50‑step teacher needs roughly $41\,\text{s}$, yielding a $7\times$ end‑to‑end speed‑up, and the first block streams in ≈$1.4\,\text{s}$.
Wide‑FOV skeleton conditioning addresses the frequent cropping of manipulators in head‑camera views. In the WorldArena test set, $99.4\%$ of frames have the narrow skeleton touching the image border; a $1.5\times$ wider field‑of‑view render raises in‑frame coverage from $0.37$ to $0.56$.
Example 1 shows border‑band keypoint error dropping from $8.9\,\text{px}$ to $6.9\,\text{px}$, with the wide branch matching the ground truth on $91\%$ of frames. Example 2, where the arm extends beyond the camera view, reduces error from $14.0\,\text{px}$ to $11.7\,\text{px}$ and aligns with ground truth on $93\%$ of frames.
Human egocentric video evaluation uses 200 clips (100 per dataset) sampled across motion strata. Metrics include ATE (cm), RPE (cm), RRE (°), hand PCK@0.1 and PCKm@0.1 (percent), detection rate, consistency, dynamics, aesthetics, PSNR (dB), SSIM, LPIPS, and FVD variants. Camera poses are recovered with DA3, hand keypoints with MediaPipe (confidence 0.3).
Scaling curves (Figs. 6‑8) plot performance versus human‑video pre‑training hours on a log scale. Held‑out human video (Fig. 6) shows ATE, RPE, RRE, and LPIPS decreasing with more data, while PCK, PSNR, and SSIM improve. Transfer scaling (Fig. 7) compares three methods on three benchmarks, with $\omega$‑HOME consistently leading. Cross‑evaluation (Fig. 8) repeats the trend across three datasets, confirming the scaling behavior.
Additional quantitative results (Tables S2, S3, S4) break down performance per dataset, show seen/unseen robot splits, and report long‑horizon rollouts. The 4‑step distilled student matches or exceeds the 50‑step teacher across all rollouts while running $7\times$ faster, confirming the efficacy of the distillation pipeline.
**Figure 6.** Scaling on held-out human video. Zero-shot evaluation of the 5B model on 200 held-out egocentric clips versus human-video pre-training hours (log scale). The eight panels report camera ATE, RPE, and RRE; hand PCK@0.1 and PCKm@0.1; and PSNR, SSIM, and LPIPS. PCK counts missed hand detections as failures, whereas PCKm scores matched hands only; see Sec. C.1 for the evaluation protocol. Dashed lines indicate fitted trends.
**Figure 7 | Human-to-robot transfer scaling.** PSNR, SSIM, and LPIPS on $\Psi_0$ [117], Xp-Teleop, and $\omega$-HOME [65] versus human-video pre-training hours (log scale); each point is a pre-training checkpoint post-trained on the target benchmark before evaluation. Colors and marker shapes distinguish benchmarks; lines connect measured checkpoint results.
**Figure 8 | Cross-evaluation scaling on the DreamDojo [34] benchmark.** PSNR, SSIM, and LPIPS on the In-lab, EgoDex [46], and DreamDojo-HV evaluation sets versus human-video pre-training hours (log scale), under an identical GR-1 post-training budget; the largest-scale point uses Ropedia W0 (5B). Colors and marker shapes distinguish evaluation sets; lines connect measured checkpoint results.
**Figure S5.** Distillation of Ropedia W0 into a few-step causal student. Top: causal distillation initializes the few-step autoregressive generator. Bottom: asymmetric DMD updates the causal generator $G_\theta$ using a frozen bidirectional real-score teacher and an online-trained bidirectional fake-score model. The diagram illustrates the A14B MoE backbone; the 5B model follows an analogous procedure with a dense backbone. LongLive subsequently extends the few-step model for long-video streaming (Fig. S6).
**Figure S6.** LongLive-based streaming with the distilled student. Successive segments reuse the previously generated visual context while accepting updated controls (skeleton, camera trajectory, and text). Each block is produced by the few-step causal student, so the per-segment cost is bounded while retained context carries information across segments, enabling real-time long-horizon rollout.
**Figure S7 | Wide-FOV skeleton branch, example 1** (four moments in time order; columns labeled above). Wide-FOV skeleton conditioning improves the alignment of the visible arm near the image boundary. Here the outer border-band keypoint error drops from 8.9 to 6.9 px, and the wide branch is closer to the ground truth on 91% of frames.
**Figure S8.** Wide-FOV skeleton branch, example 2 (four moments in time order; columns labeled above). When the arm extends beyond the camera view, wide-FOV skeleton conditioning improves the prediction of its visible portion. Here the border-band keypoint error drops from 14.0 to 11.7 px, with the wide branch closer to the ground truth on 93% of frames.
**Table S2.** Per-dataset results on EgoSuite and OpenAoE. Each set contains 100 clips from Table 2. ATE is in cm and PSNR in dB. Best and second-best bold and underlined.
**Table S4.** Long-horizon rollout on $\omega$-HOME ($\approx 32$ s, zero-shot; 12 chained windows). Overall metrics are frame-averaged; per-segment PSNR at $\approx 1/14/31$ s shows degradation over the rollout. Best bold, second-best underlined.
The table compares various models and initializations across "Overall" metrics (PSNR, SSIM, LPIPS) and "Per-segment PSNR" (seg. 1, seg. 6, seg. 12).
Qualitative Analysis and Limits
We examine how design choices and pretraining affect Ropedia W0’s cross‑embodiment loco‑manipulation performance.
Ropedia W0 maps heterogeneous human and robot actions into a unified kinematic interface, enabling a single video diffusion model to learn from diverse datasets. This section reports how different conditioning channels, pretraining choices, and model scales influence that capability.
**Figure S9.** Qualitative comparison on WorldArena (left: drawer + box; right: cup onto coaster). Each panel: five chronological frames (rows); columns are GT | Ropedia W0 (A14B) | BWM (Table S8). Ropedia W0 follows the commanded arm trajectory and completes the task like the ground truth (opening the drawer and placing the box, precisely seating the cup), whereas BWM's arm drifts and under-moves (it never opens the drawer).