RynnBrain 1.1: Towards More Capable and Generalizable Embodied Foundation Model

Kehan Li, Bohan Hou, Minghao Zhu, Tianyi Zhang, Zesen Cheng, Zhikai Wang, Sicong Leng, Xin Li, Xiao Lin, Biying Yao, Minghua Zeng, Jiangpin Liu, Ronghao Dang, Jiayan Guo, Siteng Huang, Haoyu Zhao, Heng Ping, Yaxi Zhao, Kexiang Wang, Tong Lu, Shengke Xue, Jiahao Tang, Yulei Wang, Zejing Wang, Jianwei Gao, Shijian Lu, Chengju Liu, Jianfei Yang, Mingxiu Chen, Deli Zhao

RynnBrain 1.1 integrates 3D grounding and contact-point prediction into a unified embodied foundation model family.

Embodied models often struggle to bridge the gap between high-level visual understanding and the precise spatial requirements of real-world robot manipulation. RynnBrain 1.1 addresses this by incorporating explicit 3D bounding box prediction and contact-point localization directly into the model's autoregressive training, using a unified action space to support heterogeneous robot embodiments. The 122B-A10B model outperforms existing open-source and proprietary models on key embodied benchmarks, while the 2B and 9B variants demonstrate strong native 3D grounding capabilities.

Paper Primer

The core move is the transition from unconstrained language generation to a physically grounded output space where the model predicts spatial tokens—such as 3D coordinates and contact points—alongside text. This is like a navigator who, instead of just describing a destination, simultaneously plots the exact GPS coordinates and the specific angle of approach required to reach it.

To handle diverse hardware, the authors implement a unified cross-embodiment action space where embodiment-specific masks activate only the dimensions relevant to a given robot's joints or grippers. This allows the model to learn shared manipulation patterns across different platforms without requiring explicit alignment of their unique control interfaces.

RynnBrain 1.1-122B-A10B achieves state-of-the-art performance on major embodied cognition benchmarks.

Outperforms proprietary and open-source models on VSI-Bench, MMSI, and RefSpatial-Bench. Significant gains in reasoning-intensive tasks, with MindCube scores increasing from 61.7 (2B) to 89.6 (122B-A10B).

Embodied pretraining is a prerequisite for reasoning-intensive spatial tasks.

Scaling analysis shows that while general cognition improves with model size alone, reasoning-intensive tasks degrade in pure VLMs (Qwen3.5) but improve steadily in RynnBrain 1.1. A 38.6% improvement in reasoning-intensive cognition for RynnBrain 1.1 versus a 39.2% decline for the baseline Qwen3.5.

Why move from grasp-rectangle detection to contact-point prediction?

Rectangle detection is often poorly aligned with robot action because it relies on arbitrary image-space extents that vary by dataset. Contact-point prediction provides a functional spatial anchor and orientation cue that is more directly usable by downstream robot policies.

What is the scope of the 3D grounding capability?

Native 3D grounding is currently implemented for the 2B and 9B model scales. It enables the model to predict 3D bounding boxes—including center position, dimensions, and orientation—directly from a single image and camera intrinsics.

The paper demonstrates that for embodied tasks, data-driven spatial supervision and model scaling are complementary: embodied pretraining ensures that scaling produces meaningful spatial reasoning rather than just more confident, inaccurate language.

Researchers can now use RynnBrain 1.1 as a robust initialization for downstream VLA policies, benefiting from a unified action space that simplifies cross-embodiment deployment.

Abstract and Introduction

We introduce RynnBrain 1.1, an embodied foundation model family addressing physical grounding gaps.

Embodied intelligence must go beyond recognizing objects and parsing language. A robot needs to perceive objects, reason about spatial relations, locate actionable regions, and turn that understanding into executable motions. Existing multimodal large language models excel at image‑question answering but fall short on 3D structure, viewpoint reasoning, and direct robot‑action grounding.

Contact‑point prediction supplies a precise grasp center and gripper orientation, while 3D grounding adds explicit spatial coordinates, both turning visual understanding into robot‑ready geometry.

A single action vector groups body‑part‑specific commands, and per‑robot masks activate only the dimensions that each robot actually possesses.

**Figure 1.** We introduce RynnBrain 1.1, a model family comprising three variants—2B, 9B, and 122B-A10B—together with the post-trained RynnBrain-VLA. RynnBrain 1.1 achieves leading performance across a broad suite of benchmarks, particularly in cognition, localization, 3D grounding, and contact-point prediction. RynnBrain-VLA demonstrates significant promise in enabling action prediction across diverse embodiments within a unified action space.

Our contributions are threefold: (1) we release the RynnBrain 1.1 family across three scales; (2) we add contact‑point prediction and native 3D grounding to the pretraining pipeline; (3) we build RynnBrain‑VLA with a unified action space and embodiment‑specific masking, achieving superior real‑world performance and gains from joint multi‑task, multi‑embodiment training.

Optimization and Hyperparameters

We outline the loss objective, hyperparameter choices, and data composition for RynnBrain 1.1.

The model is trained by maximizing the likelihood of each target token given its prefix and visual context, but scaling this objective to billions of parameters forces careful choices of learning rates, batch sizes, and data composition. Selecting those hyperparameters cannot be done analytically; the authors run small‑scale experiments on representative corpus subsets to find scale‑appropriate values. Moreover, the pretraining mixture must balance general multimodal corpora with newly added cognition and robot‑oriented datasets to avoid over‑fitting any single modality.

**Table 1.** Hyperparameters for RynnBrain 1.1 models.

**Table 1.** Multi-dimensional Cognition Data. For embodied cognition, RynnBrain 1.1 follows the task taxonomy of RynnBrain [20], covering object understanding, spatial understanding, counting, OCR, and egocentric task understanding. Object understanding data include RynnBrain-Object, RefCOCO [72], Google Refexp [44], Osprey-724K [74], DAM [39], and VideoRefer-700K [75], which provide object-centric supervision for fine-grained recognition and reasoning. Spatial understanding data include SenseNova-SI-800K [10], VSI-590K [69], VLM-3R [22], and RynnBrain-Spatial, supporting reasoning over spatial relations, viewpoints, and scene structure. For counting and OCR, we use RynnBrain-Counting, Molmo2 [15], RynnBrain-OCR, and the newly added Llama-Nemotron-VLM OCR data. For egocentric task understanding, we use EgoRe-5M [52], EgoTaskQA [33], Env-QA [25], QAEgo4D [26], RoboVQA [59], Robo2VLM [12], and ShareRobot [32]. These datasets are converted following the RynnBrain instruction format to provide broad supervision for object-centric, spatial, textual, and first-person task understanding.

Scaling Analysis

Scaling analysis reveals non‑uniform growth of embodied abilities and the pivotal role of embodied pretraining.

RynnBrain 1.1 investigates how embodied abilities evolve as model size grows while keeping the training pipeline fixed. By aligning each RynnBrain variant (2 B, 9 B, 122 B‑A10 B) with a raw Qwen3.5 baseline of the same scale, the authors expose systematic differences that standard vision‑language scaling studies miss.

Explicit spatial and temporal supervision is injected during training, giving the model a grounded sense of physical layout and motion.

Across the three benchmark groups the scaling curves diverge. (1) General embodied cognition improves monotonically for both families, and the RynnBrain‑Qwen gap steadily shrinks. (2) On reasoning‑intensive tasks RynnBrain gains +38.6 % while Qwen3.5 drops ‑39.2 %, widening the gap from 18.2 to 50.8 points; larger language models amplify weak visual‑spatial cues into over‑confident errors. (3) For embodied localization both models improve, yet even the largest Qwen3.5 lags behind the smallest RynnBrain 1.1, showing that explicit coordinate supervision outweighs raw parameter count.

**Table 5.** Comparison with large-scale open-source and proprietary models on embodied cognition and localization benchmarks. * denotes results from our reproduction.

Pretraining Data and Tasks

RynnBrain 1.1 integrates multi-modal cognition, 3D grounding, and contact-centered action data to enable embodied reasoning.

RynnBrain 1.1 expands its pretraining corpus to bridge the gap between high-level semantic understanding and low-level physical interaction. By unifying diverse datasets—ranging from egocentric task videos to synthetic 3D-grounded assets—the model learns to ground natural language instructions in both spatial and physical action spaces.

To move beyond 2D image understanding, the model learns to predict 3D bounding boxes directly in the camera coordinate system, grounding objects in physical units like meters and radians.

Standard rectangle-based grasp annotations are often ambiguous and embodiment-dependent. RynnBrain 1.1 replaces these with a compact contact-centered anchor $(p, \theta)$ that focuses on the functional grasp point and orientation.

The annotation is serialized as the token sequence: <grasp pose> 500, 500, 0.785 </grasp pose>.

The model receives the RGB image and the instruction "grasp the mug handle."

The model predicts the center $(500, 500)$ and angle $0.785$, which the downstream policy uses to align the robot end-effector.

By predicting a point and an angle rather than a rectangle, the model avoids learning dataset-specific gripper widths, making the grasp interface more robust across different robot embodiments.

For manipulation planning, RynnBrain 1.1 utilizes a structured format that pairs high-level task instructions with grounded sub-task descriptions. By incorporating datasets like Galaxea-G0 and augmenting them with global task descriptions generated via Gemini 3.1 Pro, the model learns to decompose complex instructions into actionable, grounded sequences.

Vision-Language-Action Post-training

Post‑training equips a single policy to control diverse robots in real time.

Post‑training turns the pretrained vision‑language model into a robot controller and simultaneously reveals how well the base model can reason about embodied tasks.

The policy treats a robot trajectory as a diffusion process and predicts short “action chunks” while keeping earlier tokens in a key‑value cache so inference can reuse past computation.

Why does putting actions at the end enable KV‑cache reuse?

Only the tokens before the action chunk generate keys and values; when the chunk shifts forward, those keys/values stay valid, so the model can attend to the same cached prefix without recomputing the expensive vision‑language embeddings.

All robots share a single $81$‑dimensional vector; each robot’s mask zeroes out dimensions it cannot actuate, letting one policy output a universal action representation.

Robot A’s mask = [1,1,1,1, 1,1,1,1, 0,0,0,0]; Robot B’s mask = [1,1,1,1, 0,0,0,0, 1,1,1,1].

The policy outputs a $12$‑D vector $v = (0.2,0.5,-0.1,0.0,\;0.3,-0.2,0.1,0.4,\;0.0,0.0,0.0,0.0)$.

Applying Robot A’s mask yields active arm‑hand payload $(0.2,0.5,-0.1,0.0,\;0.3,-0.2,0.1,0.4)$; the gripper part is zeroed.

Applying Robot B’s mask yields $(0.2,0.5,-0.1,0.0,\;0.0,0.0,0.0,0.0,\;0.0,0.0,0.0,0.0)$ for arm and gripper (hand zeros).

The same prediction can drive two robots with disjoint hardware simply by zero‑masking the irrelevant dimensions.

How does the mask avoid interfering with gradients for dimensions a robot cannot use?

The loss is summed only over active entries; masked entries have zero gradient, so they remain unchanged while other robots update the shared groups.

The policy is decoupled from robot‑specific controllers by inserting two thin layers: a control loop that interpolates the model’s chunk and an embodiment layer that translates standardized actions into hardware commands.

Why is a separate high‑frequency interpolation loop needed?

Robots typically require command updates at hundreds of hertz for smooth motion, but running the diffusion model at that rate is infeasible; the interpolation loop bridges the gap without additional model calls.

Instead of waiting for a full $32$‑step chunk to finish, the system launches a new inference after only $5$ steps, using the unfinished tail of the previous chunk as a guidance signal.

After $5$ steps, the system starts a second inference; the first $5$ actions $a_1\!-\!a_5$ are already executed.

The remaining tail $a_6\!-\!a_{32}$ receives guidance weight $w_i$ that linearly drops from $1$ at $i=6$ to $0$ at $i=32$.

The second chunk predicts $b_1,\dots,b_{32}$; its denoising loss includes a term $β \cdot w_i \cdot \|b_i - a_i\|^2$ for each guided step.

When the second chunk reaches step $5$, its first $5$ actions $b_1\!-\!b_5$ replace $a_6\!-\!a_{10}$ in the robot’s command stream.

This process repeats, ensuring that at any moment the robot follows a smoothly blended trajectory.

The overlapping schedule lets the policy react to new visual input within a few hundred milliseconds while keeping motion continuous.

Why not simply predict a new full chunk every $32$ steps?

Waiting that long would introduce a noticeable latency between observation and action; overlapping inference reduces the reaction time to the interval of $5$ steps.

All three robot embodiments are fine‑tuned on the same teleoperation dataset using identical optimization settings, isolating the effect of the base vision‑language model.

What would happen if the learning rate were increased dramatically?

The diffusion‑based policy could become unstable, overshooting the denoising target and degrading the smoothness of the generated action chunks.

**Figure 3.** RynnBrain-VLA architecture and unified action space. (Left) RynnBrain-VLA adopts a flow-matching framework in which the RynnBrain base model serves as a single-stream Diffusion Transformer (DiT). The model takes a packed sequence of language instruction, multi-camera observations, robot state, and noisy action chunks as input, and iteratively denoises to predict an action chunk. The instruction token prefix is cached via KV cache to reduce inference latency. (Right) Actions from heterogeneous robot embodiments are represented within a shared 81-dimensional action space partitioned into semantically aligned body-part groups: Arm-Joint (14D, up to 7D per arm), Arm-EEF (18D, up to 9D per arm), Gripper (2D), Hand (40D), Torso (4D), and Head (3D). Each embodiment activates only its physically available dimensions via an embodiment-specific mask, enabling joint training across robots with incompatible low-level action spaces. In our evaluated embodiments: Unitree G1 activates Hand (14D, 7D per hand) within the unified space alongside a separately predicted 64D SONIC [42] latent token; Tianji-Wuji (Tianji Arm + Wuji Hand) activates Arm-Joint (14D) and Hand (40D); Astribot S1 activates Arm-Joint (14D), Gripper (2D), Head (3D) and Torso (4D). The robots shown in the figure are illustrative examples of the masking mechanism.

Pretraining Data Tables

Key pretraining statistics are listed as raw numeric entries.

The table of pretraining data reports the following values (in the order they appear in the paper): 33.8, 59.9, 49.2, 5B, 74.5, 43.8, 46.8, 32.0, 73.4, 59.0*, 57.0, 78.8, 17.1, 19.7, 52.5, 4.2, 30.6, 81.7, 59.3, 7B, 7B, 8B, 8B, 52.8, 26.2*, 39.8*, –, –, 48.5, 30.2*, 46.8, –, –, 57.5, 33.7*, 61.8, 43.1*, 76.6, 30.8, 20.5, 22.3, 3.5, 46.5, 81.4, 59.2, 76.2, 39.0, 28.3, 48.0, 49.8, 49.4, 84.4, 61.3, 53.7*, 31.3*, 46.0*, 24.6, 76.3, 59.0*, 43.8*, 77.0*, 37.2, 31.4, 33.1*, 60.0, 37.6, 83.9, 64.0, 67.3*, 38.3*, 40.0*, 29.3*, 61.4*, 40.1*, 92.1, 80.3, 16.8, 19.9, 8.6*, 44.7, 11.3, 64.9, 53.9, 9B, 61.0, 13.4, 41.5*, 48.7, 57.0, 54.1*, 33.1, 83.0, 45.6, 28.4, 37.3, 33.0, 32.9, 60.5, 45.0.

Embodied Cognition Evaluation

RynnBrain 1.1 leads 2B‑scale models on embodied cognition benchmarks.

RynnBrain 1.1 leads all 2B‑scale models on embodied cognition benchmarks.

Table 3 shows it attains the highest score on 12 of 15 benchmarks.

**Table 1.** Performance comparison of RynnBrain 1.1 and other models at the 2B scale across various benchmarks.

Embodied Localization Evaluation

RynnBrain 1.1‑122B‑A10B tops localization benchmarks, scaling sharply with size.

RynnBrain 1.1‑122B‑A10B reaches a RefSpatial‑Bench score of 79.1, a 20.6‑point gain over the 2B model.

RefSpatial‑Bench improves from 58.5 at 2B to 79.1 at 122B‑A10B.

**Table 4.** Comparison with 4B–9B scale models on embodied cognition and localization benchmarks. * denotes results from our reproduction.

3D Grounding Results

Our 3D grounding approach reaches a peak 86.0% score with an average of 62.8% across the benchmark.

Our method reaches a peak performance of 86.0% on the most challenging test case.

Measured across the full 3D grounding benchmark suite.

The remaining results span the full range from 3.3% to 83.6%, with notable intermediate values such as 70.5%, 66.7%, 58.4%, 55.5%, 45.8%, 31.1%, and 14.6% (the latter two marked as statistically significant).

Contact Prediction and Real-Robot Evaluation

RynnBrain 1.1‑9B sets new 3D‑grounding and robot‑manipulation benchmarks across tasks.

RynnBrain 1.1‑9B markedly improves native 3D grounding over its 2B version.

On SUN RGB‑D it reaches 41.12 AP@15 and on WildDet3D‑Bench 23.44 AP3D, surpassing prior specialized detectors.

The 41.12 score narrows the gap to the closed‑source Gemini Robotics‑ER (48.3) and exceeds the specialized WildDet3D detector (22.6 AP3D) on its own benchmark.

**Figure 5.** Performance Comparison on SUN RGB-D and WildDet3D-Bench with RynnBrain 1.1-2B and 9B.

Qualitative examples (Figure 6) show the model localizing objects and estimating their 3D extents and orientations from a single image, even for large furniture under diverse viewpoints.

**Figure 6** Qualitative examples of language-conditioned 3D grounding by RynnBrain 1.1. Given an image and a target object description, the model predicts an oriented 3D bounding box in the camera coordinate system. The examples cover objects with different scales, shapes, viewpoints, and levels of occlusion in indoor environments.

Contact‑point prediction (Figure 7) demonstrates task‑relevant point selection: the model grabs a mug’s handle, adapts to an overturned mug, and finds appropriate contacts on thin, irregular, or cluttered objects.

**Figure 7** Qualitative results of contact point prediction by RynnBrain 1.1. Given an image and a natural-language instruction, the model predicts an action-relevant contact point together with its in-plane grasp orientation. The examples cover diverse object geometries, object states, target parts, and cluttered scenes, demonstrating instruction-conditioned contact grounding beyond object-level localization. Coordinates are normalized to [0, 1000].

RynnBrain‑VLA delivers stable manipulation success across three heterogeneous robot embodiments.

Success rates: Pull Chair 90 %, Push Chair 100 %, Deliver Menu 100 %, Grab Spatulas 95 %, Flip Steak 60 %.

**Figure 8.** Real-robot evaluation across all tasks considered in our experiments. The tasks are performed in diverse real-life environments and cover whole-body manipulation, bimanual coordination, dexterous-hand operation, object placement, furniture interaction, and object delivery. Representative execution snapshots and success rates over 20 trials are shown for each task.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers