GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives
Qize Yu, Lianrui Fan, Boyu Chen, Jiaqi Liang, Xini Ding, Yue Chen, Zetian Song, Yuran Wang, Yi Zou, Kaixuan Wang, Tianxing Chen, Wenxuan Song, Bohan Zhou, Mingleyang Li, Siqiao Huang, Yuqi Ye, Caigao Jiang, Wei Wei, Ruihai Wu, Hang Zhang, Yixiao Ge, Shuchang Zhou, Shilong Liu, Xianming Liu, Ping Luo, Shiyu Huang
GroundingPI is a 4B-parameter foundation model that treats precise spatial perception as a native, language-conditioned capability to improve physical intelligence.
How can a compact grounding foundation model achieve the precision and speed required for closed-loop robotic control?
General-purpose vision-language models often fail at the precise, dense, and tiny-object localization required for physical interaction, forcing downstream action learners to recover basic perception from scarce data. GroundingPI unifies diverse perception tasks—including grounding, OCR, and visual prompting—into a single autoregressive interface that generates quantized coordinates as a shared vocabulary. On 34 grounding benchmarks, it achieves a state-of-the-art 73.68% average, outperforming larger models and demonstrating significant transfer gains in robotic manipulation and autonomous driving.
Paper Primer
The model architecture couples a MoonViT-V2 visual encoder with a Qwen3-4B language decoder, using a learnable projector to map visual features into the language embedding space. It learns through a staged recipe: multimodal pretraining, supervised fine-tuning on coordinate-aligned data, and reinforcement learning using Group Relative Policy Optimization (GRPO) to directly optimize localization and coverage.
GroundingPI establishes a new state of the art in grounding performance.
Across 34 benchmarks spanning 11 perceptual capabilities, it achieves an average score of 73.68%. It outperforms the larger GPT-6 Astra (71.54%) and significantly exceeds similarly sized baselines on dense and tiny-object detection.
Precise grounding improves downstream action-learning efficiency and generalization.
In robotic manipulation (RoboCasa-GR1), GroundingPI trained with 50% of demonstrations outperforms all baselines trained with 75%. It achieves relative improvements of up to 24.8% over the strongest backbone in out-of-distribution settings.
Why prioritize a dedicated grounding model over using a larger, general-purpose vision-language model?
General-purpose models are not optimized for the precise, task-conditioned perception required for physical interaction, creating a bottleneck where scarce action data must simultaneously learn perception and control. GroundingPI provides a "perception-native" foundation that supplies the "where to act" before action learning begins.
What is the role of OCR in a model designed for physical intelligence?
The authors hypothesize that OCR acts as a catalyst for perceptual learning; the requirement to bind text to precise visual regions forces the model to develop sharper local boundaries and stronger fine-grained visual features that benefit dense spatial grounding.
For embodied systems, precise spatial perception should be treated as a foundational, pre-trained capability rather than an auxiliary task; this shift enables more data-efficient and robust action learning.
Introduction: The Grounding Bottleneck
We expose why current vision‑language‑action models lack the precise, fast grounding needed for closed‑loop robotic control.
Precise grounding matters: it must locate the exact object and its position even in clutter or for tiny targets, and it must do so within the latency budget of closed‑loop control. Vision‑language‑action (VLA) models and world‑action models (WAMs) inherit generic vision‑language backbones that are not optimized for this fine‑grained, fast perception, leading to a bottleneck for downstream robotic action.
GroundingPI is a 4 B model that directly predicts points and boxes as quantized tokens, turning visual grounding into a language‑conditioned generation problem.
Visual primitives are the elementary units—points and bounding boxes—encoded as discrete coordinate tokens, allowing diverse grounding tasks to share a common representation.
**Figure 1** Overview of GroundingPI, featuring strong visual perception capabilities. Across 34 perception benchmarks, GroundingPI outperforms 44 baselines and leading VLA/WAM models in driving and robotics, suggesting that stronger perception can be a better foundation for physical intelligence than generality alone.
Current VLA models cannot meet the millimeter‑scale precision and sub‑100 ms latency that closed‑loop robotics demand.
Architecture and Training Pipeline
We replace generic vision‑language features with a precise grounding foundation to enable fast, accurate control.
Generic vision‑language backbones produce coarse tokens that cannot be directly used for precise robot actions; the bottleneck is the lack of a dedicated grounding layer that maps language queries to exact visual primitives.
First we teach the model to emit correctly formatted grounding tokens (SFT), then we fine‑tune the same model with a reward that directly measures spatial accuracy (GRPO).
Rollout 1 predicts
Rollout 2 predicts
Set reward $R_{\text{set}}$ selects the higher IoU among the two rollouts (rollout 2).
Strict reward $R_{\text{strict}}$ penalizes the 4‑pixel $y$ deviation, so rollout 1 receives a lower score.
Weighted reward $A_i = 0.7\,Z(R_{\text{set},i}) + 0.3\,Z(R_{\text{strict},i})$ yields $A_1=0.65$, $A_2=0.88$.
Even with a tiny toy example, the weighted reward pushes the model toward the rollout that best satisfies both coverage and precise localization.
GRPO extends standard policy gradients to a group of $G$ parallel rollouts, rewarding the best‑matching grounding set while penalizing violations such as duplicate boxes.
IoU for Rollout 1 ≈ 0.92, IoU for Rollout 2 ≈ 0.88 → $R_{\text{set},1}=0.92$, $R_{\text{set},2}=0.88$.
Both rollouts have no duplicate or oversized boxes, so $R_{\text{strict},1}=R_{\text{strict},2}=1.0$.
Normalization $Z$ leaves the values unchanged (already in $[0,1]$).
Compute $A_1 = 0.7 \cdot 0.92 + 0.3 \cdot 1.0 = 0.944$, $A_2 = 0.7 \cdot 0.88 + 0.3 \cdot 1.0 = 0.916$.
GRPO’s gradient will favor Rollout 1, nudging the decoder toward predictions that improve IoU.
Even a few‑pixel shift changes the blended advantage, showing how GRPO tightly couples spatial precision to policy updates.
How does GRPO differ from standard PPO, which also uses clipped surrogate objectives?
Standard PPO treats each sampled trajectory independently and clips the ratio of new to old policy probabilities. GRPO first groups $G$ parallel rollouts, computes a set‑wise coverage reward, and then blends it with a strict reward before clipping. The grouping lets the optimizer compare candidates directly, which is essential for grounding where multiple plausible box configurations exist.
Empirical Evaluation
GroundingPI sets new performance records across 34 benchmarks, demonstrating strong perception-to-action transfer.
We evaluate GroundingPI on 34 benchmarks that span perception (boxes, points, OCR, layout) and downstream physical‑intelligence tasks such as autonomous driving and robot manipulation.
GroundingPI outperforms similarly sized models on the overall grounding benchmark.
Achieves a 73.68 % average score, a 2.14 % gain over the next best model.
**Figure 5.** Grounding across perceptual capabilities. GroundingPI and leading baselines across box, point, text, and exemplar-conditioned tasks.
**Figure 6.** Transfer to physical intelligence. Manipulation success rates and reciprocal nuScenes open-loop L2 errors (higher is better in all panels). Manipulation comparisons fix the action expert, action data, and training budget within each benchmark.
**Figure 8** Action-data efficiency on RoboCasa-GR1 ID. Success rate at four demonstration fractions with the remaining action-learning setup fixed.
**Figure 9.** Grounding and action scaling. Grounding Avg and RoboCasa-GR1 ID/OOD success as grounding-training exposure increases under a fixed downstream recipe.
The figure displays a horizontal bar chart comparing the "Grounding Score (34-Bench Avg.)" across five different configurations: "Full (Quantized + MoonViT-V2 (Kimi K3) + Stage III)", "Textual (0.25x speed)", "MoonViT (Kimi-VL)", "Qwen3-ViT (Qwen3-VL-4B)", and "w/o Stage III".
**Figure 12** Generation cost versus predicted object count. GroundingPI generation time and output-token count across box-count ranges.
A foundation model with broad and precise grounding can provide a strong perceptual basis for physical intelligence.
Embodied Foundation Design
GroundingPI maps language queries to precise visual primitives, boosting speed and accuracy of closed‑loop control.
The central premise is that a grounding foundation that directly maps language to visual primitives yields faster, more accurate closed‑loop control.
Physical intelligence is the ability of a system to perceive, reason about, and act upon the physical world in a task‑directed manner.
How does “Physical Intelligence” differ from generic physical reasoning in prior VLA models?
Prior VLA models treat perception as a high‑level language embedding, leaving spatial detail to the downstream policy. Physical intelligence, as defined here, supplies explicit visual primitives that the policy can consume directly, eliminating the need to infer fine‑grained geometry from language alone.
D.1 lists the architectural variants compared: RynnBrain‑2B (Qwen3‑VL), RynnBrain1.1‑2B (Qwen3.5 with linear‑attention and gating), and GroundingPI (MoonViT‑V2 + Qwen3‑4B decoder), all sharing the DeepStack injection mechanism.
D.2 derives sensitivity bounds for gated memory and attention, showing that perturbations to a value vector are attenuated by the product of gate coefficients $\beta$ and query norm Q.
D.3 examines multi‑level visual injection, expressing the action readout sensitivity Bₗ to perturbations of visual features fₗ injected at layer ℓ.
**Figure 7. Perception before action.** A conceptual progression toward a foundation with native grounding capabilities. Segment sizes are illustrative and do not represent measured parameter or data proportions.
Benchmark Robustness and Latency
GroundingPI reaches 62.98 F1 mIoU on COCO, outperforming prior models.
We evaluate GroundingPI across a comprehensive suite of grounding benchmarks, measuring task success, robustness under distribution shift, and observation‑to‑action latency while keeping planner and action modules fixed.
GroundingPI sets a new state‑of‑the‑art F1 mIoU on the COCO box‑grounding benchmark.
Table 19 reports 62.98 F1 mIoU, surpassing the previous best (Astra) by 0.23 points.
**Table 19** COCO. Complete box-grounding metrics. Model-name stars denote external scores from Table 2 of Jiang et al. (2026).
Potential Applications
Potential applications are evaluated on recall, precision, and F1, with feasibility flags.
The authors assess how well GroundingPI can be deployed across a suite of nine candidate tasks, reporting three standard retrieval metrics—Recall (R), Precision (P), and F1—together with a feasibility flag indicating whether the task is currently supported (YES), unsupported (NO), or uncertain (UNK).
Feasibility flags show that six of the nine tasks are currently supported (YES), while three remain uncertain (UNK) or unsupported (NO), guiding future integration efforts.
Qualitative Analysis: Scene Grounding
GroundingPI sets new benchmarks across object pointing, OCR, and token efficiency.
GroundingPI outperforms Astra on HumanRef object pointing, achieving a higher F1@Point.
GroundingPI reaches 88.79 % F1@Point, while Astra’s score is several points lower.
**Table 1.** Performance comparison of various models on robot and spatial pointing tasks and GUI grounding benchmarks.
**Table 5.** Output token efficiency. SEED1.5-VL values are from (Jiang et al., 2026). This cross-model comparison measures serialization rather than matched latency.
**Table 18.**
**Table 20.** LVIS. Complete box-grounding metrics. The starred SEED1.5-VL scores are from Table 3 of Jiang et al. (2026).
**Table 21.** Dense200. Complete box-grounding metrics. Local BAGEL and DeepSeek runs did not reproduce the reference results, so only available external scores are reported for these models. The starred BAGEL scores are taken from Table 1 of Han et al. (2026). No matching external result is available for full DeepSeek-VL2; Small and Tiny checkpoint results are not substituted. The starred DeepSeek-VL2-Small and SEED1.5-VL scores are from Table 4 of Jiang et al. (2026), also reported in Table 2 of Wang et al. (2026a).
**Table 23.** HumanRef. Complete box-grounding metrics. The starred BAGEL scores are taken from Table 1 of Han et al. (2026), because local runs did not reproduce the reported performance. The starred SEED1.5-VL scores are from Table 5 of Jiang et al. (2026).
**Table 24** RefCOCOg validation and test. Complete recorded F1 metrics at IoU 0.50, 0.95, and mIoU. The starred BAGEL scores are taken from Table 1 of Han et al. (2026), because local runs did not reproduce the reported performance. The starred SEED1.5-VL scores are from Table 5 of Jiang et al. (2026).
Qualitative Analysis: Occlusion Handling
GroundingPI’s quantitative edge across multiple grounding benchmarks.
GroundingPI achieves 72.47 F1mIoU on the SROIE benchmark, outperforming LocateAnything Slow NTP (65.49) and Astra (53.57).
Table 25 reports the SROIE F1mIoU values for the three models; GroundingPI’s 72.47 is the highest.
**Table 25.** RefCOCO family. The three datasets are reported separately; their F1mIoU arithmetic mean is RefCOCO avg in the main text.
**Table 26.** Referring object pointing. Starred BAGEL scores are retained despite unreliable support for the unified pointing protocol. Kimi-K3, both MiMo variants, and both DeepSeek variants are N/A under that protocol. These outcomes do not establish intrinsic pointing capability. The starred Molmo and SEED1.5-VL scores are from Table 7 of Jiang et al. (2026); its Molmo checkpoint is Molmo-7B-D.
**Table 27.** Object pointing on COCO and LVIS. Starred BAGEL scores are retained despite unreliable support for the unified pointing protocol. Kimi-K3, both MiMo variants, and both DeepSeek variants are N/A under that protocol. These outcomes do not establish intrinsic pointing capability. The starred Molmo and SEED1.5-VL scores are from Table 7 of Jiang et al. (2026); its Molmo checkpoint is Molmo-7B-D.
Beyond the RefCOCO family, GroundingPI also excels on the ScreenSpot‑V2 benchmark, attaining 96.15 % overall accuracy and near‑ceiling icon scores (96.43 % desktop, 97.54 % web), though Astra still leads on the aggregate metric.
In layout grounding, GroundingPI reaches 85.08 % F1mIoU on DocLayNet, surpassing DocLayout‑YOLO (81.10 %) and Astra (77.54 %). At IoU 0.95 the model records 43.47 % versus Astra’s 34.93 %.
M6Doc results show GroundingPI achieving 74.82 % F1mIoU, beating LocateAnything Hybrid (65.94 %) and Astra (60.59 %). The strict IoU 0.95 F1 score (92.31) also exceeds Astra’s 80.84.
Visual Prompting and OCR
Visual prompting and OCR performance of GroundingPI across benchmarks.
GroundingPI attains 87.04 F1 at IoU 0.50 on VisDrone visual prompting, slightly exceeding Rex‑Omni.
VisDrone results: GroundingPI 87.04 F1 (IoU 0.50), 3.49 F1 (IoU 0.95), 57.99 F1 mIoU; Rex‑Omni 57.15 F1 mIoU.
On the COCO visual prompting benchmark, GroundingPI reaches 82.62 F1 mIoU and 71.53 F1 at IoU 0.95, with balanced recall (84.03) and precision (81.25), outperforming Astra’s 68.43 F1 mIoU and 40.54 F1 at IoU 0.95.
For LVIS visual prompting, GroundingPI records 78.32 F1 mIoU and 65.86 F1 at IoU 0.95, surpassing Astra (64.96 F1 mIoU) and Rex‑Omni (49.36 F1 mIoU) while maintaining balanced recall (77.39) and precision (79.27).
**Table 28** Object pointing on Dense200 and VisDrone. Starred BAGEL scores are retained despite unreliable support for the unified pointing protocol. Kimi-K3, both MiMo variants, and both DeepSeek variants are N/A under that protocol. These outcomes do not establish intrinsic pointing capability. The starred Molmo and SEED1.5-VL scores are from Table 7 of Jiang et al. (2026); its Molmo checkpoint is Molmo-7B-D.
**Table 29.** Robot and spatial pointing. Point-in-mask accuracy is reported for each dataset. The starred RefSpatial baselines follow Table 11 of Jiang et al. (2026); RoboRefer uses the setting without a depth prior. Values retain the precision recorded in our evaluation tables.
**Table 30.** OCR on HierText and ICDAR2015. Each dataset reports four loose-match F1 measures and parse-error rate. GroundingDINO is N/A because OCR is unsupported. Kimi-K3 and both DeepSeek variants are N/A because their outputs do not satisfy the evaluation protocol; this does not establish a lack of OCR capability. SenseNova-Vision uses the HierText and ICDAR2015 F1mIoU scores of 31.20 and 49.50 reported in Table 1 of Han et al. (2026), because its local evaluation prompts could not be aligned. Other metrics for these datasets are unavailable; TotalText and SROIE use local results. The starred PaddleOCRv5 and SEED1.5-VL scores use the BBOX results in Table 10 of Jiang et al. (2026).
Industrial Applications
GroundingPI sets new OCR benchmarks and demonstrates versatile real‑world applications.
GroundingPI achieves the highest SROIE F1@0.50 score among all evaluated Vision‑Language Models.
Table 31 shows GroundingPI at the top of the SROIE column, surpassing the next best model by over 30 points.
Industrial inspection can specify defects via language or exemplar images, while OCR links part markings to their visual regions. Embodied data annotation ties descriptions like “the cup beside the plate” to object boxes in egocentric frames, aiding road‑scene labeling. Aerial and retail imagery benefit from instance localization for counting ships, vehicles, or fruit, with referring expressions disambiguating crowded targets. Document extraction preserves the spatial layout of receipts and labels, enabling field verification. GUI interaction maps commands such as “open settings for this project” to precise click locations in screenshots.
**Table 31.** OCR on TotalText and SROIE. Each dataset reports four loose-match F1 measures and parse-error rate. GroundingDINO is N/A because OCR is unsupported. Kimi-K3 and both DeepSeek variants are N/A because their outputs do not satisfy the evaluation protocol; this does not establish a lack of OCR capability. The starred PaddleOCRv5 and SEED1.5-VL scores use the BBOX results in Table 10 of Jiang et al. (2026).
**Table 32** ScreenSpot-Pro. Action accuracy is broken down by domain and target type, followed by overall action accuracy and parse-error rate. GroundingDINO is N/A because GUI grounding is unsupported. BAGEL is N/A because its outputs do not satisfy the unified protocol. The starred JEDI, UI-R1, and UI-TARS scores are from Table 8 of Jiang et al. (2026); GUI-Owl-32B scores are from Table 3 of Wang et al. (2026a).
Qualitative analysis visualizes grounding outputs on diverse tasks, preserving source images and annotations. Panels labeled “User‑curated candidate” illustrate curated examples, while “GT‑completed display” shows ground‑truth completions for reference.
Qualitative Analysis: Dense Scenes
GroundingPI sets new accuracy records on ScreenSpot‑V2 and OSWorld‑G benchmarks.
GroundingPI achieves the highest exact‑accuracy on the ScreenSpot‑V2 benchmark.
GroundingPI records 97.94% exact accuracy, while no other evaluated model reports a comparable value.
Across all evaluated Vision‑Language models, GroundingPI is the only system reporting exact accuracy, and its action accuracy surpasses the next best baselines that omit several modality columns.
Document Layout Metrics
GPT-6 Astra dominates DocLayNet box grounding, achieving near‑perfect recall.
GPT-6 Astra attains the highest recall (98.97 %) and F1‑score (96.58 %) on DocLayNet, surpassing all other Vision‑Language Models in the >1 T parameter regime.
Table 34 shows GPT-6 Astra reaching 98.97 % recall and 96.58 % F1, outpacing the next best model by more than 1 % absolute.
The table also reveals that SenseNova‑Vision leads the <10 B subgroup, while most specialized detectors lag behind Vision‑Language Models in recall but trade off precision.
**Table 34.** DocLayNet. Complete box-grounding metrics. GroundingDINO lacks a compatible document-region interface; it, Kimi-K3, and both DeepSeek variants are N/A. Starred MiMo and BAGEL scores retain observations under suspected output-protocol incompatibility and are not formal capability measurements. The starred DocLayout-YOLO and SEED1.5-VL scores are from Table 9 of Jiang et al. (2026).
Dense Object Grounding Benchmarks
GroundingPI sets a new high in dense object grounding accuracy.
GroundingPI reaches 95.61% accuracy, surpassing the next‑best model by 4.06 points.
LocateAnything Slow NTP scores 91.55% on the same benchmark; Qwen3.5‑9B scores 5.50%.
Referring Grounding Performance
GroundingPI sets new records on dense object grounding and visual prompting.
GroundingPI achieves the highest mIoU (84.23) on the M6Doc dense object grounding benchmark.
Table 35 reports 84.23 mIoU for GroundingPI, surpassing all other Vision‑Language models in the <10B category.
**Table 35.** M6Doc. Complete box-grounding metrics. GroundingDINO lacks a compatible document-region interface; it, Kimi-K3, and both DeepSeek variants are N/A. Starred MiMo and BAGEL scores retain observations under suspected output-protocol incompatibility and are not formal capability measurements. The starred SEED1.5-VL scores are from Table 9 of Jiang et al. (2026).
**Table 36** FSC147 visual prompting. Complete box-grounding metrics. LocateAnything variants lack a supported visual-prompt interface; both DeepSeek variants have incompatible output protocols. Their entries are N/A. Starred GroundingDINO scores come from an unsupported visual-prompt task interface; starred MiMo and BAGEL scores are retained observations under suspected output-protocol incompatibility.
Qualitative Analysis: Aerial Scenes
Dense point grounding performance highlights GroundingPI’s superiority.
GroundingPI achieves the highest IoU 0.50 on dense point grounding, reaching 69.70 %.
Table 37 shows GroundingPI’s IoU 0.50 of 69.70 % versus the next best 37.91 % (Qwen3.5‑9B).
**Figure 17.** Dense point grounding in an aerial urban scene.
Table 37 reports dense200 visual‑prompting box‑grounding metrics; many competing models are marked N/A because they lack a supported visual‑prompt interface, whereas GroundingPI attains the top scores across IoU 0.50, IoU 0.95, and mIoU.
Qualitative Analysis: Summary
GroundingPI sets the top F1 scores on COCO visual prompting, surpassing all compared models.
GroundingPI achieves the highest F1 scores across all IoU thresholds on the COCO visual‑prompting benchmark.
Table 38 shows GroundingPI F1 22.72 (IoU 0.50), 19.76 (IoU 0.95), and 4.11* (mIoU), exceeding every other listed model.
Other vision‑language models either lack the necessary interface (LocateAnything variants) or fall short on F1: Rex‑Omni reaches a high IoU 0.50 recall (78.69 %) but provides no precision/F1, while Qwen‑family models attain moderate F1 scores (e.g., Qwen3‑VL‑7B ≈ 73 % F1) but still trail GroundingPI’s overall balance.
**Figure 13.** Multi-scale grounding in a crowded classroom scene.
Conclusion
We summarize the model’s impact, ethical considerations, and plans for reproducibility.
GroundingPI is a 4 B‑parameter foundation model that delivers broad and precise visual perception.
Across 34 grounding benchmarks it surpasses same‑size competitors and remains competitive with GPT‑6.
The Astra variant transfers effectively to autonomous driving and robotic manipulation tasks.
Table 5 shows that GroundingPI achieves higher output‑token efficiency than SEED1.5‑VL, a metric that measures serialization rather than raw latency.
Reliable physical interaction demands linking language instructions to exact spatial targets, which generic semantic understanding cannot guarantee.
Dedicated grounding pre‑training directly fills this perceptual gap, improving manipulation generalization and reducing action‑data requirements.
Our analysis identifies basic grounding as a crucial foundation for embodied tasks.
Dense grounding further enhances physical interaction performance.
Optical character recognition (OCR) emerges as a potential catalyst for perceptual learning.
These findings motivate two embodied foundations: broad reasoning for System 2 planning and precise perception for reliable System 1 execution, the direction pursued by GroundingPI.
We used generative AI to assist code development, writing polish, and figure creation, but not to devise research ideas or interpret experiments.
The work involves no human subjects; all datasets are publicly sourced, and evaluations were performed offline on simulation benchmarks.
We will release code, model weights, and evaluation data, along with detailed training and evaluation protocols, to enable systematic reproducibility.
The research was conducted solely for academic purposes during an internship at Xpeng Inc., with no commercial deployment or use of private data.