System One Spatial Intelligence

Luozheng Qin, Yuhan Zhu, Changhai Zhou, Qian Qiao, Jun Gao, Yuhua Zhou, Harry Zhang, Xiangyu Zeng, Yuxuan Liu, Kaijie Chen, Andrew Chen, Pony Ma

MJev-Spatial replaces autoregressive text generation with direct candidate classification for faster, more accurate spatial reasoning.

How can vision-language models perform spatial reasoning tasks (like pointing or measuring) at interaction rates by predicting from a fixed set of candidates rather than generating free-form text?

Vision-language models (VLMs) typically answer spatial questions by generating free-form text, which forces the model to learn formatting and boilerplate while incurring significant decoding and parsing overhead. MJev-Spatial treats spatial tasks as classification over finite candidate sets, using the backbone's pretrained attention projections to score options, numerical bins, or image regions in a single forward pass. This approach achieves the highest overall score across twelve benchmarks while running 5.6× faster than its autoregressive base model.

Paper Primer

The model unifies diverse spatial tasks—such as selecting an option, estimating a distance, or identifying a point—by casting each as a classification problem. It uses a hierarchical coarse-to-fine refinement strategy: for numerical and point tasks, the model first selects a broad interval or region, then re-encodes the selected crop to perform subsequent, more precise classification stages.

The core mechanism is a shared candidate scorer: the model computes scores by taking the inner product of the final attention layer's pretrained query and key projections, bypassing the need for additional prediction heads or new trainable parameters. This is like a mail sorter that reads the address on each envelope and drops it into one bin, so only that bin's carrier ever sees what's inside.

MJev-Spatial outperforms its autoregressive base model (Qwen3.5-4B) on all twelve tested spatial benchmarks.

The model achieves an overall score of 65.64%, ranking first on seven of the twelve benchmarks. It delivers a 5.6× speedup over the base model and a 2.5× speedup over the fastest autoregressive baseline.

Joint supervision and optimized batching significantly improve training throughput.

Supervising all refinement stages in a single forward pass, combined with length-aware global batching and cross-GPU load balancing, increases training throughput. 30.8% improvement over the original batching strategy.

Why is this approach faster than standard autoregressive VLMs?

Standard models must generate answer tokens sequentially and then parse the resulting text. MJev-Spatial evaluates all candidate classes in a single forward pass and generates no answer tokens, eliminating both decoding and parsing latency.

Does this method require adding new parameters to the pretrained backbone?

No. The model reuses the backbone's existing query and key projections from the final attention layer to score candidates, meaning no new prediction heads or trainable parameters are introduced.

For spatial tasks with a bounded output space, direct candidate classification is a more efficient and accurate alternative to autoregressive text generation. Researchers can now prioritize System 1 decision models to achieve real-time spatial reasoning without sacrificing performance.

Spatial Intelligence and Interaction Rates

We expose the latency bottleneck of autoregressive VLMs for spatial tasks and propose a candidate‑selection reformulation.

Spatial intelligence lets an agent perceive geometry and act in the world, requiring rapid judgments of relations, distances, or placement.

Vision‑Language Models (VLMs) answer spatial questions by autoregressive text generation, coupling the answer with token‑by‑token decoding and subsequent parsing.

MJev‑Spatial reformulates spatial reasoning as a candidate‑selection problem, scoring each candidate in a single forward pass and eliminating the latency of token generation.

Each candidate corresponds to a supplied option, a discretized numerical bin, or an image region; coarse‑to‑fine classification refines bins or regions to reach fine‑grained values or points.

Training jointly supervises all stages with teacher forcing in one causal pass, while length‑aware global batching and cross‑GPU load balancing raise training throughput by 30.8 %.

Built on Qwen 3.5‑4B and 12.3 M QA pairs, MJev‑Spatial achieves the highest overall score (65.64 %) among eight models, leads on seven benchmarks, and runs 2.5× faster than the fastest autoregressive baseline and 5.6× faster than its base model.

It is the ability to perceive geometric relationships and act on them quickly, enabling navigation, manipulation, and interaction with the physical world.

**Figure 1.** Overview of the proposed MJev-Spatial. (a) Top scores on 7 of 12 spatial understanding benchmarks. (b) The highest mean score and lowest mean latency among eight evaluated models. (c) Direct option selection and coarse-to-fine numerical and point prediction. MJev-Spatial unifies these three spatial task types within a classification framework, enabling fast-thinking decisions with high accuracy and low inference latency.

The key latency gap in current VLMs for spatial tasks stems from autoregressive decoding; reformulating the problem as candidate selection removes this bottleneck.

Prior Approaches to Spatial Reasoning

We situate MJev‑Spatial among prior decision‑interface and spatial VLM research.

This section surveys prior work on decision‑interface models and spatial reasoning with vision‑language systems, highlighting how MJev‑Spatial builds on and differs from them.

Jev treats a task as a fixed‑size decision problem: the model scores a predefined set of candidate answers instead of generating free‑form text.

KEV extends Jev by learning separate query and key projections, enabling richer interactions between the input and each candidate.

Extends the Jev decision interface to multimodal inputs, allowing images or other modalities to be scored against the same candidate set.

Applies a shared candidate scorer to temporal grounding and video question answering, caching intermediate results for queries about the same video.

Reads candidate‑token probabilities directly from a language‑model head; experiments found no consistent accuracy gain from adding a dedicated typed head.

Early MJev‑Spatial version that used LoRA adapters and an extra decision head on top of a Molmo2‑ER backbone.

Enriches class descriptions and combines multiple augmented views using entropy‑based weighting and optimal transport to improve VLM classification efficiency.

Uses self‑supervised prompt adaptation to handle new classes without per‑image optimization at inference time.

Learns spatial relations and produces numerical answers from spatial supervision within a vision‑language model.

Generates spatially grounded text by training on spatially annotated data, enabling precise coordinate predictions.

Vision‑language model that supports pointing through coordinate text, providing a foundation for spatial interaction.

Enables robotic agents to select visual patches and output precise point coordinates via a language interface.

Provides referential grounding for robotic tasks, mapping language references to spatial locations in the visual field.

Selects visual patches and refines locations through a sequence of specialized grounding tokens, offering a multi‑step spatial decision process.

Formulates monocular depth estimation as ordinal regression over logarithmically spaced depth intervals, a precedent for discretizing continuous predictions.

Iteratively refines image regions to recover details needed for answering visual queries, analogous to MJev‑Spatial’s candidate‑classification region selection.

Acquires evidence iteratively in long videos, interleaving clue seeking with segment inspection, mirroring MJev‑Spatial’s conditional classification path.

The MJev-Spatial Framework

MJev-Spatial reformulates spatial reasoning as a multi-stage candidate-selection process, enabling precise prediction without autoregressive text generation.

MJev-Spatial reformulates spatial reasoning tasks as candidate-selection problems. Instead of generating text tokens, the model predicts by selecting from a set of task-specific candidates in a single forward pass, avoiding the latency of autoregressive generation.

Rather than predicting the next word in a sequence, the model treats every spatial question as a classification task where the "answer" is chosen from a set of valid candidates.

To achieve high precision without an unmanageably large candidate list, the model narrows down the search space in stages: first selecting a coarse region, then a fine-grained cell within that region.

Stage 1: The model scores 30 coarse intervals $[e_j, e_{j+1}]$ and selects the interval containing 12.5 m (e.g., $[10, 15]$).

Stage 2: The model receives the selected interval and scores 8 fine cells within it (e.g., $[10, 10.6], [10.6, 11.2], \dots, [12.4, 13.0]$).

The model selects the cell $[12.4, 13.0]$, and the decoder returns the midpoint: 12.7 m.

By splitting the decision into two stages, the model achieves high precision (sub-metre) while only ever scoring a small, fixed number of candidates at once.

**Figure 2.** Task formulation and coarse-to-fine inference. Every stage uses a shared candidate scorer. Choice tasks select a supplied option. Numerical estimation selects a coarse interval and then a fine interval, returning its midpoint. Point prediction uses three 3 × 3 region selections, re-encoding crops between stages and returning the final cell center.

Model Architecture

Describes how MJev‑Spatial scores candidates using pretrained query/key projections.

The final MLP in the pretrained language layer tends to push representations toward lexical prediction, which weakens the semantic matching needed for spatial candidate selection. To avoid this degradation we compute candidate scores before the MLP, using the layer’s original query and key projections. This preserves the pretrained matching capability while eliminating the extra head.

The model reuses a pretrained Vision‑Language Model (VLM) to embed each candidate description, then matches a single decision query against all candidate keys with a lightweight inner‑product scoring rule.

For head 0, compute $Q_{t,0}^{\top}K_{t,A,0}=4$; scaled value $4/\sqrt{8}\approx1.41$.

For head 1, compute $Q_{t,1}^{\top}K_{t,A,1}=4$; scaled value $1.41$.

For head 2, compute $Q_{t,2}^{\top}K_{t,A,2}=4$; scaled value $1.41$.

For head 3, compute $Q_{t,3}^{\top}K_{t,A,3}=4$; scaled value $1.41$.

Average over the four heads: $z_{t,A}= (1.41+1.41+1.41+1.41)/4 = 1.41$.

Repeating the same steps for Candidate B yields $z_{t,B}=1.35$ because one dot product drops to $3$.

The averaging rule amplifies consistent agreement across heads while penalizing a single mismatched head, giving a clear margin between candidates even with tiny differences.

How does this scoring differ from adding a conventional classification head on top of the VLM?

A standard classification head would introduce a new linear layer that maps the final hidden state to class logits, requiring extra parameters and training from scratch. Here we reuse the pretrained query/key projections, keep the original attention machinery, and compute a similarity score directly—so no new head is learned and the pretrained multimodal alignment is preserved.

Compute candidate scores using pretrained Q/K projections before the final MLP.

**Figure 3. Architecture of MJev-Spatial.** Each candidate is followed by the same End marker. After contextualization by the language backbone, the state at each End position is passed through the same pretrained $W_K$ to form a separate candidate key; the final Decide state supplies the query through $W_Q$. Their similarities determine the selected candidate, with no added prediction head or new parameters. Native normalization, rotary transformations, and head averaging are retained but omitted from the diagram. Probability bars are schematic.

Training Recipe

How we train MJev‑Spatial on 12.3 M QA with a single forward pass.

Standard multi‑stage training would require a separate forward pass for each refinement, inflating latency and GPU idle time. Our recipe folds all stages into one language‑backbone pass, then uses clever batching to keep the compute tight.

We turn every spatial annotation into a question–answer pair, then supervise every refinement stage together, while grouping sequences by length and balancing work across GPUs to avoid padding waste.

Compute the negative log‑likelihood for QA 1: $-\log 0.8 = 0.223$.

Compute the sum for QA 2: $-\log 0.6 -\log 0.7 = 0.511 + 0.357 = 0.868$.

Average over stages for QA 2: $0.868 / 2 = 0.434$.

Average over the two QA: $L = (0.223 + 0.434) / 2 = 0.3285$.

This tiny example shows how the loss normalizes across QA of different stage counts, ensuring that a multi‑stage point does not outweigh a single‑stage choice.

How does this joint stage supervision differ from the usual multi‑pass training where each refinement is trained separately?

In the usual pipeline each stage would require its own forward‑backward pass, incurring extra GPU memory and synchronization overhead. Here all stages share a single forward pass; teacher forcing supplies the true previous targets, so every stage loss is computed in parallel and the total loss is simply the average over stages.

After expanding each QA into its stage sequence, sort all sequences in the optimizer batch by total length (including any image crops).

Form micro‑batches of sequences with similar lengths to minimize padding within each micro‑batch.

Assign the longest‑work‑first micro‑batch to each GPU, using a cost model that adds padded token length and visual‑token work.

Run the language backbone forward pass on each GPU in parallel; gradients are accumulated locally for eight steps before a global synchronization.

Because each QA remains an independent causal row, no additional isolation of the Gated DeltaNet states is needed.

**Table 2.** Performance comparison of MJev-Spatial with seven autoregressive VLMs on 12 spatial understanding benchmarks. Best and second-best scores are bold and underlined, respectively.

Performance and Ablation Studies

Key design choices are dissected to reveal why MJev‑Spatial outperforms alternatives.

MJev‑Spatial treats spatial reasoning as a candidate‑selection problem, letting the model output a precise coordinate in a single forward pass instead of generating a token sequence.

Global‑balanced layout boosts training throughput by roughly 30 % compared with the original ordering.

Table 6 shows QA/s = 129.54 for the global‑balanced layout versus 99.06 for the original order.

Pretrained candidate scoring compares three heads: a native Q/K scorer, a linear classifier, and a KEV pointer. The native Q/K scorer attains the best RefSpatial accuracy (51 %) and the lowest length‑estimation MAE (0.3341 m), while the linear head wins on Where2Place and RoboSpatial pointing.

**Table 3.** Comparison results on pretrained candidate scoring. “Linear” and “KEV” are abbreviation for linear classifier and KEV pointer, respectively.

Numerical discretization experiments test whether finer fine‑stage bins improve prediction. Using 32 fine bins (960 cells) yields a quantization MAE of 0.0391 m and a prediction MAE of 0.3277 m, whereas the compact 4/8‑bin design (216 cells) reduces prediction MAE to 0.3202 m despite higher quantization error.

**Table (a) Fine-stage candidate count** and **(b) Fine-stage decoding**

Visual refinement studies compare four visual‑input strategies. Crop re‑encoding preserves the highest RefSpatial (51 %) and Point‑Bench (56.84 %) scores, while using the original ViT features or replaying K/V drops performance dramatically, underscoring the need for fresh local crops.

Multi‑point prediction evaluates a spatial probability tree versus a simple repeat‑until‑stop scheme. The tree raises Point‑Bench counting success from 19.39 % to 38.78 % and, with root pruning, cuts the mean decision nodes from 91 to 51.10 and inference time from 4.716 s to 2.638 s (1.79× speedup).

The table presents performance metrics for different layout strategies, including "Original order," "Local length," "Global balanced," and "Global + buffer 128." The columns are "Layout," "QA/s," "Padding (%)," "Step CV (%)," and "Tail (ms)."

Training Speed and Scheduling

Ablation study quantifies how scheduling tricks affect training throughput and padding.

We evaluate how different scheduling choices change training efficiency and then describe the sequence layout and scoring mechanism that enable fast candidate selection.

Instead of generating answer tokens one by one, the model treats each spatial question as a selection among a fixed set of candidates, turning a sequential decoding problem into a classification problem.

How does this classification formulation differ from standard autoregressive generation?

Autoregressive generation produces a token sequence of length $L$, requiring $L$ forward passes; the classification formulation predicts the answer in a fixed number of stages (≤ 3), eliminating the dependence on output length.

The question, images, and each candidate are laid out in a single causal sequence with explicit markers, letting the model attend to the current option and the decision point.

Why can the candidate set vary freely across questions?

Because each candidate is encoded as a separate token block; the classifier reads the current set of candidate embeddings rather than a fixed number of slots, so adding or removing candidates only changes the sequence length.

The model reuses the final‑layer query and key projections to compute a score for each candidate, avoiding a dedicated prediction head.

What ensures that these reused projections still produce meaningful candidate scores?

The query and key matrices were trained on massive text data to capture semantic similarity; when applied to the decision and candidate states they produce a similarity score that correlates with the correct answer, and the softmax normalizes over the current candidate set.

Local length grouping raises throughput by 17.9 QA/s and cuts padding to 12.4 %.

Throughput increases from 99.06 QA/s to 116.91 QA/s; padding drops from 35.17 % to 12.36 % (Table 6).

Global grouping with cross‑GPU balancing reaches 129.54 QA/s, a 30.8 % higher throughput than the baseline.

Throughput 129.54 QA/s versus the original 99.06 QA/s; padding reduced to 1.77 % and ready‑time spread to 145.7 ms.

A 128‑QA cross‑update buffer lowers padding to 1.03 % but yields no throughput gain and increases step‑time variability.

Padding 1.03 %; throughput unchanged relative to the original; observed higher variability in step times.

Readout Controls and Optimization

We compare three readout variants to test if a pretrained scorer can replace a classification head.

We evaluate three readout variants to see whether a pretrained candidate scorer can replace an added classification head without sacrificing spatial performance.

Native mean readout outperforms the linear classifier by 1.58 points on CV‑Bench.

Table 3 reports a 1.58‑point gain for native mean versus the linear head.

On RoboSpatial VQA, native mean gains 7.72 points over the linear classifier.

Table 3 shows a 7.72‑point improvement for native mean.

RefSpatial performance rises by 2.00 points with native mean compared to the linear head.

Table 3 lists a +2.00 improvement for RefSpatial.

Mean absolute error (MAE) drops by 8.93 % when using native mean instead of the linear classifier.

Table 3 records an 8.93 % reduction in MAE.

Where2Place accuracy declines by 8.01 points with native mean relative to the linear head.

Table 3 shows an 8.01‑point loss on Where2Place.

RoboSpatial pointing performance falls by 3.28 points when switching to native mean.

Table 3 reports a –3.28 drop for RoboSpatial pointing.

Compared to the KEV pointer, native mean improves RefSpatial and length error but does not uniformly beat KEV on choice‑type tasks.

**Table 12.** Completed Qwen experiments on the fixed evaluation union. All adapted models use the 72K-QA recipe and the historical length codebook; these are not the 12.3M-QA model. Higher is better except VST MAE. Point-Bench is recomputed as the five-category macro average from retained category scores. Original run provenance and differing sequence budgets are retained.

**Table 13.** Category-level main comparison. Percentages use each task's recorded scorer. Historical length codebook and root-pruned Point-Bench. Categories do not share a common difficulty scale.

Depth and Head Analysis

Ablations reveal how readout depth and aggregation affect performance across tasks.

We evaluate three readout strategies—mean, concat, and merge—at eight attention depths (L4 … L32) while keeping the training data, optimizer settings, and evaluation protocol identical.

**Figure 4.** The complete $8 \times 3$ depth/aggregation experiment, shown on three representative metrics. Early visual readouts are inadequate, while later layers exhibit task-dependent rather than monotonic gains. All 24 models’ benchmark values are provided in Section E.3.

Mean readout improves CV‑Bench from 51.41 at L4 to 85.38 at L32, a +33.97 increase.

Table E.3 reports 51.41 CV‑Bench at L4 and 85.38 CV‑Bench at L32 for the mean readout.

Concat readout at L32 yields a lower MAE (0.3297 m) than mean readout (0.3341 m), a -0.0044 m improvement.

Figure 4’s rightmost chart lists MAE 0.3297 m for concat and 0.3341 m for mean at L32.

Merge readout at L28 raises RoboSpatial pointing to 67.21, roughly +2.0 points over the mean readout at comparable depths.

Section E.3 reports 67.21 RoboSpatial pointing for merge at L28, while mean readout at nearby depths lags behind.

**Figure 5.** Centered candidate-logit RMS for each head, averaged per QA along the all-head trajectory. Values share a common scale and measure response amplitude. H02 is strongest and H11 weakest in all three task groups.

Extended Readout Results

Extended readout variants show mixed gains, with notable accuracy boost on Where2Place.

Adding the $L28$ readout layer to the $L32$ baseline raises Where2Place accuracy by +6.0 points.

Where2Place improves from 59.98% ($L32$) to 65.98% ($L32+L28$) in Table 14.

**Table 13.** Category-level main comparison (continued).

The table presents performance metrics across various benchmarks (SAT real, CV-Bench, RoboSpatial VQA, BLINK val, MMSI-Bench, RefSpatial, Where2Place, RoboSpatial pointing, Point-Bench (macro), and VST-300 MAE) for different model configurations (L32, +L28, +L24, +L28+L24).

Additional Readout Findings

Readout variants and head interventions reveal modest performance gains and limited redundancy.

Removing any single head changes classification benchmark scores by at most 0.020 pp.

Table 17 shows the largest classification $\Delta$ is 0.020 pp (head H03).

Across depths, the merge readout yields higher CV scores than mean or concat, especially at L20 where it reaches 85.48. Mean and concat variants lag behind but show modest improvements as depth increases.

The table presents performance metrics across different readout methods (mean, concat, merge) applied to various layers (L4, L8, L12, L16, L20) evaluated on five benchmarks: SAT, CV, Robo VQA, BLINK, and MMSI.

**Table 15.** Complete depth/readout sweep: classification metrics (continued).

Spatial Refinement Studies

Ablations dissect visual input and decoding choices, revealing their impact on spatial performance.

We evaluate three families of visual‑input ablations—crop removal, alternative visual encodings, and direct visual‑token candidates—alongside decoding‑strategy ablations to quantify how each component contributes to spatial reasoning performance.

Removing pixel crops from the fixed‑weight model drops RefSpatial from 51.00 % to 9.00 %.

Table 18 shows RefSpatial = 9.00 % after crop removal (vs. 51.00 % with crops).

Training with matched original‑image context recovers RefSpatial to 46.50 %.

Table 5 reports RefSpatial = 46.50 % when the model is trained on original‑image inputs without crops.

ROI K/V replay improves Where2Place to 68.95 % but reduces RoboSpatial pointing to 54.92 %.

Table 5 shows Where2Place = 68.95 % and RoboSpatial pointing = 54.92 % for the KV‑replay variant.

Direct visual‑token candidates (KEV) achieve RefSpatial ≈ 38.5 % and Where2Place ≈ 68.0 %.

Table 19 lists RefSpatial = 38.50 % and Where2Place = 67.96 % for the KEV argmax decoder.

Using visual‑token readouts yields a VST‑300 MAE of 0.334 m, comparable to the hierarchical crop baseline.

Table 20 reports VST‑300 MAE = 0.3341 m for the visual‑token configuration.

Probability‑tree decoding attains 38.78 % official success, matching the full‑tree variant.

Table 21 shows official success = 38.78 % for both full and root‑pruned trees.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers