ABot-N1: Toward a General Visual Language Navigation Foundation Model

Ruiyan Gong, Yingnan Guo, Junjun Hu, Jintao Kong, Xiaoxu Leng, Tianlun Li, Weize Li, Fei Liu, Zhicheng Liu, Jia Lu, Minghua Luo, Chenlin Ming, Yanfen Shen, Jiyue Tao, Zhengbo Wang, Mingyang Yin, Minqi Gu, Zihao Guan, Wei Guo, Guoqing Liu, Huachong Pang, Menglin Yang, Zeqian Ye, Xiaoxiao Geng, Zhining Gu, Honglin Han, Di Jing, Hongyu Pan, Mingchao Sun, Kuan Yang, Jianfang Zhang, Yanghong Chen, Ye He, Wei Mei, Jiahao Shi, Xiangpo Yang, Yanqing Zhu, Yang Cai, Jingjing Ma, Shihui Su, Zixiao Tang, Linbo Zheng, Zedong Chu, Xiaolong Wu, Ziqiao Li, Mu Xu

ABot-N1 decouples high-level reasoning from low-level control using a slow-fast architecture to unify diverse navigation tasks.

How can we decouple high-level deliberative reasoning from low-level reactive control in visual navigation to achieve a generalist foundation model?

Monolithic navigation policies struggle to balance semantic reasoning with precise motor control, often leading to coordinate drift and brittle performance in complex, open-world environments. ABot-N1 factorizes the agent into a slow, deliberative reasoner that generates Chain-of-Thought traces and pixel-space anchors, and a fast, reactive controller that executes continuous waypoints based on those anchors. This dual-system approach achieves state-of-the-art results, including a 35% boost in POI arrival rates and high success rates across diverse indoor and outdoor navigation benchmarks.

Paper Primer

The core mechanism is a hierarchical, asynchronous coupling of two Vision-Language Models (VLMs). The slow system (4B parameters) performs deliberative reasoning to produce a human-readable Chain-of-Thought and a set of "pixel goals"—specific image coordinates that define the next safe, traversable waypoint or final target. The fast system (2B parameters) acts as a low-latency action expert, consuming these pixel goals and linguistic traces to output continuous motion commands at the robot's native control frequency.

ABot-N1 significantly improves navigation success in urban-scale environments compared to monolithic baselines.

Performance on Point-of-Interest (POI) arrival tasks increased by 35.0%.

The architecture maintains high reliability across diverse navigation modalities, including indoor and outdoor scenes.

Achieved 95.4% success rate in complex indoor scenes and 92.9% in outdoor scenes. Near-universal task coverage (Point-goal, Object-goal, POI-goal, Instruction-following, Person-following).

Why does the paper use a "slow-fast" architecture instead of a single, end-to-end model?

Monolithic models suffer from a "spectral mismatch" between slow semantic reasoning and fast motor control, forcing conflicting computational processes into a single cycle. Decoupling them allows the reasoner to perform explicit planning without stalling the high-frequency control loop.

How does the pixel-goal interface improve safety compared to standard coordinate-based navigation?

Standard coordinate targets often drift into physically infeasible regions like vehicle lanes or flower beds. By re-grounding the goal into image-space "affordance pixels" at every step, the agent automatically corrects its trajectory toward the nearest safe, traversable pathway.

Introduction and Motivation

We expose why monolithic navigation policies falter and motivate a slow‑fast factorization.

Monolithic navigation policies collapse high‑level reasoning and low‑frequency control into a single update, which creates a spectral mismatch: slow semantic deliberation competes with fast motor execution. This coupling yields coordinate drift, brittle handling of long‑tail semantics, and opaque decision making. ABot‑N1 addresses these failures by factorizing the policy into an asynchronous slow system that performs Chain‑of‑Thought reasoning and emits pixel‑space goals, and a fast action expert that consumes the linguistic trace and pixel anchors to produce continuous waypoints at the robot’s native frequency.

**Figure 1.** Overview of the dataset and model performance. The left side shows a sunburst chart representing the distribution of 8,423 scenes and 482km of paths across indoor and outdoor environments. The right side shows a radar chart comparing the performance of various models (ABot-N1, Qwen-RobotNav-4B, Previous SOTA, ABot-N0, NavFoM, Uni-NaVid, Qwen-VLA) across different benchmarks (EVTBench, R2R, RxR, Ovon-Short, Object-Goal, Point-Goal, AbotN-PointBench (Indoor/Outdoor), AbotN-PoiBench, POI-Goal).

**Figure 1. Overview of ABot-N1.** The model trained on 30M samples across five tasks adopts a slow–fast control architecture: a slow system performs CoT reasoning and emits pixel goals, while a fast action expert consumes this dual language-and-vision guidance to execute safe waypoints. Closed-loop evaluation is conducted on our newly proposed ABotN-PointBench and ABotN-POIBench, together with three established benchmarks (VLN-CE R2R/RxR, Short-horizon OVON, and EVT-Bench). ABot-N1 achieves leading performance across all 5 benchmarks.

The key insight is that decoupling cognition from control—moving from monolithic to slow‑fast navigation policies—yields safer, more interpretable, and more versatile embodied agents.

Prior Navigation Paradigms

We categorize prior VLN work by model generality, brain–body decoupling, and embodied reasoning.

We organize prior work along three axes relevant to ABot‑N1: (i) general‑purpose navigation foundation models, (ii) dual‑system brain–body decoupling, and (iii) embodied reasoning that combines language and vision.

Early VLN work focused on task‑specific architectures (e.g., R2R, RxR, object‑goal search). Recent efforts consolidate many navigation tasks into a single video‑conditioned VLM‑based policy (NavFoM, ABot‑N0) and explore training‑free routes such as OpenFrontier.

Across robotics, autonomous driving, and humanoid control, a high‑capacity “big brain” (slow, deliberative) is paired with a lightweight “small brain” (fast, reactive). The split mirrors Kahneman’s System 2/System 1 dual‑process theory and appears in autonomous‑driving stacks, hierarchical VLAs, and NVIDIA’s GR00T N1.

Recent work couples explicit linguistic Chain‑of‑Thought (CoT) prompting with visual grounding (pixel‑level goals, visual prompts). This unifies textual reasoning (NavCoT, NavGPT‑2, ECoT) and visual spatial reasoning (VPN, SpatialVLM) into a single deliberative core.

GNM is the shared backbone that encodes visual observations and pixel‑goal targets into a latent representation used by both slow and fast controllers.

ViNT is the fast controller that consumes the pixel‑goal heatmap from GNM and produces high‑frequency motor commands.

Problem Formulation

Define a goal‑conditioned visual policy and split it into a slow VLM reasoner and a fast action module.

Embodied navigation must satisfy heterogeneous user goals—precise metric points, object searches, or free‑form instructions—using only on‑board RGB cameras and outputting low‑level motion commands at the control rate.

It is a deterministic function that consumes the current observation and a goal description and immediately returns the next motion command.

Given a language goal and a short visual history, the policy predicts a short horizon of SE(2) waypoints together with a completion flag.

Separate the heavy visual‑language reasoning (slow) from the lightweight motion generation (fast) so they can run at different frequencies without sacrificing overall responsiveness.

Fast policy $\psi_{\xi}$ receives $s_t$, the two latest $64\times64$ RGB frames, and the instruction embedding; it computes the first waypoint $(x_t,y_t)=(0.3,\,0.4)$ and heading $\theta_t=30^\circ$.

It also predicts the completion flag $c_t=0$ because the target is not yet reached.

Because $H=2$, the policy repeats the same computation for the next step, yielding $(x_{t+1},y_{t+1})=(0.6,\,0.8)$, $\theta_{t+1}=45^\circ$, $c_{t+1}=1$ (goal reached).

The resulting action chunk $a_{t:t+2}$ is sent to the robot controller at the control rate.

The VLM is only invoked every 4 control cycles; in the intervening cycles the fast policy re‑uses the last $s_t$.

The factorization lets the expensive VLM run sparsely while the cheap policy supplies high‑frequency commands, preserving responsiveness without sacrificing the rich language‑vision reasoning.

How does this differ from a monolithic policy that directly maps $(I_{1:t},\ell)$ to actions?

In a monolithic design the heavy visual‑language computation would be performed at every control step, incurring large latency and GPU load. The factorized design runs the VLM only intermittently, caches its output as $s_t$, and lets the lightweight $\psi_{\xi}$ produce actions at the full control frequency, achieving the same functional behavior with far lower per‑step cost.

Because the factorization operates asynchronously, the overall system can satisfy both high‑level reasoning (via the slow VLM) and low‑level reactivity (via the fast policy) without requiring an explicit map.

The Slow-Fast Architecture

Describes the dual slow‑fast architecture, its CoT reasoning, and asynchronous execution.

High‑level reasoning stalls a reactive controller because the planner must wait for a costly deliberation step; the solution is to split planning and actuation into a slow deliberative reasoner and a fast reactive expert.

The navigator is factorized into a low‑frequency “slow” reasoner that writes a textual chain‑of‑thought and pixel anchors, and a high‑frequency “fast” expert that consumes those anchors to produce smooth motion.

How does this differ from a hierarchical reinforcement‑learning controller?

Hierarchical RL typically learns a high‑level policy that selects sub‑goals and a low‑level policy that executes them, both trained jointly. Here the slow system is a frozen large vision‑language model that explicitly writes a textual chain‑of‑thought and concrete pixel anchors; the fast system is a separate lightweight model that never learns the high‑level reasoning but merely follows the anchors in real time.

Step 1: The language model parses the instruction and generates the CoT “The chair is on the left side of the room; first move forward to the doorway, then turn left.”

Step 2: It projects the next safe waypoint onto the current tri‑view, yielding an Affordance Pixel at image coordinates (140, 95).

Step 3: Because the chair becomes visible in the left camera, it also emits a Target Pixel at (30, 200).

Step 4: The fast system receives C₁ and the two pixel goals; it translates the Affordance Pixel into a 3 m forward waypoint and the Target Pixel into a final heading adjustment.

Step 5: The fast controller executes a short sequence of five SE(2) actions that first approach the doorway, then turn left toward the chair, finally stopping when the Target Pixel is reached.

The pixel goals turn abstract language instructions into concrete image‑space anchors that the fast controller can track frame‑by‑frame, eliminating the need for the fast system to understand language.

Asynchronous inference loop for the dual‑system navigator.

Heterogeneous Pretraining

Pretraining the slow and fast systems on a heterogeneous corpus resolves multimodal regression issues.

Smooth‑L1 regression collapses when the target distribution is multimodal—averaging left‑turn and right‑turn trajectories yields an invalid middle path. The paper avoids this by factorizing navigation into a slow system that first commits to a discrete decision (CoT trace + pixel goals) and a fast system that then regresses a single, well‑defined waypoint.

The slow system resolves the high‑level “which way?” choice via a chain‑of‑thought and pixel goals; the fast system only needs to follow the chosen direction, turning a multimodal regression into a stable uni‑modal one.

Sample a radial offset $r$ ∼ 𝒩(0, 10²) m → $r = 4.3 m$.

Sample a tangential offset $t$ ∼ 𝒩(0, 2²) m → $t = -1.1 m$.

Compute the perturbed target position by adding $(r, t)$ to the ground‑truth pixel.

Calculate the angular deviation; here it is 22° < 30°, so the sample is kept.

The model sees the perturbed pixel and learns to fall back on visual cues to locate a reachable goal.

Training with these bounded perturbations forces the slow system to rely on visual affordances rather than trusting a possibly drifted coordinate.

The training protocol also injects noise into the fast system’s inputs, but with a more conservative schedule because the fast controller is highly sensitive to single‑frame pixel errors.

We expose the slow system to aggressive perturbations (its temporal context smooths occasional errors) while the fast system receives milder noise, preserving stability when upstream predictions are stale.

Original affordance pixel = (120, 85).

Sample jitter $\Delta$ ∼ 𝒩(0, 0.5²) → $\Delta$ = (‑0.3, 0.4).

Perturbed pixel = (119.7, 85.4).

Select a reference frame 3 steps earlier; its slow‑system prediction is (118, 84).

Fast system receives (119.7, 85.4) together with the stale prediction (118, 84) as inputs.

Because the jitter is tiny, the fast controller can still rely on the stale prediction for continuity, while the slow system’s larger perturbations are mitigated by its temporal context.

The heterogeneous pretraining corpus supplies millions of samples across five navigation tasks, each with task‑specific input configurations that respect the dual‑system split.

**Figure 3. Data Pipeline and Composition.** The data engine (left) provides diverse indoor and outdoor simulation scenes; trajectory generation (middle) produces expert and Dagger rollouts; the resulting samples (right) span both stages—the five pre-training navigation tasks broken down by slow-system (high-level) and fast-system (low-level) counts, together with the post-training composition stratified into Safe, Critical, Danger, and discarded data.

**Figure 4** Data Construction Pipeline for the Point-Goal Corpus. Left: the data construction pipeline in two parts. The top half is the CoT data construction, which generates affordance pixels from the traversability and road-graph annotations and perturbs the target coordinate; the bottom half is the VLN data construction, comprising sub-optimal trajectory and OOD-correction trajectory synthesis. Right: an example structured sample with tri-view observations and affordance pixel annotation.

**Figure 5. Data Construction Pipeline for the Instruction-Following Corpus.** Left: a three-stage pipeline that decomposes long natural-language instructions into short sub-instructions, aligns each sub-instruction to its corresponding frame range along the milestone path, and generates and verifies affordance and target pixels for CoT and VLN data. Right: an example structured sample showing tri-view observations with the language instruction and pixel-level annotations for affordance and target.

**Figure 6. Data Construction Pipeline for the Object-Goal Corpus.** The left panel comprises two parts: the top half illustrates the iterative data flywheel that constructs the CoT rationales, scaling high-capacity VLM data seeds to 110K high-quality structured samples through A* consistency filtering and self-play harvesting; the bottom half depicts the VLN pipeline that produces the low-level supervision, including pixel annotation and OOD-correction trajectory generation. The right panel demonstrates the resulting structured tuple, featuring tri-view observations, explicit object and affordance pixel grounding, and detailed two-block CoT rationales.

**Figure 7** The Data Construction Pipeline for the POI-Goal Corpus. Left: the three-stage construction flow—generating geometric seed annotations via monocular depth (Stage 1), scaling and filtering 31 M street-view pairs using a distilled VLM (Qwen-3.5-4B) to yield 8 M valid paths (Stage 2), and synthesizing tri-view episodes into positive and negative sample pairs that harden the system’s rejection capability under missing-target conditions (Stage 3). Right: an example structured sample.

Post-Training Alignment

Post‑training aligns the slow system via GRPO, optimizing safety‑aware pixel goals from expert demos.

Supervised pre‑training aligns the slow system to expert trajectories, but those demonstrations omit safety margins and ignore social constraints. Moreover, token‑level supervision does not directly maximize episode‑level success, leaving a performance gap that post‑training must close.

We treat the slow system as a language‑model policy that emits pixel‑goal tokens, then fine‑tune it with reinforcement‑learning rewards that capture safety, format correctness, and task success.

Compute mean $\mu_R = (0.8+0.4+0.2)/3 = 0.467$ and standard deviation $\sigma_R \approx 0.30$.

Normalize each reward: $A_1 = (0.8-0.467)/(0.30+\varepsilon) \approx 1.11$, $A_2 \approx -0.22$, $A_3 \approx -0.89$.

Form importance ratios $\rho_{i,t}$ (here all equal to 1 for illustration) and apply clipping with $\epsilon=0.2$; the clipped term equals the raw term for $A_1$ and $A_2$, while $A_3$ is limited to $-0.2$.

Aggregate the clipped advantages to obtain the policy gradient estimate for this state.

Group‑relative advantage highlights the best candidate without requiring an absolute reward scale, making learning robust to varying episode lengths and reward magnitudes.

How does GRPO differ from standard PPO, which also uses clipped importance ratios?

GRPO normalizes rewards across a *group* of sampled candidates before clipping, so the advantage reflects relative quality within the batch. Standard PPO clips a per‑sample ratio against a fixed baseline and does not perform this intra‑batch normalization, making GRPO better suited for sparse, high‑variance rewards like safety penalties.

The ABotN Benchmark Suite

We evaluate ABotN’s two new benchmarks that test closed‑loop, socially aware navigation.

The central premise—that monolithic policies cannot reconcile high‑level reasoning with low‑frequency control—motivates a factorized slow‑fast architecture. The Benchmark Suite quantifies how well that factorization meets real‑world navigation demands.

ABotN Benchmark Suite comprises 31 high‑fidelity scenes (16 indoor, 15 outdoor) and 163 distinct points of interest.

Scene collection and POI annotation yield 31 reconstructed 3DGS environments and 163 named entrances across 11 commercial regions.

A paired set of closed‑loop tests—PointBench for coordinate goals and POIBench for named‑entrance goals—that evaluate navigation across indoor and outdoor spaces while enforcing social‑rule compliance.

How does ABotN‑PointBench differ from prior point‑goal benchmarks such as CityWalker?

CityWalker evaluates only open‑loop waypoint regression on homogeneous street blocks, ignoring collision handling and social rules. ABotN‑PointBench runs closed‑loop episodes in heterogeneous indoor/outdoor scenes, enforces a walkability map, and measures success with collision‑budget‑aware SR, thus testing both safety and social compliance.

**Figure 9** Overview of the ABotN Benchmark Suites and their Unified Scene Construction Pipeline. Top: Dataset statistics and hierarchical distance splits for ABotN-PointBench (left) and ABotN-POIBench (right). Bottom: The unified three-stage generation pipeline: (1) high-fidelity data collection via LiDAR-inertial SLAM; (2) photorealistic 3DGS scene modeling initialized by aligned dense point clouds; and (3) traversability-aware query sampling and ground-truth reference trajectory generation using A* on MoGE-V2-derived 2D occupancy grids.

Existing benchmarks miss real‑world social navigation because they lack closed‑loop interaction, indoor‑outdoor diversity, and explicit rule‑compliance scoring.

Simulation Evaluation

Simulation results show ABot‑N1 sets new state‑of‑the‑art across five navigation tasks.

ABot‑N1 reduces navigation error to 3.32 m on R2R‑CE Val‑Unseen, the lowest among all reported methods.

Table 1 shows NE = 3.32 m for ABot‑N1, while the next best baseline (HPN+DN*) reports 4.71 m.

**Table 1.** Comparison of navigation methods on the R2R-CE and RxR-CE validation-unseen splits. Methods are categorized by their input modalities: sensor-heavy (top), foundation models/VLMs (middle), and our proposed ABot-N1 (bottom). Best results are bolded.

**Table.** Comparison of single-system VLN policies on long instructions.

The table compares the performance of various navigation methods (GNM, ViNT, NoMaD, CityWalker, SocialNav, ABot-N0, ABot-N1†, and ABot-N1) across two metrics, $SR_{<1\text{col}}$ and $SPL$, evaluated on All, Low (L), and High (H) difficulty tiers.

**Table 4.** Quantitative Point-Goal Results on the Indoor Split of ABotN-PointBench. We report the success rate under a strict zero-collision criterion (SR$_{<1col}$) and SPL, stratified by difficulty tiers: Low (L) and High (H). Our proposed ABot-N1 establishes new SOTA performance by achieving the highest SR (95.4%) and SPL across all difficulty levels, demonstrating its exceptional safety and precision in constrained indoor environments.

**Table.** Comparison of navigation methods on the test set. The table evaluates various methods using $SR_{<2m}$ and $SPL$ metrics.

**Table.** Performance comparison of various navigation methods across Single-Target (STT), Distracted (DT), and Ambiguity (AT) tasks, measured by Success Rate (SR), Trajectory Rate (TR), and Collision Rate (CR).

Real-World Deployment

Real‑world tests show ABot‑N1 runs end‑to‑end on‑robot at 10 Hz across five navigation tasks.

We mounted the full ABot‑N1 stack on the AMap TuTu quadrupedal robot and ran it end‑to‑end in both indoor and outdoor settings. The platform’s 12‑DOF legs, omnidirectional LiDAR, RTK‑GNSS, and 270° tri‑camera suite provide the perception backbone.

ABot‑N1 successfully executes all five real‑world navigation tasks on‑robot.

Runs the complete slow‑fast pipeline on a Jetson AGX Orin 64 GB at a sustained 10 Hz control loop, with no off‑board computation.

Hardware upgrades include integrating the perception stack (BEV mapping, legged odometry, obstacle filtering) into a unified onboard framework, eliminating inter‑process latency. The compute module was upgraded from Jetson Orin NX to Jetson AGX Orin with 64 GB RAM, providing the memory bandwidth needed for concurrent slow‑fast inference.

The edge deployment runs the full pipeline at 10 Hz. The slow system uses Qwen‑3.5‑2B (≈2 B parameters) while the fast system employs a DiT with 306 M parameters and a DINOv2‑Base visual encoder. They operate asynchronously: the slow system updates plans at a lower rate, the fast system continuously consumes cached plans and fresh observations to generate waypoints.

Qualitative results span five core tasks. In each episode the slow system emits Chain‑of‑Thought (CoT) traces and pixel‑level affordance/target predictions, while the fast system converts these pixels into smooth locomotion.

Long‑distance outdoor navigation is broken into four segments: narrow‑road obstacle avoidance, construction‑zone detour, fork selection, and traffic‑light compliance. In each segment the affordance pixel anchors the robot to a safe traversable lane and the target pixel guides it toward the intended waypoint.

Object‑goal episodes demonstrate fine‑grained reasoning: the robot finds an outdoor bench under dappled shade, distinguishes a chair holding a water bottle from other chairs indoors, and locates a partially occluded fire extinguisher while navigating around obstacles.

POI‑goal episodes show commercial‑sign recognition and path feasibility: the robot identifies a Lanzhou noodle restaurant sign at an oblique angle, avoids street‑level obstacles en route to a McDonald’s, and negotiates a slope while bypassing an impassable staircase to reach a Luckin Coffee.

During instruction‑following, four decision moments are highlighted: stair descent, gym entry, gym exit, and final approach to a bar counter. At each moment the slow system produces a CoT sub‑instruction and an affordance pixel; near the goal a target pixel is emitted to terminate the episode.

Person‑following cases include outdoor tracking with pedestrian distractors, outdoor stair‑climbing, and indoor corner rounding with temporary occlusion. The slow system maintains robust identity reasoning, while the fast system provides stable low‑level control across elevation changes and tight turns.

**Figure 10.** Point-Goal Deployment. Four segments of a long-range outdoor episode showcasing obstacle avoidance on narrow roads, construction area detour, correct fork selection, and traffic-light-compliant crosswalk traversal.

**Figure 11.** Object-Goal Deployment. Three cases—outdoor bench under dappled tree shade at long range, indoor chair with a water bottle (spatial reasoning), and partially occluded fire extinguisher—with CoT, affordance, and target pixel overlays.

**Figure 12.** POI-Goal Deployment. Locating a Lanzhou noodle restaurant (large viewing angle), McDonald's (obstacle avoidance en route), and Luckin Coffee (slope navigation and staircase avoidance).

**Figure 13.** Instruction-Following Deployment. Slow-system reasoning at four critical moments—stair descent, gym entry, gym exit, and bar approach—with CoT, affordance pixel, and target pixel visualizations.

**Figure 14. Person-Following Deployment.** Outdoor tracking under pedestrian distraction, stair-climbing following and indoor corner-rounding with temporary occlusion.

Conclusion

Summarizes ABot‑N1’s fast‑slow design, benchmark dominance, real‑world deployment, and team contributions.

ABot‑N1 adopts a fast‑slow architecture that cleanly separates high‑level reasoning (the Slow System) from low‑level execution (the Fast System), enabling each component to specialize without interference.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers