Embodied-Navigator: Point, Think, Memorize, and Align for Efficient Navigation

Hongyan Feng, Sunlai Chen, Xuanyu Liu, Miao Pan, Yangfan Xie, Yuxiang Cui, Zhongxiang Zhou, Rong Xiong, Wenqi Zhang, Jianwei Yin, Yueting Zhuang, Xuhong Zhang

TAMP-Nav decouples high-level 2D visual prompting from low-level 3D execution to improve navigation efficiency and reasoning.

How can we enable VLMs to perform efficient embodied navigation by aligning their 2D reasoning capabilities with 3D spatial action spaces?

Large Vision-Language Models (VLMs) struggle with embodied navigation because they are forced to perform complex 3D geometric reasoning that contradicts their 2D pre-training, while rigid reasoning schedules and inefficient memory management lead to high latency and context overflow. TAMP-Nav addresses this by treating the VLM as a 2D "pointer" that selects pixels for a local SLAM controller, while dynamically triggering Chain-of-Thought reasoning only at critical topological nodes and compressing path intervals into lightweight Space-Time Indicators. This approach achieves state-of-the-art performance on R2R-CE (66.2% SR) while significantly reducing inference time and training data requirements compared to dense-reasoning baselines.

Paper Primer

The framework hinges on three core moves: Pixel-to-3D Action Formulation, Anchor-Trajectory Memory, and Two-Level Group Relative Policy Optimization (GRPO). The Pixel-to-3D move is like a laser pointer: the VLM identifies a target in a 2D image, and the system projects that coordinate into 3D space for the robot to follow, bypassing the need for the VLM to learn 3D geometry.

TAMP-Nav achieves state-of-the-art navigation success rates with significantly higher inference efficiency.

On the R2R-CE validation unseen split, the model reaches 66.2% Success Rate (SR) while averaging 16.58s per task, compared to 37.47s for the StreamVLN baseline. The model requires only 90k training trajectories, a fraction of the millions of interactions used by previous state-of-the-art methods.

Selective reasoning (Reasoning-on-Demand) maintains performance while drastically reducing computational overhead.

By triggering Chain-of-Thought only at critical nodes, the model achieves 66.2% SR with a 26.3% reasoning ratio, nearly matching the 66.8% SR of a "Dense CoT" (100% ratio) baseline. The proportion of reasoning steps in straight corridors drops from 38% to 11% after RL alignment.

Why does the model use a "Pixel-to-3D" approach instead of just regressing 3D coordinates directly?

Foundational VLMs are pre-trained on 2D image-text pairs; forcing them to learn 3D geometric transformations leads to spatial hallucinations and poor sample efficiency. By acting as a 2D pointer, the model leverages its native 2D visual priors while offloading 3D execution to a deterministic SLAM controller.

How does the Two-Level GRPO framework improve navigation over standard reinforcement learning?

Standard RL often suffers from sparse feedback in long-horizon tasks. The Two-Level GRPO superimposes global trajectory rewards (task success) with local step rewards (collision avoidance, reasoning utility), providing dense supervision that stabilizes intermediate state optimization.

For researchers in embodied AI, this paper demonstrates that decoupling high-level semantic planning from low-level spatial execution—and aligning that planning via multi-level RL—is more effective than forcing a single model to handle both geometry and reasoning.

Introduction and Motivation

We expose the core misalignment that hampers VLM‑based embodied navigation.

Large Vision‑Language Models excel at 2D visual‑text reasoning, yet existing embodied navigation methods force them into 3D action spaces that clash with their 2D pre‑training, creating three critical bottlenecks.

VLN‑CE tasks an embodied agent with following natural‑language instructions while moving continuously through a 3D scene, requiring tight integration of visual perception, language understanding, and low‑level motion control.

**Figure 9. Visualization of a simulated navigation trajectory.** The agent successfully executes the complex instruction: “Go up the stairs and turn left. Wait at the doorway to the bedroom straight ahead.” by triggering selective reasoning at key decision points and outputting point-and-click actions.

The primary obstacle is the misalignment between 2D VLM pre‑training and the 3D action space required for embodied navigation.

The TAMP-Nav Framework

We introduce TAMP-Nav, a framework that unifies pixel-to-3D actions, selective reasoning, and two-level GRPO.

Existing VLM‑based navigators force models into unnatural 3D action spaces that clash with their 2D pre‑training, leading to inefficient reasoning and poor sample efficiency.

TAMP‑Nav orchestrates three tightly coupled modules—Pixel‑to‑3D actions, selective reasoning with an Anchor‑Trajectory Memory, and a two‑level GRPO optimizer—to turn vision‑language inputs into efficient embodied navigation.

How does TAMP‑Nav differ from a conventional step‑by‑step chain‑of‑thought pipeline?

Instead of reasoning at every timestep, TAMP‑Nav only invokes deep reasoning at dynamically selected anchor nodes, dramatically cutting computation while still preserving the most informative reasoning moments.

The “Point” module treats navigation as a 2D pointing problem: the model outputs a pixel on the current view, which is then lifted into a 3D coordinate for execution.

Normalize pixel coordinates: x = (u − cx)/f ≈ −0.381, y = (v − cy)/f ≈ −0.072.

Back‑project to 3D: X = d·x ≈ −0.762 m, Y = d·y ≈ −0.144 m, Z = d = 2 m.

Pass (X, Y, Z) to the local SLAM controller, which generates a short motion plan to reach the target while avoiding obstacles.

The pixel‑to‑3D conversion isolates the stochastic visual decision from the deterministic motion planner, enabling the policy to focus on high‑level perception.

Why not predict the 3D target directly instead of using a pixel?

Direct 3D prediction forces the model to learn a high‑dimensional distribution from limited 2D pre‑training data, which is unstable and data‑hungry. Predicting a pixel leverages the strong 2D vision‑language priors and leaves the geometry to a deterministic back‑projection step.

**Figure 1** The architecture of TAMP-Nav. Given visual-textual inputs, the agent compresses long-horizon history into an Anchor-Trajectory Memory, autonomously triggering reasoning at critical nodes. As a visual pointer, it selects the optimal multi-camera view and predicts a 2D pixel, which is projected into 3D for SLAM execution. Finally, the policy is optimized via Two-Level GRPO.

Multi‑branch rollout for the two‑level GRPO optimizer.

The “Point” mechanism bridges 2D vision and 3D navigation by turning a simple pixel regression into a reliable 3D target, letting the policy exploit powerful 2D VLM priors while delegating precise motion control to a deterministic SLAM module.

Pixel‑to‑3D pointing decouples high‑level visual intent from low‑level motion, delivering a compact, sample‑efficient action space for embodied navigation.

Anchor-Trajectory Memory

Anchor‑Trajectory Memory gives the agent long‑horizon spatial awareness while keeping visual processing cheap.

TAMP‑Nav runs a continuous loop: at each navigation step the agent merges its current visual observation with an Anchor‑Trajectory Memory, optionally triggers Selective Reasoning, predicts a pixel waypoint via the Pixel‑to‑3D formulation, and hands the resulting 3D point to a low‑level planner. The whole process is trained with Two‑Level GRPO to maximize success while keeping computation low.

It stores precise spatiotemporal anchors together with a lightweight stream of space‑time tokens, giving the model long‑horizon awareness without overwhelming visual input.

t=0: ESTI(0,0,0,0°) = MLP[RoPE₂D(0,0); RoPE₁D(0); RoPE₂D(sin0, cos0)] → token e₀.

t=1: Anchor created. ESTI(1,1,0,90°) → token e₁ stored as M₁^{STI}; visual feature v₁ captured; CoT state s₁ saved. Form triplet A₁ = ⟨e₁, v₁, s₁⟩.

t=2: No anchor. ESTI(2,1,1,180°) → token e₂ added to redundant interval T₁ = [e₂].

Dynamic context C₂ = concatenate(Working Memory {e₁, e₂}, Long‑term Memory {A₁, T₁}).

Only the critical anchor stores expensive visual data; the intermediate step keeps a cheap STI token, preserving trajectory fidelity while dramatically reducing memory load.

How does Anchor‑Trajectory Memory differ from a conventional replay buffer used in RL?

A replay buffer stores raw observations and actions indiscriminately, leading to large, redundant datasets. Anchor‑Trajectory Memory stores compact spatiotemporal tokens for every step but keeps full visual features only at explicitly selected anchors, thus preserving essential geometry while discarding unnecessary visual redundancy.

Supervised Fine-Tuning and Reward Design

We combine fine‑grained step rewards with a holistic trajectory reward to steer both local actions and long‑range planning.

Standard RL reward shaping often treats each timestep in isolation, causing agents to waste reasoning on trivial moves and miss the bigger navigation goal.

Think of the reward as a budget split between a daily‑expense ledger (local step rewards) and a yearly‑summary statement (global trajectory rewards); the agent must allocate effort to both to stay efficient and finish the mission.

Compute $R_{\text{local}}(t) = 0.6 + 0.9 + 0.8 + 0.4 + 1.0 = 3.7$.

Aggregate the global reward: $R_{\text{global}} = 1 + 0.7 + 0.2 = 1.9$.

The combined reward signal fed to GRPO is the pair $(R_{\text{local}}(t), R_{\text{global}})$, encouraging the agent to keep local actions strong while also improving the overall trajectory.

Equal weighting shows how each component contributes additively; in practice the $\lambda_i$ and $\omega_i$ are tuned so that safety (collision) dominates early steps while reasoning density is penalized later.

How does this Dual Reward System differ from the usual single‑scalar reward used in standard PPO?

Standard PPO treats the whole episode as one scalar advantage, which forces the agent to trade off all objectives implicitly. The Dual Reward System explicitly separates per‑step safety/logic (local) from episode‑level success/efficiency (global) and lets GRPO adjust their relative importance via $\lambda_i$ and $\omega_i$, preventing any single objective from silently drowning out the others.

Two-Level Policy Optimization

We introduce Two-Level GRPO, a hierarchical rollout scheme that balances global and local advantages.

Step‑by‑step expert trajectories constrain exploration, forcing the policy to follow narrow demonstrations. Our method replaces that bottleneck with hierarchical rollouts that let the agent discover diverse paths.

The policy first samples a set of complete trajectories (global level) and, within each step, samples a small pool of candidate actions (local level), then combines their standardized advantages into a single signal.

How does Two-Level GRPO differ from standard PPO that also uses advantage estimation?

Standard PPO computes a single advantage from the whole episode and applies a clipped surrogate loss. Two-Level GRPO splits the advantage into a global trajectory return and a locally normalized reward for the actually executed action, then adds them. This separation lets the policy learn from both long‑range success and immediate visual cues, which PPO’s single‑advantage signal cannot capture.

Step 1: Sample actions $a_1$ (reward 2) and $a_2$ (reward 1). With $β_1=0.5$, compute selection probabilities $P(a_1)=\frac{e^{1}}{e^{1}+e^{0.5}}\approx0.62$, $P(a_2)\approx0.38$.

Step 2: Execute $a_1$ (chosen) and record its local reward 2. Z‑score this reward against the two candidates (mean 1.5, std 0.5) → $A_{\text{local}}=(2-1.5)/0.5=1.0$.

Step 3: After completing the trajectory, compute total return (e.g., 5) and Z‑score across the two trajectories (mean 4.5, std 0.5) → $A_{\text{global}}=1.0$.

Step 4: Combine: $S = 1.0 + 1.0 = 2.0$, which is fed to the GRPO loss.

The annealed $β_k$ gradually shifts the selection from reward‑biased (step 1) to near‑uniform (later steps), preserving exploration while still exploiting high‑reward actions early on.

**Figure 2.** The Two-Level GRPO paradigm. TAMP-Nav superimposes trajectory-level rollouts (competing for global navigation success) with step-level candidate rollouts (exploring diverse 2D visual-spatial actions via VLM temperature sampling).

Two-Level GRPO rollout generation and advantage computation.

Experimental Setup

Key performance of TAMP‑Nav on the VLN‑CE benchmark.

TAMP‑Nav achieves the highest Success weighted by Path Length (SPL) of 66.2 on the R2R‑CE validation‑unseen split, surpassing all prior methods.

Table 1 shows SPL = 66.2 for TAMP‑Nav, compared to 65.0 for ETPNav and lower values for other baselines.

The result reflects a consistent advantage across all metrics: TAMP‑Nav also improves Success Rate and Oracle Success Rate while reducing Navigation Error relative to baselines.

Main Results

TAMP-Nav learns to trigger reasoning only at critical waypoints, cutting unnecessary computation.

The central premise of TAMP‑Nav is to align the action space with the visual pre‑training and to invoke chain‑of‑thought reasoning only when needed.

TAMP‑Nav concentrates reasoning at critical topological nodes, reducing reasoning in straight corridors to 11 % while preserving a 66.2 % success rate.

Table 2 shows the automatic trigger strategy achieves 66.2 % SR with a CoT Ratio of 26.3 %; Figure 3 visualizes the sparse trigger density.

**Figure 3.** Spatial Heatmap of Reasoning (CoT) Triggers. Comparison between the SFT model (left) and the RL-aligned TAMP-Nav (right).

Component Ablation Studies

Component‑wise ablations reveal how each design choice impacts navigation success.

We evaluate each architectural block in isolation on the R2R‑CE and RxR‑CE validation‑unseen splits, keeping all other components, data splits, and the non‑learned SLAM controller fixed.

**Table 3.** Success Rate (SR) under varying multiplicative depth noise levels ($\sigma$).

Replacing the Pixel‑to‑3D action block with a NavFoM‑style metric‑waypoint output reduces SR by 24.8 points on R2R‑CE and 25.2 points on RxR‑CE.

Table 5, Action block comparison

Adding the GRPO block’s local advantage improves SR by 4.1 points on R2R‑CE and 4.2 points on RxR‑CE compared to using only trajectory‑level advantage.

Table 5, GRPO block comparison

Introducing annealed guidance on top of the GRPO local advantage yields an additional SR gain of 2.8 points on R2R‑CE and 3.1 points on RxR‑CE.

Table 5, Two‑Level GRPO with vs. without annealed guidance

Full AT‑Mem boosts SR by 4.3 points on R2R‑CE and 4.6 points on RxR‑CE relative to a full‑history memory baseline.

Table 5, AT‑Mem vs. Full history

Removing the STI component from the memory block decreases SR by 2.6 points on R2R‑CE and 3.3 points on RxR‑CE.

Table 5, Memory block with vs. without STI

Switching from Dense CoT to the Auto variant cuts CoT calls by 73.7 % on R2R‑CE while incurring only a 0.6‑point SR drop; on RxR‑CE the SR loss is 0.4 points.

Table 5, Dense CoT vs. Auto -73.7 % CoT calls (R2R‑CE), -0.6 points SR (R2R‑CE), -0.4 points SR (RxR‑CE)

**Table.** Ablation study of different components in the navigation model. The table evaluates variants of Action, GRPO, Memory, and Reasoning blocks across R2R-CE and RxR-CE benchmarks using NE, OS, SR, and SPL metrics.

CoT Supervision Analysis

We isolate how different chain‑of‑thought (CoT) supervision sources affect navigation performance.

We keep the action space, policy architecture, and training schedule identical across experiments; only the chain‑of‑thought (CoT) annotations differ.

Qwen2.5‑VL‑7B CoT supervision hurts performance relative to the Action‑only baseline.

On R2R‑CE the SR drops from 50.6 % to 49.4 %; on RxR‑CE it drops from 46.1 % to 44.8 %.

GLM‑4.5V‑108B CoT closes the gap to the proprietary Gemini 2.5 Flash teacher.

Its SR is within 1.3 % (R2R‑CE) and 1.4 % (RxR‑CE) of Gemini’s values.

Human quality assessment shows Gemini’s CoT passes 94.0 % of sampled annotations.

Two annotators evaluated 200 random CoTs per teacher; Gemini achieved the highest pass rate.

GLM‑4.5V‑108B CoT attains a 91.5 % human pass rate.

Same assessment protocol as Gemini; GLM’s pass rate is slightly lower.

Beyond CoT supervision, we also evaluate the memory mechanisms that support long‑horizon reasoning.

Removing Space‑Time Indicators (STI) reduces success rate from 49.8 % to 45.6 %.

Experiments with TAMP‑Nav(w/o STI) show a 4.2 % absolute SR drop on the combined benchmark.

**Figure 4.** Performance distribution on long-horizon navigation tasks. The bars show the trajectory count in each length bin, and the curves show the Success Rate (SR) of the evaluated methods as trajectory length increases.

GRPO Dynamics and Real-World Deployment

Ablation analysis of GRPO components and real‑world deployment performance.

We first isolate the contribution of each GRPO component by removing it and measuring the impact on training reward.

Removing local step advantages drops the reward from 0.59 to 0.64, showing that intermediate signals stabilize learning.

Baseline reward 0.59 vs. Two‑Level GRPO without guidance 0.64.

Adding annealed guided sampling further raises the final reward to 0.68 and speeds convergence.

Full Two‑Level GRPO (with guidance) achieves 0.68.

**Figure 5** Learning curves of success rewards under different GRPO configurations. The plot illustrates the training progress over 800 steps for three variants: the full Two-Level GRPO, the version without annealed guided sampling (“without guidance”), and the standard GRPO using only trajectory-level rewards (“only trajectory reward”).

We then evaluate the complete system on a physical robot across diverse indoor and outdoor tasks.

**Figure 6.** Real-world deployment results. (a) Visualization of the execution trajectory. (b) Success rate comparison.

Related Work

Related work on VLM navigation, covering action formulation, reasoning & memory, and reinforcement learning.

Prior VLM‑based navigation approaches can be grouped into three strands: action formulation, reasoning‑memory trade‑offs, and reinforcement‑learning strategies.

Recent VLM‑based navigators either emit predefined textual atomic actions or regress continuous 3D coordinates, which clashes with the 2D‑centric pre‑training of VLMs and leads to geometric hallucinations.

Decoupling high‑level semantic planning from low‑level execution has been proposed, but many designs require costly continuous map construction or restrict exploration to precomputed trajectories.

Existing navigation methods struggle to balance deep reasoning with memory efficiency; dense chain‑of‑thought reasoning at every step inflates inference latency, while aggressive compression of history can discard critical topological information.

Selective reasoning at critical decision nodes and the use of lightweight spatiotemporal tokens for routine path segments have been introduced to bound computational cost while preserving global topological awareness.

GRPO has been applied to embodied navigation, but prior work either relies on extremely sparse trajectory‑level rewards—making credit assignment hard—or forces strict expert action matching, limiting open‑ended exploration.

A two‑level GRPO paradigm that combines global terminal advantages with local step advantages provides dense supervision without the need for step‑by‑step expert forcing.

Key Node Selection Strategy

Describes the algorithm that picks sparse key nodes to trigger reasoning only at critical waypoints.

Reasoning at every timestep is wasteful because most frames carry redundant information. By selecting a handful of “key nodes” we force the model to reason only where the scene changes or the instruction becomes ambiguous. This sparsity preserves trajectory coverage while cutting compute.

We rank every timestep by a combined semantic‑visual importance score, then greedily keep only those that are far enough apart spatially, finally inserting extra nodes when large gaps appear.

Top‑30 % of $5$ frames is $\lceil0.3\cdot5\rceil=2$ frames → indices $\{2,4\}$ (scores $0.8$ and $0.9$).

Initialize $\mathcal{K}=\emptyset$. Add $t=2$ (first element). Distance to $t=4$ is $>D_{\min}$, so add $t=4$ → $\mathcal{K}=\{2,4\}$.

Check the gap between $t=2$ and $t=4$: $\text{dist}_{\text{spatial}}(p_2,p_4)=7>D_{\max}=6$, so insert the highest‑scoring intermediate frame $t=3$ (score $0.5$).

Final key node set $\mathcal{K}=\{2,3,4\}$ preserves coverage while respecting the spatial thresholds.

Even with a tiny trajectory, the algorithm automatically balances importance and spatial dispersion, inserting a middle node only when the gap would otherwise be too large.

Key Node Selection – concise pseudocode.

**Algorithm 1** Key Node Selection Strategy **Input:** Trajectory $\mathcal{T}$ (with spatial poses $p_t$ and visual frames $v_t$), Instruction $Q$, Spatial Thresholds $D_{min}, D_{max}$ **Output:** Key Node Set $\mathcal{K}$ // 1. Importance Scoring (Vectorized & Normalized) Compute $S(t) \leftarrow S_{sem}(t) + S_{vis}(t)$ for all $t \in [1, T]$ // 2. Distance-based Greedy Filtering $\mathcal{I} \leftarrow \text{Argsort}(S)[0 : 0.3T]$ {Top-30% indices descending} $\mathcal{K} \leftarrow \emptyset$ **for** $t \in \mathcal{I}$ **do** **if** $\mathcal{K} = \emptyset \lor \min_{k \in \mathcal{K}} \text{dist}_{spatial}(p_t, p_k) \geq D_{min}$ **then** $\mathcal{K} \leftarrow \mathcal{K} \cup \{t\}$ **end if** **end for** // 3. Temporal In-filling (Padding) Sort $\mathcal{K}$ by time index **while** $\exists$ adjacent pair $(t_{curr}, t_{next})$ in $\mathcal{K}$ s.t. $\text{dist}_{spatial}(p_{t_{curr}}, p_{t_{next}}) > D_{max}$ **do** $\mathcal{K} \leftarrow \mathcal{K} \cup \{\arg \max_{t \in (t_{curr}, t_{next})} S(t)\}$ Sort $\mathcal{K}$ by time index **end while** **return** $\mathcal{K}$

The dataset consists of navigation trajectories annotated with the key nodes selected by the algorithm, providing a chain‑of‑thought supervision signal for embodied agents.

How does this key‑node selection differ from simply sampling frames uniformly?

Uniform sampling ignores both semantic relevance to the instruction and visual scene changes, and it may place nodes arbitrarily close together, causing redundant reasoning. The algorithm instead ranks frames by a joint importance score and enforces a minimum spatial separation, guaranteeing that each selected node contributes new information.

STI Token Analysis

Analyzes how the positional encoder encodes pose into STI tokens and visualizes their fidelity.

We first assess whether the positional encoder preserves the robot’s pose by encoding the initial pose into an STI token and then visualizing the resulting embedding space.

An STI token is a dense vector that encodes a robot’s 2‑D position ($x$, $y$) and orientation ($\text{yaw}$) so that downstream vision‑language models can treat pose as just another feature.

We map $e$ to the latent variable $z$ = ($z_x$, $z_y$, $z_s$, $z_c$) by a learned linear projection, yielding $z$ = [0.2, 0.8, 0.5, 0.1].

Recover the pose: $\hat{x}=z_x=0.2$ m, $\hat{y}=z_y=0.8$ m, $\hat{\text{yaw}}=\operatorname{atan2}(z_s,\,z_c)\approx\pi/2$.

Translate to HSV: hue = $\hat{\text{yaw}}$ (≈ 180°), saturation = $\hat{x}$ (scaled to 0–1), value = $\hat{y}$ (scaled to 0–1).

This toy example shows that the encoder’s output can be inverted to recover pose components, and the HSV mapping makes the geometric relationship visually apparent.

**Figure 8.** Visualization of STI token. We employ a clustering algorithm (McInnes et al. (2018)) to map high-dimensional embeddings into a 3D space.

Beyond the token analysis, we illustrate a full navigation episode where the agent follows a multi‑step instruction, invoking its chain‑of‑thought reasoning at key waypoints such as the staircase and doorway.

The qualitative trajectory demonstrates that the model can ground visual cues, plan over topological structures, and halt precisely at the target, validating the reasoning‑on‑demand paradigm.

Finally, we provide prompt templates that structure the CoT generation, enabling reproducible reasoning steps for future experiments.

Reward Formulations

Defines local and global reward components and the GRPO loss used for training.

This appendix enumerates the scalar reward terms that shape each step and the overall episode, and presents the GRPO objective that ties them together.

Prompts and Training Details

Details the prompt pipeline and training configurations for TAMP‑Nav.

The prompt engineering pipeline for TAMP‑Nav consists of four stages, each defined by a system prompt and a user‑prompt template. Tables 7–11 enumerate the exact prompts used to construct the MultiNav‑CoT dataset. This pipeline guides the model from filtering raw instructions to producing a polished navigation rationale.

Table 7 implements Instruction Quality Filtering: a pure‑action command lacks a target, while any instruction containing a target or landmark is retained. If four or more pure actions appear consecutively, the filter outputs **DROP**, otherwise **KEEP**.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers