DriveZero: End-to-End Driving Beyond Human Demonstrations
Hao He, Chengcheng Hu, Zirun Su, Heng Zhang, Haisong Liu, Jinke Li, Haochen Tian, Zhenwei Shen, Hongyang Li, Zhichao Li, Yunchen Yang, Bochao Huang, Siyu Zhang, Kuangye Chen, Xiongjie Zhang, Wentao Dai, Hengchen Dai, Siyuan Liu, Zehao Huang, Naiyan Wang
DriveZero decouples perception and action to learn autonomous driving policies from reinforcement learning rather than human logs.
How can we train an autonomous driving policy that performs well without relying on human driving demonstrations?
Autonomous driving systems typically rely on imitating human logs, which limits their performance to the quality of recorded data and prevents them from learning recovery maneuvers or exploring states outside the human distribution. DriveZero decomposes the problem into a perception model (DriveVFM) and an action model (DriveRL), pretraining each in its optimal regime—massive visual data for perception and closed-loop interaction for action—before distilling them into a unified camera-only planner. This approach achieves state-of-the-art performance on NAVSIM and HUGSIM benchmarks, surpassing human driver scores without using any human trajectory supervision.
Paper Primer
The core challenge is that imitation learning is constrained by the "single realized future" in human logs, making it difficult for models to handle safety-critical deviations or recover from errors. DriveZero solves this by using a privileged reinforcement learning teacher that interacts with simulated worlds to generate diverse, goal-conditioned supervision signals that human data cannot provide.
DriveZero is a distillation-based planner: it uses a frozen RL teacher to generate trajectories in closed-loop, then trains a camera-only student to match these trajectories using a winner-takes-all objective. The perception backbone, DriveVFM, is a consolidated representation distilled from multiple frozen foundation models (DINOv3, SigLIP2, SAM, Depth Anything V2) to capture semantics, geometry, and spatial structure without task-specific labels.
DriveZero outperforms human drivers on the NAVSIMv1 navtest benchmark.
The system achieves a Predictive Driver Model Score (PDMS) of 95.3, surpassing the human driver score of 94.8.
DriveRL enables robust closed-loop driving behavior without imitation pretraining.
On the nuPlan benchmark, DriveRL achieves a mean score of 93.57, consistently outperforming the Log-Replay expert and prior RL-based methods like CaRL and GigaFlow.
Why is it necessary to separate the perception and action models?
Perception requires massive, diverse visual data to understand the world, while action requires high-throughput, closed-loop feedback to learn interaction. Separating them allows each to be pretrained in its optimal regime before being unified into a deployable planner.
How does the system handle out-of-domain cases at inference time?
DriveZero uses value-guided test-time action search. It samples multiple candidate actions from the policy, rolls them out for a short horizon using the teacher's critic, and selects the action with the highest estimated return if it exceeds the modal action's score by a set margin.
Introduction and Motivation
We expose the limits of imitation learning and motivate closed‑loop RL with privileged teachers.
End‑to‑end driving systems that learn by imitating human logs inherit the logs’ blind spots: each scene supplies only a single realized future, rare safety‑critical maneuvers are missing, and the policy never sees states it would induce itself.
Relying solely on human‑recorded trajectories caps what a learned driver can do, because the data lack alternative actions, recovery behaviors, and coverage of edge cases.
**Figure 1.** Overview of our end-to-end driving system. (1) Decoupled pretraining. The action model, DriveRL, is a privileged policy trained from scratch with closed-loop RL in mixed-agent interactive worlds built from real nuPlan logs. At inference, a value-guided test-time search rolls out several sampled actions, scores them using short-horizon rewards and the critic, and conditionally replaces the modal action with a higher-value candidate. The perception model, DriveVFM, distills frozen vision foundation models into a single driving backbone from raw images, without task-specific labels. (2) Unification by distillation. DriveZero initializes its encoder from DriveVFM and learns from DriveRL rollouts instead of human trajectories, yielding a camera-only end-to-end planner. (3) Performance. DriveRL exceeds the log-replay expert and prior methods on all six nuPlan closed-loop settings; DriveZero surpasses the human driver on NAVSIMv1 and sets the state of the art on NAVSIMv2 and HUGSIM without any human trajectory supervision.
Shifting from pure imitation to closed‑loop RL with a privileged teacher unlocks driving behaviors that human logs cannot provide.
The Privileged Teacher: DriveRL
DriveRL learns a privileged teacher by training in a massive closed‑loop simulator.
Human driving logs only show what the driver did, offering no way for a model to explore corrective actions. DriveRL addresses this gap by training a privileged teacher inside a simulator that reacts to the ego’s own decisions.
DriveRL learns a privileged teacher policy by letting the ego vehicle act in a massive simulated world that reacts to its own decisions, so the policy discovers safe driving without any imitation data.
Initialize 4 worlds with identical initial states.
Sample actions from the random policy for each world and apply them to the ego vehicle.
Simulate 3 steps of traffic dynamics, compute the reward $r_t$ for each step.
Aggregate rewards across worlds and perform a PPO gradient update.
Repeat the loop, gradually improving the policy.
Parallelism turns what would be weeks of single‑world training into minutes, while preserving the closed‑loop feedback essential for robust behavior.
How does DriveRL differ from standard behavior cloning on logged data?
Behavior cloning merely copies the logged actions, so the model never sees the consequences of its own decisions. DriveRL, by contrast, lets the policy act in a simulator that updates the scene based on those actions, exposing it to novel states and enabling true reinforcement learning.
The privileged teacher receives a full, structured observation that includes future ego states, surrounding agents, and a detailed map, allowing it to compute optimal actions that a camera‑only student could never infer.
Why not train the teacher directly on camera inputs?
Camera inputs lack the future ego pose and detailed map information that the privileged teacher exploits. Without these signals the teacher would be forced to make decisions under uncertainty, reducing its performance and weakening the distillation target for the student.
Closed‑loop RL trains the policy by repeatedly feeding its own actions back into the simulator, so the agent experiences the consequences of its decisions rather than only seeing static logged trajectories.
Why can’t we simply replay logged trajectories for training?
Replay confines the policy to the distribution of logged actions; it never sees the novel states that arise when the policy deviates from the log. Closed‑loop RL forces the policy to handle its own induced distribution, leading to more robust behavior.
The network encodes ego, surrounding agents, and map tokens into a shared representation, then uses cross‑attention to fuse goal and map context before a final MLP predicts the Beta action parameters.
The simulator populates each world with a mix of log‑replayed actors, rule‑based models like IDM, and optionally learned policies, providing realistic traffic dynamics for closed‑loop training.
How does self‑play differ from simply using more log replay?
Log replay follows a fixed trajectory regardless of the ego’s actions, so it cannot generate new interaction patterns. Self‑play allows background vehicles to react to the ego’s decisions, creating richer, more realistic traffic dynamics that improve the policy’s robustness.
At inference, DriveRL samples alternative first actions, rolls them out a few steps using the same dynamics and reward, and picks the one with the highest predicted return—provided it beats the policy mode by a safety margin.
Why not just use the mode action directly without search?
The mode is a single point estimate that may miss higher‑value actions due to stochasticity or local optima. The short‑horizon search explores nearby alternatives, potentially uncovering safer or more efficient maneuvers while still respecting the policy’s learned preferences.
Vision Foundation Models for Driving
DriveVFM distills multiple frozen vision models into a unified backbone for camera‑only driving.
The privileged teacher policy receives rich scene information, but the deployable student must infer everything from raw camera images. DriveVFM solves this gap by replacing hand‑crafted perception heads with a collection of frozen vision foundation models, each providing a different driving‑relevant cue.
DriveVFM unifies heterogeneous frozen vision models into a single backbone by matching their features with lightweight adaptors, letting the student learn all needed visual cues without any task‑specific annotation.
Adaptor for DINOv3 projects the backbone summary to $s_1^{\text{pred}} = [0.4, 0.6]$ and the patch to $p_1^{\text{pred}} = [0.2, 0.8]$.
Adaptor for SigLIP2 projects the backbone summary to $s_2^{\text{pred}} = [0.4, 0.6]$ (no patch projection needed).
Cosine loss for DINOv3 summary: $1 - \frac{[0.4,0.6]\cdot s_1}{\|[0.4,0.6]\|\|s_1\|}$; MSE loss for DINOv3 patch: $\|[0.2,0.8] - p_1\|^2$.
Cosine loss for SigLIP2 summary: $1 - \frac{[0.4,0.6]\cdot s_2}{\|[0.4,0.6]\|\|s_2\|}$.
PHI‑S rescales $p_1^{\text{pred}}$ so its variance matches that of $p_1$, preventing DINOv3 from dominating the gradient.
Even with only two tiny models, the asymmetric supervision forces each adaptor to focus on the features it can actually predict, and PHI‑S guarantees that the patch‑level loss does not drown out the summary‑level loss.
How does DriveVFM differ from conventional multi‑task learning that adds separate perception heads?
Standard multi‑task learning trains each head on hand‑labeled data, requiring large annotation budgets and producing redundant parameters. DriveVFM instead freezes pretrained foundation models and only learns lightweight adaptors that align the backbone’s tokens to the models’ existing features, eliminating the need for task‑specific labels and keeping the backbone compact.
Distilling the Camera-Only Planner
DriveZero distills a closed-loop RL teacher into a camera-only planner using trajectory and score supervision.
Human‑recorded logs pair camera images with structured scene states, but they cover only the trajectories actually driven. This scarcity limits end‑to‑end planners that must rely solely on images. DriveZero sidesteps the gap by learning from a privileged teacher that already knows how to act in the structured state space.
DriveZero is a camera‑only planner that imitates a frozen RL teacher by ingesting multi‑view images, ego kinematics, and a navigation command, then generating multiple candidate trajectories each scored for quality.
Encode each image with DriveVFM → visual tokens of shape $(2,\,64)$.
Compress tokens via learnable registers → $4$ scene tokens.
Form ego token from ego speed $=3\,$m/s and command “turn left”.
Attach ego token to $M=2$ trajectory queries and cross‑attend to scene tokens → two 5‑point trajectories.
Score each trajectory with the scoring decoder → predicted PDM component vectors $(0.8,0.9,0.7,0.6,0.85,0.9)$ and $(0.6,0.5,0.9,0.8,0.7,0.6)$.
Select the higher aggregate score (first proposal) as the student’s output.
Even with only two proposals the student can represent distinct maneuvers (e.g., lane‑keep vs. lane‑change) because each query learns a different trajectory pattern.
How does DriveZero differ from naïve behavior cloning that simply copies the teacher’s actions?
Instead of regressing directly to the teacher’s single action at each timestep, DriveZero generates a set of full‑trajectory proposals and selects the one that best matches the teacher’s rollout (winner‑takes‑all). This lets the student capture multiple plausible maneuvers and learn a scoring function that evaluates overall driving quality.
Distillation transfers the privileged teacher’s closed‑loop behavior into the camera‑only student by jointly optimizing a trajectory‑matching loss and a proposal‑scoring loss.
Why are trajectory loss and scoring loss kept separate instead of merging them into a single loss?
The trajectory loss enforces geometric alignment with the teacher’s path, while the scoring loss evaluates higher‑level driving qualities (collision avoidance, comfort, etc.). Keeping them separate lets the student learn both precise motion replication and holistic safety metrics, which would be conflated if combined into one scalar loss.
**Figure 2. Overview of the end-to-end student policy distillation framework.** The privileged DriveRL teacher rolls out a 20-step trajectory from structured scene observations. The camera-only student DriveZero uses DriveVFM features and a command-conditioned Transformer planner to generate multiple 20-step trajectory proposals. Winner-takes-all (WTA) trajectory supervision transfers the teacher behavior, while the proposal-scoring branch is trained against PDM targets. The teacher is used only during training and is removed at inference time.
Training the Teacher
DriveRL achieves a 93.57 mean CLS score, with test‑time scaling boosting performance by 0.56 points.
We train a privileged teacher using Proximal Policy Optimization (PPO) on a large nuPlan‑derived dataset and evaluate the resulting DriveRL policies.
nuPlan is a massive driving dataset that supplies recorded scenes, each with a short agent history and a long future trajectory, used to initialise training worlds.
**Table 1.** Teacher configuration
DriveRL‑TTS raises the mean closed‑loop score by 0.56 points over the base DriveRL policy.
Mean score increases from 93.01 to 93.57 across the six nuPlan evaluations.
**Table 1.** Performance comparison of DriverRL and DriverRL-TTS with varying N values across Val14, Test14-hard, and Test14-random datasets, evaluated under NR and R conditions.
**Table 3.** Test-time Scaling of a Fixed DriverRL Checkpoint. DriverRL directly executes the Beta mode, whereas DriverRL-TTS evaluates policy-supported action candidates using a five-step rollout and a switching margin. Mean is the unweighted average over the six evaluations, and $\Delta$ is measured relative to DriverRL.
Training leverages 922 k scenes with PPO, scaling to 96 GPUs and 2 048 worlds per rank.
Training the Student
DriveZero’s scaled student achieves top scores on NAVSIM closed‑loop benchmarks.
DriveZero‑Scale attains the highest Extended Predictive Driver Model Score (57.1) on the NAVSIMv2 navhard benchmark.
Table 5 reports a 57.1 EPDMS for DriveZero‑Scale, surpassing all other listed methods.
NAVSIM is a high‑fidelity driving simulator that replays real‑world trajectories and augments them with synthetic scenarios, enabling closed‑loop evaluation of perception‑only policies.
The table presents a comparison of various autonomous driving methods across several performance metrics: NC (No-Collision), DAC (Drivable-Area Compliance), TTC (Time-To-Collision), C (Comfort), EP (Ego-Progress), and PDMS (Predictive Driver Model Score). The methods are categorized into baseline models (PDM-Closed, Human Driver), existing literature methods (TransFuser, DRAMA, VAD-v2, DiffusionDrive, Hydra-MDP++, Centaur, DriveSuprim, UniAD, PARA-Drive, Epona, OneVL, DriveLaW, AutoVLA, DriveVLA-W0, Qwen-Drive-1.0, ReCogDrive, R2SE, iPad, DriveFine, DrivoR-Scale), and the proposed "Our Methods" (DriveRL, DriveZero, DriveZero-Scale).
**Table 5.** Pseudo Closed-Loop Performance on the NAVSIMv2 navhard [3] Benchmark. * marks methods using ground-truth symbolic inputs; all others use camera inputs. “Human” indicates whether human demonstrations are used as training supervision, and “S.” indicates per-stage EPDM score. “-Scale” denotes scaling up the training set with simulation data from SimScale [76]. Best scores are **bolded**; second-best scores are <u>underlined</u>.
Scaling the student’s training set with out‑of‑distribution simulation data is crucial for achieving state‑of‑the‑art closed‑loop driving performance.
Performance Evaluation
DriveZero‑Scale sets new closed‑loop driving benchmarks on HUGSIM and NAVSIM.
End‑to‑end driving is limited by human‑recorded logs; DriveZero trains a privileged teacher via closed‑loop RL and distills it into a camera‑only student. This section reports the resulting performance.
DriveZero‑Scale achieves a state‑of‑the‑art average HD‑Score of 46.6 on HUGSIM, 8.1 points above the previous best.
Table 6 shows DriveZero‑Scale leading all methods across Easy, Medium, and Hard tiers, with an average of 46.6 versus 38.5 for the prior top method (GigaPixel).
Ablation Studies
Ablations quantify how each component affects PDMS driving quality on the NAVSIM benchmark.
We evaluate three controlled ablation studies on the NAVSIMv1 navtest benchmark, reporting PDMS and its five components to trace quality changes to specific model aspects.
**Table 7.** Visual foundation model.
Replacing DriveVFM with a DINOv3 ViT‑S backbone reduces PDMS by 0.53 points.
DriveVFM PDMS = 94.41 versus DINOv3 PDMS = 93.88 on NAVSIMv1 navtest.
Adding SAM and Depth Anything V2 teachers to the DINOv3 + SigLIP2 baseline improves PDMS by a total of 0.72 points.
Baseline 93.69 → + SAM 94.10 → + DA2 94.41.
Goal augmentation on top of DriveRL trajectories raises PDMS by 0.80 points over DriveRL alone.
DriveRL trajectories PDMS = 93.61 versus DriveRL + goal aug. PDMS = 94.41.
DriveRL trajectories alone achieve 0.49 points lower PDMS than human trajectories.
Human trajectories PDMS = 93.92 versus DriveRL trajectories PDMS = 93.61.
Feature Visualization
Visualization shows DriveZero’s planning outperforms baselines and human drivers.
**Figure 3. Feature Representation Visualization.** From left to right: input images, PCA visualizations of DINOv3 and DriveVFM patch features, and their cosine-similarity heatmaps relative to the ground prototype. Blue indicates higher similarity to the ground and red indicates lower similarity. Boxes mark the long-tail obstacles of interest.
DriveZero‑Scale produces smoother, longer trajectories than a human driver who hesitates on clear road, while maintaining a safe distance from surrounding agents. Compared with the previous camera‑only SOTA (DrivoR‑Scale), DriveZero‑Scale avoids collisions and stays within drivable boundaries across multiple navtest scenes.
In two closed‑loop rollouts from HUGSIM, DriveZero‑Scale decelerates in time when the lead vehicle brakes, preserving a safe gap, and makes only a slight lateral adjustment when an oncoming vehicle passes close, staying in its lane. The baseline model either collides or drifts onto the curb under the same conditions.
Related Work
Related work surveys prior approaches to closed‑loop RL, visual foundations, and end‑to‑end driving.
Closed‑Loop Reinforcement Learning for Driving enables policies to learn from the states their own actions generate, rather than from static expert trajectories. Prior works such as Urban Driver, BC‑SAC, CarPlanner, CaRL, and PlannerRFT each advance this idea by scaling model‑free training, improving robustness, or fine‑tuning imitation‑pretrained planners with efficient closed‑loop RL.
Visual Foundation Models for Driving leverage large‑scale pretrained vision backbones and consolidate them into a single backbone via agglomerative distillation. Systems like RADIO (with PHI‑S) and DriveVFM distill features from DINOv3, SigLIP2, SAM, and Depth Anything, removing the need for task‑specific perception labels while combining web‑scale images with driving scenes.
End‑to‑End Autonomous Driving research traditionally relies on behavioral cloning from human logs, but recent approaches transfer interactive simulation behavior to visual policies. Methods such as ROACH, TerraTransfer, GigaPixel, and Pictura demonstrate various ways to distill or align simulated policies with camera‑only students, paving the path for systems like DriveZero that unite pretrained perception and action models without human trajectory supervision.
Contributors and References
Lists the project contributors and provides the full bibliography.
DriveRL was developed by Hao He, Chengcheng Hu, Zirun Su, and Heng Zhang; DriveVFM by Haisong Liu; DriveZero by Haisong Liu∗, Jinke Li∗, Haochen Tian∗, Zhenwei Shen, Hongyang Li, and Heng Zhang, with the asterisk indicating equal contribution.
The real‑world deployment team includes Hao He, Zhichao Li, Yunchen Yang, Bochao Huang, Siyu Zhang, Kuangye Chen, Heng Zhang, Xiongjie Zhang, Wentao Dai, Hengchen Dai, and Siyuan Liu, while project leadership is provided by Hao He, Zhichao Li, Zehao Huang, and Naiyan Wang.
**Table A1.** Training and rollout configuration. Runtime and optimization settings for DriveRL teacher.
The bibliography below cites 105 works spanning datasets, model architectures, reinforcement‑learning techniques, and prior autonomous‑driving systems that underpin the methods presented in this paper.
DriveRL Supplementary Details
Provides detailed training, observation, policy, reward, and analysis specifications for DriveRL.
The simulation stack runs a distributed PPO trainer across 12 nodes (96 GPUs) with a 4‑step rollout horizon and a cosine learning‑rate schedule, completing training in roughly 21 hours.
**Table A2.** Structured Teacher input schema. Listed capacities are maximum counts after policy filtering.
The mixed‑agent simulator connects background actors to three providers: IDM (lane‑center search, headway, integration), a front‑vehicle braking module (probabilistic braking candidates), and log replay.
Observation tensors are capped at 128 actors, filtered to 96 tokens, and include goal anchors, ego state, actor histories, vector‑map segments, and traffic‑light state, each with fixed dimensionalities.
**Table 1.** Comparison of goal sampling between PPO training and formal nuPlan deployment.
Goal conditioning uses two 2‑D anchor points; during training the anchors are sampled from future‑log positions, while deployment derives them from the current ego state and route polyline, preserving permutation‑invariant semantics.
**Table A4. Action and vehicle-dynamics parameters.** Physical command ranges and constants used by the kinematic bicycle model.
The policy head emits beta‑distribution parameters ($\alpha$, $\beta$) for each action dimension, applies a softplus + 1 transform, and maps the normalized outputs through an affine transform to respect vehicle‑dynamics limits.
**Table A6.** Performance metrics for the DriverRL-SelfPlay model across different evaluation splits and modes (NR: Non-Random, R: Random).
The scalar reward combines a hard collision penalty, a goal‑arrival bonus, and a normalized soft term that averages six driving‑quality scores over a 110‑step horizon.
To expose richer supervision, the scalar reward is decomposed into eight channels (hard, goal, and six soft criteria), each receiving its own value‑function and advantage estimate before being summed for PPO updates.
The table compares two training settings: **Single-Ego Teacher** and **Self-Play Teacher** across five parameters: Policy-controlled actors, Traffic composition, Training samples, Worlds per rank, and Global rollout worlds.
Self‑Play extends the teacher policy to up to ten NPCs, replacing half of the IDM‑controlled traffic with policy‑controlled agents and reducing the number of rollout worlds per rank.
**Table A8.** Comparison of policies with and without traffic-light conditioning.
**Table.** Exposing traffic-light state improves scores in all six evaluation settings.
Including traffic‑light state as an input consistently raises scores across all evaluation splits, while adding a dedicated traffic‑light reward reduces violation rates but slightly lowers aggregate metrics.
The traffic-light reward lowers the policy-only violation rate from 3.88% to 2.03%, while all six aggregate scores decrease. This trade-off shows that the aggregate nuPlan score alone does not capture the improvement in red-light compliance, since traffic-light violations are not part of the benchmark score. We therefore report the violation rate alongside the aggregate metrics.
The table compares performance across different splits and modes (Default, Near, and Far) with corresponding anchor configurations. The default and near-only layouts have similar average performance, while duplicating the far anchor lowers all six scores. This indicates that the near anchor provides important local guidance; the far anchor is most useful when paired with, rather than substituted for, the near anchor.
Goal‑anchor ablations reveal that keeping the near anchor (local guidance) is crucial, while duplicating the far anchor degrades performance across all splits.
**Table A12.** Model and Training Configuration for DriveZero-Scale.
Finally, Table A12‑A18 detail the full model and training configuration for the large‑scale DriveZero variant, the impact of traffic‑light conditioning, and the sensitivity of goal‑anchor layouts across evaluation modes.