Alaya-Evoke: From Linear-Scaling Supervision to Endless World

Yuanyang Yin, Gongxuan Wang, Yifan Zhan, Chuanhao Li, Kaipeng Zhang, Feng Zhao

Evoke decouples persistent world state from denoiser context using an external geometric bank to enable bounded-cost, long-horizon interactive video generation.

How can we generate indefinitely long, consistent video without the compute cost of the model's memory growing linearly with time?

Interactive world models struggle to maintain persistent memory and responsive control over long sessions because history-tracking methods force the denoiser context to grow, eventually crashing performance or requiring information loss. Evoke solves this by externalizing persistent scene geometry into a camera-indexed world state bank, while training a few-step student using a teacher explicitly designed for long-horizon supervision and per-chunk conditioning. This architecture keeps the denoiser context and computational cost independent of session duration, allowing for open-ended generation that remains stable over hour-long rollouts.

Paper Primer

The core mechanism is a bounded recurrent interface that separates transient denoiser history from persistent spatial state. The student generates video chunks by reading relevant geometry from an external bank based on the current camera pose, then writing new observations back to that bank, ensuring the denoiser only ever sees a fixed-size window of recent frames and retrieved geometry.

The teacher is redesigned to provide supervision across long temporal horizons using chunk-wise sparse attention, which replaces quadratic attention growth with linear scaling. This allows the teacher to expose long-range content drift and dynamic prompt changes, which are then transferred to the student via a 30-second distribution-matching objective: the teacher acts as a long-horizon supervisor, while the student acts as a low-latency, three-step executor.

Evoke achieves bounded-cost, open-ended generation with stable photometric statistics over hour-long sessions.

Quantitative evaluation of eight 65.5-minute rollouts shows that photometric statistics stabilize after an initial transient, with no runaway degradation as the session progresses. Each 1.5-second chunk is generated in 2.11 seconds on a single H200 GPU, with computational cost remaining constant regardless of total session length.

The model achieves state-of-the-art performance on the WBench navigation split among few-step systems.

Evoke leads in Video Quality, Setting, and Physical group averages, while remaining competitive with many-step systems on VBench-2.0 and VBench-Long. Overall VBench-2.0 score of 66.77 and VBench-Long score of 85.11, despite using only three sampling steps and no classifier-free guidance.

Why does the model use an external world state bank instead of just increasing the denoiser's context window?

Increasing the context window causes the cost of each denoising step to grow with session history, eventually hitting a hard limit. Externalizing geometry allows the denoiser to operate with a fixed token budget, making the system's performance independent of how long the session has been running.

What is the specific role of the "long-horizon teacher" in this architecture?

The teacher is designed to expose long-range content drift and mid-session instruction changes that a short-horizon teacher would miss. By distilling these capabilities into the student, the model gains resistance to progressive drift without needing to perform expensive, many-step inference at runtime.

Evoke demonstrates that persistent world state and interactive responsiveness can be decoupled, allowing developers to build long-running world models that do not suffer from the "context-growth" bottleneck.

The Challenge of Persistent World Models

Interactive world models face a clash between persistent memory and compute efficiency.

Interactive world models must juggle three demands—persistent memory, low‑latency interaction, and long‑horizon generation—but the mechanisms that satisfy one tend to break another, creating a fundamental tension.

When a model keeps every past frame in its denoiser context, the amount of data the denoiser must process grows linearly with time, inflating both memory use and compute cost.

**Figure 1.** Two hours of uninterrupted generation. Representative two-hour rollouts under continuous camera control, generated in three steps per chunk without classifier-free guidance. Each row shows six frames sampled uniformly over the full session, illustrating sustained coherence over hour-scale generation.

**Figure 12.** Qualitative rollouts across four domains. Nine sessions generated by Evoke in three steps per chunk without classifier-free guidance, one per row, each shown as five frames evenly spaced over the clip: 5.9 s in (a–c) and 8.9 s in (d), where a timed event fires in the last third. The rings in the lower corners are the engine’s own joystick overlay.

The core conflict is that preserving a growing persistent memory inside the denoiser inflates compute, whereas low‑latency interaction demands a bounded, cheap denoiser.

Externalizing World State

We externalize persistent geometry in a pose‑indexed bank and redesign the teacher for long‑horizon, chunk‑wise supervision.

The denoiser’s context grows with every frame, so compute and memory scale linearly with session length. This makes long‑horizon generation prohibitively expensive.

A camera‑indexed repository that stores lifted scene geometry and can render any previously seen view on demand, keeping the denoiser’s token budget fixed.

Step 1: At pose P₀ the observation of S₁ is lifted and stored in slot 0.

Step 2: At pose P₁ the observation of S₂ is lifted and stored in slot 1.

Step 3: At pose P₂ the bank renders a view using slots 0 and 1, producing pixel conditioning for the generator.

Step 4: A new observation at pose P₃ would overwrite the oldest slot (slot 0) because K = 2.

The bank always supplies exactly two conditioning frames regardless of how many total steps have been taken, keeping the denoiser’s input size constant.

How does the World State Bank differ from a traditional KV cache?

KV caches store raw token embeddings keyed by position and grow with every timestep; the World State Bank stores explicit geometry indexed by camera pose and has a fixed capacity, so recall cost stays constant and the conditioning is view‑aligned rather than position‑aligned.

Having fixed the denoiser’s context, we turn to the teacher side: long‑horizon supervision and dynamic conditioning must be delivered without re‑introducing quadratic attention growth.

Each temporal chunk sees a small local window, a handful of distant frames, and a linear‑attention global state, while an independent text prompt is attached to the chunk; this yields linear‑time attention yet retains long‑range information.

Chunk‑wise processing with sparse local, sampled distant, and linear global attention.

Why isn’t standard sparse attention sufficient for long‑horizon consistency?

Standard sparse patterns (e.g., fixed‑stride or block‑local) still limit the number of tokens that can influence a given position, so information from very distant chunks may never reach the current token. The added linear global state guarantees a cheap, O(N) pathway for any token to affect any other, preserving long‑range consistency.

In summary, (i) we decouple persistent geometry from the denoiser via a bounded, pose‑indexed World State Bank; (ii) we redesign the teacher with chunk‑wise sparse attention and per‑chunk conditioning, achieving linear‑time long‑horizon supervision; and (iii) we transfer these capabilities to a three‑step, CFG‑free student that supports on‑the‑fly prompt changes while remaining computationally constant.

Supervising Long-Horizon Generation

Long-horizon supervision expands the training window to stabilize recurrent generation over extended rollouts.

Even with bounded recurrence, errors accumulate when the student conditions on its own past generations, causing drift and loss of scene consistency. The only way to curb this drift is to let the training objective see enough of the rollout to penalize long‑range mistakes.

Instead of supervising only the next chunk, we ask the model to match a window of $W$ consecutive chunks, forcing it to keep its trajectory coherent over a longer span.

For $k=1$, compute divergence $D\big(q_\theta(1\!:\!2)\,\Vert\,p(1\!:\!2)\big)=0.12$.

For $k=2$, compute divergence $D\big(q_\theta(2\!:\!3)\,\Vert\,p(2\!:\!3)\big)=0.08$.

For $k=3$, compute divergence $D\big(q_\theta(3\!:\!4)\,\Vert\,p(3\!:\!4)\big)=0.15$.

For $k=4$, compute divergence $D\big(q_\theta(4\!:\!5)\,\Vert\,p(4\!:\!5)\big)=0.09$.

Average the four values: $L_2(\theta)=\frac{0.12+0.08+0.15+0.09}{4}=0.11$.

Even with a tiny window, the loss aggregates information from overlapping pairs, exposing how early errors propagate into the next chunk.

How does this differ from standard teacher‑forcing where only the next token is supervised?

Standard teacher‑forcing matches the student’s prediction to the immediate ground‑truth token, which cannot penalize errors that only manifest after several conditioning steps. Long‑horizon supervision instead matches distributions over $W$ consecutive chunks, so the student receives a gradient that reflects the cumulative effect of its own past predictions.

The Evoke Teacher

Evoke introduces a chunked sparse teacher that provides constant‑time long‑horizon supervision.

The student’s long‑horizon stability is limited by the temporal horizon of its teacher. By extending the teacher’s supervision window while keeping per‑step cost constant, Evoke removes this bottleneck and yields markedly more stable rollouts.

Instead of a fixed teacher that only sees a short window, Evoke’s teacher scores the entire rollout in small, overlapping chunks, so each chunk’s computation stays bounded regardless of total sequence length.

How does the Evoke teacher differ from a conventional teacher that simply attends over the whole sequence?

Conventional teachers use full‑attention, whose cost grows quadratically with sequence length, making long‑window scoring prohibitive. Evoke replaces full‑attention with chunk‑wise sparse attention and a linear‑attention global state, so each chunk’s cost stays constant while still receiving information from the entire trajectory.

Start from a ground‑truth prefix chunk $x^{gt}_0$.

Generate a self‑forced rollout of 20 chunks $x_1,\dots,x_{20}$ (189 latent frames total).

Score every latent frame with the shared teacher‑critic backbone, producing $s_{\text{real}}$ and $s_{\text{fake}}$.

Compute the mask $\Omega$ that excludes the prefix and $x_1$, then evaluate $\Delta s$ and $\nu$.

Apply the loss $L_{\text{gen}}$ to the student predictions for chunks $x_2$–$x_{20}$.

Back‑propagate gradients independently for each chunk (history detachment), keeping activation memory bounded.

Global sink (frame 1 of chunk 1), local overlap (frame 3 of chunk 1), compressed nearby frames (average of frames 4–5), one selected distant frame (frame 8), and the linear‑attention global state accumulated from chunk 1.

The source set size is 5, independent of total chunks.

Chunk 2’s attention matrix therefore costs $O(5 \times 3)$ instead of $O(9 \times 3)$.

When the same process repeats for chunk 3, the global state is updated linearly, and the source set remains 5.

Even as the video grows arbitrarily long, each chunk only ever looks at a constant‑size window plus a compact global state, guaranteeing constant per‑step compute.

**Figure 10** The geometric path costs 38% of the denoiser and 93% of a recurrent step that skips it. Per-chunk wall clock on one H200 at 384x640, 3 denoising steps, CFG off; arms differ only by the named flag. (a) Warp rendering and coarsest-stage token admission are separable costs, and stages 1 and 2 are unchanged throughout—tokens, not geometry, carry denoiser cost. (b) Geometry adds 1.84 s per chunk, scaling with warp coverage: unobserved space costs less exactly where memory offers less.

Geometric World State

Geometric World State stores pose‑indexed geometry for constant‑time retrieval and bounded inference.

Memory bloat forces per‑step compute to grow linearly with time, breaking long‑horizon generation. Evoke sidesteps this by moving persistent geometry out of the bounded denoiser context into an external, pose‑indexed store, keeping each recurrent call constant‑time.

The model keeps a lightweight bank of world‑space geometry indexed by camera pose, so when the camera revisits a location the exact observed surface can be fetched instantly.

Step 1: At pose A the model writes $G_A$ (2 frames) into the bank.

Step 2: Pose B arrives; $G_B$ is written, bank now holds A and B.

Step 3: Pose C arrives; $G_C$ is written, bank holds A, B, C.

Step 4: The camera returns to pose A. Retrieval queries $M_k$ with pose A, finds $G_A$, ranks it as fully co‑visible, and renders a warped view.

Step 5: Visibility mask for the warped view yields $v=0.8$ on most pixels, so $\sigma$ is drawn from $[0,0.135]$, allowing the geometry to condition the denoiser.

The bank provides instant, exact geometry for revisited poses without expanding the denoiser context; only the visibility mask decides whether that geometry is trusted.

Inference proceeds in three CFG‑free denoising passes per chunk (12×20 → 24×40 → 48×80). Geometric conditioning is injected only at the coarsest resolution, establishing large‑scale layout before finer stages add detail. Visibility‑based pruning removes unsupported geometry, so the conditioning cost scales with bank coverage, not with total session length.

**Figure 7.** The world state bank puts a place back when the camera returns. Three held-out street scenes follow one authored 5.9 s trajectory, generated by Evoke in three steps per chunk without classifier-free guidance: a 45° turn that carries the opening view out of frame, and back. *Left to right*: opening view; turn endpoint; the bank's return render; the frame generated from it—the black band is what the bank no longer holds and the model must inpaint. *Right*: retaining only the latest 1.5 s of geometry buys nothing while the view is new and costs 0.173 of coverage at the return. Coverage bounds recall from above; it is not a fidelity measure.

Related Approaches

Evoke keeps per‑step compute constant by externalizing persistent state into a pose‑indexed bank.

Interactive world models suffer from memory bloat because the denoiser must retain an ever‑growing token history; Evoke sidesteps this by moving the persistent state into a separate, pose‑indexed bank, so each generation step sees a fixed‑size context.

Prior work falls into three camps: (1) keep history inside the denoiser via expanding KV caches; (2) externalize geometry and render view‑aligned pixels; (3) compress long‑horizon teachers into few‑step students through distillation.

Evoke combines two complementary pieces: a bounded recurrent student that reads view‑aligned geometry from an external bank, and a teacher that supplies long‑horizon, per‑chunk supervision for distillation.

The student generates a fixed‑size video chunk using only a short local history and geometry fetched from the external bank, so each step’s compute stays bounded regardless of how long the session runs.

Both the local history $h_k$ and the World State Bank $M_k$ operate under fixed budgets, so extending a session only adds more recurrent calls without enlarging the context length, positional span, or per‑step computational footprint.

Performance Overview

Genie achieves the highest average quality score across all evaluated models.

Genie 3 Happy Oyster Evoke outperforms every baseline on the Quality avg. (6) metric.

Quality avg. scores: Yume 1.5 – 78.12, Matrix‑Game 2.0 – 73.83, HY‑World 1.5 – 73.83, ar‑distill – 73.83, HY‑GameCraft – 73.83, LingBot‑World fast – 78.12, LingBot‑World v2 fast – 78.12, Genie – 82.79.

Beyond the headline metric, Genie also leads on most individual dimensions—Video Quality, Aesthetic, Imaging, and Flickering—while matching or exceeding baselines on Dynamic, Smoothness, and HPSv3‑Norm.

Long-Session Evaluation

Long rollouts stay stable and compute stays constant, confirming Evoke’s bounded runtime.

Evoke’s per‑step inference time stays constant across hour‑long rollouts, processing each 1.5 s chunk in 2.11 s on a single H200 GPU.

Measured over eight 65.5‑minute sessions (2,619 chunks each) with the World State Bank limited to the most recent 90 s of observations.

**Figure 4.** An hour-long session stays bounded rather than degrading. One continuous 65.5 min Evoke rollout, three steps per chunk without classifier-free guidance (2619 chunks; faint per-chunk, bold two-minute mean). (a) Color is flat over the hour. (b) Scene identity plateaus at cosine 0.523, the level real footage scores against itself 60 s apart. (c) Opening-window shift stays far below unrelated-scene drift. A stability claim, not fidelity (n = 1).

**Figure 6** A student inherits photometric stability from a long-horizon teacher. Two three-step students with identical distillation recipes and teachers differing only in temporal horizon; clips, trajectory, prompt, seed and inference flags are shared, so only the checkpoint differs. (a) The short-horizon student settles at 74% of its opening brightness, the long-horizon one at 101%. (b) Seven of eight clips improve (Wilcoxon p = 0.016). (c) Brightness separates the two most; sharpness not at all. Claim: exposure stability, not image quality.

Recall and Cost Analysis

Quantitative ablations reveal how the World State Bank, timed text control, and geometric memory affect recall, instruction fidelity, and compute.

We first ablate the World State Bank by measuring pose‑addressed recall: PSNR between two 12‑second windows rendered from identical camera poses after a leave‑and‑return trajectory.

Recall improves by roughly $2.5$ $dB$ once the retention window is at least as long as the time spent away.

Across 20 of 21 controlled comparisons the PSNR gain ranged from $2.3$ to $3.2$ $dB$, producing a plateau of $15.4$–$17.8$ $dB$.

Next we evaluate timed text control by inserting a clause mid‑session and measuring whether it is realized.

Mid‑session instructions succeed $63$ % more often when they target content not yet anchored in the World State Bank.

Realization rates are $67$ % for unanchored targets versus $4$ % when the instruction would replace geometry already stored.

Finally we ablate the geometric memory cost by measuring compute as session length grows.

Training Stability and Ablations

Ablation studies isolate the impact of long‑horizon teacher, window length, and timed‑conditioning on rollout stability.

Eight continuous rollouts (2,619 steps, 65.5 min, 94,281 frames) were generated with the world‑state bank limited to 90 s of retained observations; latency of 2.11 s measures diffusion wall‑clock only.

When the long‑horizon teacher is removed, photometric stability degrades noticeably, while the content‑descriptor similarity remains statistically indistinguishable.

Sharpness does not improve under the same teacher‑removal condition, confirming that the benefit is specific to low‑level appearance consistency.

The final checkpoint, taken after the long‑horizon distillation, shows no measurable drift advantage over the intermediate checkpoint, indicating that the benefit does not accumulate indefinitely.

Varying the scoring‑window length $W$ from 1 to 13 chunks yields only marginal changes once $W\ge2$; no abrupt performance jump appears, disproving a simple window‑coverage explanation.

Across 5,749 logged training steps, stochastic variation among teacher‑critic evaluations dwarfs the systematic drift signal, though averaging across evaluations can recover the drift trend.

Timed‑conditioning ablation shows that mid‑session prompt changes are supported by the per‑chunk conditioning representation, yet short‑ and long‑horizon teacher variants do not differ significantly in event‑realization rate at the current sample size.

**Figure 11.** Four-minute Evoke Teacher rollouts under a twelve-prompt schedule. Four sessions (a–d) generated by the Evoke Teacher rather than the three-step student, each driven by twelve instructions held for 20 s apiece. The grey strip above each block is that schedule on a common time axis: the eight segments illustrated below are filled dark, the four that are not remain pale. Each tile is one frame selected inside its own segment rather than sampled at a fixed offset.

**Figure 9** Both distillation stages train stably. *Long-distill* (Evoke Teacher, 6 × 8 GPUs, 1981 steps) and the short *post-distill* continuation that yields the released student; faint per-step, bold 120-step exponential moving average. (a) DMD stays bounded in both. (b) The critic converges. (c) The normalized gradient entering the loss plateaus; we release at its onset. (d) Gradient norms stay bounded across 48 GPUs and eight scheduler restarts.

**Table 2.** Evoke on two public leaderboards, against the top-10 of each. Higher is better; the full tables are Tables 3 and 4 in the appendix.

The table presents a comparison of various video generation models across two main categories: "Quality dimensions" and "Semantic dimensions". The models listed include IPOW, Vidu Q1, IPOC, Wan2.1, MiracleVision V5, Evoke (3 step), Veo 3, and LanDiff. The metrics evaluated include Subject Cons., Background Cons., Temporal Flicker, Motion Smooth., Aesthetic, Imaging, Dynamic Degree, Object Class, Multiple Objects, Human Action, Color, Spatial Rel., Scene, Appearance Style, Temporal Style, and Overall Cons.

The protocol deviations listed for VBench‑2.0 (single sample, 5.875 s clips, 640 × 384 resolution, prompt augmentation) and VBench‑Long (single sample, 8.875 s clips) introduce small variance but preserve the relative ordering of models.

VBench-Long Leaderboard

VBench results rank models and detail per‑dimension performance, highlighting Evoke’s strengths.

This section reports VBench‑Long leaderboard rankings and per‑dimension VBench‑2.0 scores for the evaluated models.

Table 3 compares models on Total, Quality, and Semantic metrics, providing a baseline for the detailed breakdowns.

Evoke samples in three steps without classifier‑free guidance, using three network evaluations per chunk versus peers’ multi‑step samplers.

The four protocol deviations of Table 3 apply unchanged, and diversity augmentation costs 3.38 points.

WBench Leaderboard

Shows per‑dimension VBench‑Long scores and the WBench leaderboard rankings.

This section reports the full per‑dimension VBench‑Long results (Table 6) and places Evoke on the WBench navigation split (Table 1) and the public WBench leaderboard (Table 7). All scores are de‑normalized to the common 0‑100 scale, and Evoke’s three‑step, no‑CFG sampling is contrasted with peers’ many‑step defaults.

**Table 6.** Per-dimension VBench-Long breakdown for the systems of Table 4. Raw scores in [0, 100], before the min-max normalization the aggregates apply; higher is better, bold is best, underline second best.

**Table 1.** WBench navigation split, $n = 158$ cases, identical for all systems. All scores $\in [0, 100]$, higher is better; **bold** is best, <u>underline</u> second best. Group rows are the unweighted mean over their group's metrics; peer numbers are as reported by their authors.

**Table 7.** WBench public leaderboard, Navi split ($n = 158$ cases), top ten by Average, Evoke included. Average is the unweighted mean of the five group scores; peer rows are the public leaderboard as of 2026-08-12. All scores $\in [0, 100]$, higher is better; **bold** is best, <u>underline</u> second best.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers