In-Distribution Forcing for Long Video Generation at Test Time
Jeongwoo Shin, Youngyoon Choi, Sangwoo Jo, Hyunmog Kim, Sungjoon Choi, Joonseok Lee, Jaewoong Choi, Jaemoo Choi
In-Distribution Forcing stabilizes long-horizon video generation by aligning KV caching and conditioning with training.
How can we prevent autoregressive video diffusion models from drifting (color/texture shift, motion decay) during long-horizon generation without retraining?
Autoregressive video diffusion models struggle to generate long videos because the model's internal state drifts as it extrapolates beyond its training horizon. In-Distribution Forcing (ID-Forcing) fixes this by ensuring every key-value (KV) operation at test time matches the configurations seen during training. It uses "self-caching" to prevent out-of-distribution cache states and retains the first chunk as a persistent condition. This approach enables minute-scale video generation that maintains consistent motion and color, significantly outperforming existing test-time conditioning methods.
Paper Primer
The core issue is the "KV-provenance problem": when a model generates a long video, it caches new chunks based on a context that differs from the training distribution. Because these cached entries are themselves used as conditioning for future chunks, the model's internal representation of the video's history becomes increasingly distorted, leading to color shifts and frozen motion.
ID-Forcing acts as a two-level controller for the model's memory: it forces the cache to stay in-distribution by restricting how new entries are written, and it forces the conditioning window to match training by fixing the first chunk as a permanent anchor. Self-caching is the primary mechanism: it caches chunks by attending only to a restricted, training-consistent provenance, effectively resetting the "history" of the cache to a state the model recognizes.
ID-Forcing significantly reduces drift and preserves motion dynamics in minute-scale video generation.
In a 2AFC user study, participants preferred ID-Forcing over all tested baselines (Self-Forcing, Deep Forcing, ∞-RoPE, and MemRoPE) across every evaluated dimension, including color consistency and dynamic degree. Participants preferred ID-Forcing over the base Self-Forcing model in 99% of cases for temporal flickering and 96% for overall preference.
Why is this method necessary if we could just train models on longer videos?
Training on longer horizons is computationally expensive and inherently limited; because autoregressive models are designed to generate videos of arbitrary length, they will always eventually encounter an extrapolation horizon that exceeds their training data.
How does this differ from existing "attention sink" techniques?
While attention sinks retain early KV entries to stabilize generation, ID-Forcing treats the first chunk as a distributionally unique requirement for the model's 3D-VAE, retaining it specifically to maintain the integrity of the initial latent frame.
ID-Forcing demonstrates that long-horizon stability is a matter of distributional alignment rather than just context management, allowing existing autoregressive models to scale to minute-long outputs without retraining.
Introduction and Motivation
We expose why autoregressive video models drift and introduce ID-Forcing to keep generation in‑distribution.
Autoregressive video diffusion models generate high‑quality short clips, but when they are asked to produce long videos the colors shift, textures blur, and motion dynamics decay—a phenomenon we call drift. Existing approaches rely on KV conditioning, assuming that cached key‑value entries remain in‑distribution, an assumption that breaks once generation exceeds the training horizon. Consequently, the KV cache itself becomes a source of out‑of‑distribution conditioning, leading to progressive degradation.
Drift is the gradual loss of color fidelity, texture consistency, and motion coherence that occurs when a model’s KV cache entries are no longer drawn from the training distribution.
**Figure 1.** **ID-Forcing extends autoregressive video generation far beyond the training horizon.** In-distribution KV operations suppress color and motion drift, enabling stable minute-scale generation.
Drift is fundamentally an out‑of‑distribution conditioning problem.
Autoregressive Video Diffusion
How the KV cache drives chunk generation and why its conditioning matters.
Chunked autoregressive diffusion models generate video by factorizing the joint distribution over latent chunks, writing each chunk once into a key‑value (KV) entry and later conditioning on those entries via attention. During training the model processes a fixed horizon of $N$ chunks, repeatedly denoising each chunk while attending to the growing KV cache.
Self-Forcing pretends the KV cache already contains future entries, forcing the model to generate as if the full context were present.
How does Self-Forcing differ from standard teacher forcing?
Teacher forcing feeds the ground‑truth previous token during training, while Self-Forcing feeds synthetic future KV entries during generation, so the model learns to rely on its own cached representations rather than external ground truth.
LongLive extends the KV cache window beyond the training horizon, keeping older entries alive instead of evicting them.
Why does LongLive improve over naive KV caching?
LongLive keeps KV entries alive beyond the training horizon by extending the cache window, preventing the abrupt eviction that causes a distribution shift and resulting quality degradation.
Each new chunk is generated by attending to the set of previously stored KV entries, so the model’s prediction is conditioned on the full history captured in the cache.
Chunk 0: start with $x_0^{T}\sim\mathcal{N}(0,I)$, denoise through $T$ steps using an empty KV set $\kappa_{<0}=\emptyset$, obtain clean $x_0$, then cache $\kappa_0$.
Chunk 1: denoise $x_1^{T}$ while attending to $\kappa_{<1}=\{\kappa_0\}$, producing clean $x_1$, then cache $\kappa_1$.
Chunk 2: denoise $x_2^{T}$ while attending to $\kappa_{<2}=\{\kappa_0,\kappa_1\}$, producing clean $x_2$, then cache $\kappa_2$.
After the three chunks are generated, the KV cache contains $\{\kappa_0,\kappa_1,\kappa_2\}$, which can be used for any further rollout.
The cache grows linearly with the number of generated chunks, preserving full key‑value context rather than discarding older information.
How is KV Cache Conditioning different from a simple sliding window over past frames?
A sliding window would drop older frames outright, whereas KV Cache Conditioning retains the full key‑value representations of past chunks and re‑uses them via attention, preserving richer context beyond raw pixel values.
By conditioning each new chunk on the growing KV cache, the model maintains coherence across long horizons, preparing the ground for the ID‑Forcing modifications introduced next.
The ID-Forcing Framework
ID-Forcing eliminates drift by enforcing in-distribution KV caching and conditioning during extrapolation.
Drift arises when KV entries are written under unseen contexts, causing out‑of‑distribution representations to accumulate. ID-Forcing tackles this by controlling both how KV entries are cached and how they are read.
ID-Forcing keeps every KV entry’s provenance within the distribution seen during training, preventing the cascade of out‑of‑distribution drift.
Self‑Caching writes each early KV entry by attending only to itself, guaranteeing that its provenance matches training and never becomes out‑of‑distribution.
j = 1 and j = 2 are self‑cached: \kappa_1 = E($x_1$; ∅), \kappa_2 = E($x_2$; ∅).
j = 3 is cached autoregressively on the last self‑cached entry: \kappa_3 = E($x_3$; \kappa_2).
j = 4 is cached on the previously cached entries: \kappa_4 = E($x_4$; \kappa_2,\kappa_3).
All four entries now reside in the conditioning window for generating $x_5$.
Because the first two entries never depend on external context, eviction never creates an out‑of‑distribution KV entry.
Re‑rotation rotates each KV entry by a fixed angle before reading, aligning the rolling window with the training‑time orientation.
Algorithm 1 In‑Distribution Forcing
**Figure 2.** Same conditioning, different provenance. We generate the next chunk $x_7$ from the same conditioning window $\kappa_{1:6}$, varying only how the last chunk of the horizon, $x_6$, is cached into $\kappa_6$. Case 1 (self-caching): $x_6$ attends to no prior entry, and $x_7$ shows no drifting. Case 2 (self-forcing caching): $x_6$ attends to a window never seen in training, and $x_7$ degrades. Case 3 (sink caching): keeping the first chunk does not help, as the window is still unseen in training. Thus, how a chunk is cached alone can determine whether the next chunk drifts.
**Figure 3.** **KV operations comparison.** Both illustrations show the caching window of each KV entry and the conditioning window of size $L$ used to generate chunk $x_i$. **Left:** Self-Forcing caches every entry autoregressively, so entries beyond the horizon are cached under windows unseen in training. **Right:** ID-Forcing caches the earlier entries with self-caching and the rest autoregressively on top of them, so every entry is cached under a window seen in training.
Empirical Evaluation
ID‑Forcing consistently beats baselines on drift metrics for minute‑scale video generation.
We first describe the experimental setup (4.1), then present the main results (4.2), and finally analyze component contributions (4.3).
ID‑Forcing outperforms all baselines on drift metrics, achieving a 12‑point gain in Color Drift and a 4‑point gain in Motion Drift.
Table 2 shows Color Drift improves from 60.66 to 72.37 and Motion Drift shows a comparable increase.
VBench‑Long evaluates long‑horizon video quality across several perceptual dimensions, exposing drift that short‑term metrics miss.
With self-caching, ID-Forcing yields the best score on every metric except Background Consistency and Motion Smoothness, which favor the collapsed output of Self-Forcing. Color Drift improves from 60.66 to 72.37 and Aesthetic Quality from 54.14 to 58.23, matching the reduced drifting and preserved dynamics observed in the frames.
**Figure.** Qualitative comparison of video generation methods. The figure displays two sets of video sequences (top: blue monster, bottom: corgi) generated by different techniques: Self-Forcing/LongLive (Base Model), + Deep Forcing, + $\infty$-RoPE, + MemRoPE, and + ID-Forcing (Ours). Each row represents a method, and columns represent temporal progression from 0s to 120s.
**Table 1.** Quantitative evaluation on long video generation. We report VBench-Long (Huang et al., 2024) metrics and drift metrics (color drift and motion drift) on 120- and 240-second videos, with methods grouped by base model.
**Figure 5.** Qualitative ablation on 2-minute video. Components are added from top to bottom Without self-caching, colors and textures still drift even with the sink and re-rotation, whereas the full ID-Forcing preserves the scene and color throughout the 2-minute video.
ID‑Forcing consistently outperforms baselines on drift metrics.
Supplementary Results and Ablations
Ablation study quantifies each component’s impact on video quality and drift metrics.
This appendix reports the ablation experiments that isolate the contribution of each design component.
Adding the Sink component raises Dynamic Degree by 16.86 points.
Dynamic Degree improves from 30.04 to 46.90 on the 120 s Self‑Forcing ablation.
Adding Re‑rotation on top of the Sink further raises Dynamic Degree by 18.54 points.
Dynamic Degree climbs from 46.90 to 65.44 when Re‑rotation is introduced.
Adding Self‑Caching (ID‑Forcing) on top of Re‑rotation yields a final Dynamic Degree increase of 1.75 points.
Dynamic Degree reaches 67.19 after the Self‑Caching step.
Self‑Caching also improves Color Drift by 11.71 points.
Color Drift score rises from 60.66 to 72.37.
Self‑Caching dramatically improves Motion Drift by 23.41 points.
Motion Drift score climbs from 65.65 to 89.06.
**Figure.** A user interface for comparing two AI-generated videos (Video A and Video B) based on a specific prompt describing a gold rush scene. The interface includes video players for both options and a set of evaluation criteria (Color Consistency, Background Consistency, Subject Consistency, Dynamic Degree, Temporal Flickering, and Overall Preference) where the user can select either Video A or Video B.
**Evaluation Dimensions** For each of the 6 dimensions below, you'll pick the better video. Here's what each dimension looks like when it's **bad**. * **Color Consistency**: The overall color tone shifts, looking completely different from the first few seconds. * **Background Consistency**: The background smears or changes inconsistently over time. * **Subject Consistency**: The main subject warps and reforms into something else, no longer looking like the same subject. * **Dynamic Degree**: Neither the camera nor the subject moves much at all. * **Temporal Flickering**: Brightness and texture flicker noisily from frame to frame.
**Figure 4.** Qualitative comparison on 2-minute videos. Full videos are available at this link.
Color Drift measures HSV histogram change between the first and last frames; Motion Drift measures the Dynamic Degree difference between the start and end of a video.
Both drift distances are normalized and inverted to produce scores in [0, 100], so higher values indicate more stable color and motion.
The user study employed a balanced Latin‑square design, ensuring each baseline was compared equally across 20 prompts.
Related Work and Conclusion
We review related KV‑cache methods and summarize our ID‑Forcing solution for stable long‑horizon video generation.
Video diffusion models that attend bidirectionally over the entire sequence achieve impressive visual quality, yet their computational cost grows quadratically with video length and they require a fixed output horizon at sampling time.
Autoregressive (AR) formulations instead generate video in sequential chunks, each conditioned on the key‑value (KV) cache of preceding chunks, reducing cost to linear, enabling streaming output, and allowing arbitrarily long generation at test time.
Because each AR chunk becomes the conditioning input for the next, any deviation from the training distribution accumulates during rollout, a failure commonly referred to as drifting.
Extending the training horizon to longer contexts is a straightforward mitigation, but it does not eliminate the boundary problem and becomes prohibitive for very long videos, since training cost scales with the target length.
Consequently, a test‑time extrapolation method is necessary regardless of backbone quality, and the two directions—better training and better extrapolation—are orthogonal.
Existing test‑time approaches manipulate the KV cache: Deep Forcing retains early KV entries as sinks, realigns their temporal RoPE, and compresses later entries; ∞‑RoPE reassigns temporal indices to stay within the trained range; MemRoPE caches keys without rotation and aggregates evicted entries into memory tokens, applying RoPE online.
All of these methods operate solely on KV conditioning, leaving the way each KV entry is written untouched.
In contrast, our approach tackles both KV caching and KV conditioning, ensuring that test‑time extrapolation remains within the training distribution.
We propose In‑Distribution Forcing (ID‑Forcing), a training‑free framework that stabilizes long‑horizon video generation by identifying and fixing the KV‑provenance problem—an out‑of‑distribution failure at the caching level.
Our two‑level KV management rule applies self‑caching at the caching level and enforces a window length bound $L \le N - 1$ while permanently fixing the first chunk $\kappa_0$ at the conditioning level.
Extensive quantitative and qualitative evaluations demonstrate that our method achieves highly competitive performance across diverse durations, backbones, and metrics, effectively eliminating quality drift and preserving stable video dynamics even over minute‑scale generations.