Video Generation Models: A Survey of Post-Training and Alignment
Chaoyu Li, Xiaoyi Gu, Yogesh Kulkarni, Eun Woo Im, Mohammadmahdi Honarmand, Zeyu Wang, Juntong Song, Fei Du, Xilin Jiang, Kexin Zheng, Tianzhi Li, Fei Tao, Pooyan Fazli
A comprehensive taxonomy of post-training and alignment strategies for modern video generation models.
How can we systematically align large-scale pretrained video generation models with specific user intents, controllability requirements, and domain constraints?
Pretrained video generation models often fail to follow complex user intent, maintain long-term temporal coherence, or adhere to physical constraints because they are optimized for broad data likelihood rather than specific behavioral goals. This survey introduces a unifying framework for post-training, categorizing existing methods into supervised fine-tuning, self-training and distillation, preference-based optimization, and inference-time guidance. The authors provide a structured taxonomy of these alignment techniques, mapping how different optimization signals reshape model behavior to bridge the gap between general generative priors and reliable, controllable application.
Paper Primer
The paper frames post-training as a shift from broad distribution modeling to targeted behavioral refinement. It distinguishes between implicit alignment, which modifies model parameters to internalize constraints, and explicit alignment, which uses evaluative signals to steer generation during or after training.
The core mechanism of post-training is the introduction of higher-quality, structured supervision to pretrained backbones. This allows models to adapt to specific domains or control requirements without the prohibitive cost of retraining from scratch.
Modern video generation has transitioned from GAN-based architectures to Diffusion Transformers (DiTs) as the dominant paradigm for post-training.
The survey identifies DiTs as the foundation for current alignment research due to their superior scalability and ability to model complex spatio-temporal dependencies. This shift enables the application of modular, signal-driven alignment strategies that were previously difficult to implement in earlier, less stable architectures.
Why is alignment in video generation fundamentally harder than in text or image generation?
Video alignment must manage multi-objective trade-offs—such as motion realism versus temporal stability—where small frame-level errors accumulate over time, leading to artifacts that are not present in static images or text.
What is the primary difference between supervised fine-tuning and preference-based methods in this taxonomy?
Supervised fine-tuning refines the conditional mapping from inputs to outputs using structured data, whereas preference-based methods use evaluative signals (like human feedback or reward models) to explicitly steer the model toward desired behaviors.
Introduction and Scope
We frame post‑training as the bridge from raw video priors to user‑aligned generation.
Video generation has leapt from short, low‑quality clips to high‑resolution, minute‑long sequences, yet pretrained models still stumble on intent, temporal coherence, and safety.
The pretrained video prior captures generic visual dynamics but lacks the fine‑grained behavioral constraints required for real‑world use.
Post‑training thus becomes the practical mechanism for closing the Alignment Gap: it refines the broad generative prior into a controllable, user‑aligned video generator.
**Figure 1.** An overview of the post-training and alignment in video generation, and the scope of this survey.
**Figure 2.** Research trends in post-training and alignment for video generation models (2022–Feb. 2026).
The shift from raw generation to controlled alignment is the central narrative of modern video generation research.
Video Generation Foundations
We define video generation paradigms, base models, and alignment dimensions that underpin post‑training methods.
Post‑training methods differ in alignment focus: most improve instruction adherence and temporal coherence, while a few target motion realism, aesthetic fidelity, or safety constraints.
Video generation is cast as learning a conditional distribution $p(v\mid c)$, where the conditioning $c$ determines the task.
**Figure 4 | Overview of video generation tasks.** Left: Text-to-Video (T2V) models learn a video prior to map noise to pixels guided by text prompts. Middle: Image-to-Video (I2V) injects dynamics into a static image, typically freezing spatial layers and training temporal attention modules. Right: Video-to-Video (V2V) focuses on structure-preserving editing, injecting spatial guidance to align with the source layout.
Modern video generators rely on either diffusion‑based Transformers (DiT) or autoregressive token models.
**Figure 5 | Architecture of a modern Latent Diffusion Transformer (DiT) for video generation.** The pipeline has three stages: (1) **Compression**: A spatio-temporal latent encoder compresses the input video into latent representations. (2) **Diffusion Modeling**: Stochastic perturbations are applied in latent space, and the resulting spatio-temporal tokens (blue tokens) are processed by a DiT backbone together with text embeddings (green tokens) to learn a denoising objective. (3) **Decoding**: The predicted clean latents are mapped back to pixel space by a spatio-temporal latent decoder.
Methods that fine‑tune a pretrained video model on labeled instruction‑response pairs, directly optimizing for instruction adherence.
Techniques that generate pseudo‑labels from the model itself or a teacher, then refine the model without explicit human annotations.
Methods that learn a reward model from human preferences and fine‑tune the generator to maximize that reward.
Approaches that modify generation at inference without altering model weights, often via conditioning injection or latent manipulation.
**Figure 3.** Taxonomy of post-training and alignment in video generation models. Methods are grouped by alignment type: Supervised fine-tuning and self-training and distillation methods provide implicit alignment. Preference- and reward-based methods achieve explicit alignment. Inference-time methods function as hybrid mechanisms, supporting both implicit and explicit alignment. The bottom node lists representative post-training datasets, benchmarks, and evaluation protocols.
Alignment in video generation must satisfy multiple, sometimes competing, objectives beyond pure likelihood.
Supervised Fine-tuning
Supervised fine-tuning aligns pretrained video generators to user intent without full retraining.
Supervised fine-tuning is the main lever for turning generic video generators into controllable, domain‑aware systems while keeping the original backbone intact.
SFT adjusts a pretrained generator by feeding it paired inputs $x$ and target videos $v$, letting the model learn a tighter mapping from user signals to desired outputs.
How does SFT differ from preference‑based alignment methods?
Preference methods optimise a reward model that scores generated videos, whereas SFT directly minimises a pixel‑ or latent‑space loss against explicit target videos, giving deterministic control without needing a learned reward.
These methods fine‑tune a pretrained diffusion model on a tiny set of text‑video pairs, often a single example, by adding a spatio‑temporal attention module that re‑uses the strong image prior for video synthesis.
These approaches inject a static image as an additional conditioning signal, allowing the pretrained video prior to be steered by visual context while still following textual instructions.
These systems edit existing videos according to natural‑language commands, preserving temporal coherence and physical plausibility through curriculum learning, optical‑flow guidance, or multimodal instruction modules.
These works focus on reducing the computational and memory footprint of fine‑tuning by updating only a small set of parameters or redesigning attention blocks, while still achieving strong instruction alignment.
Domain Adaptation and Control
We map domain adaptation and controllability approaches, highlighting their mechanisms, supervision signals, and efficiency trade‑offs.
Domain adaptation aligns pretrained video generators to new distributions while preserving general spatiotemporal priors; controllability methods then expose explicit control signals for fine‑grained manipulation.
LCT extends a pretrained single‑shot diffusion model to handle entire scenes by enlarging the context window and introducing interleaved 3D positional embeddings together with an asynchronous noise schedule.
Mission Balance addresses medical video synthesis under severe data scarcity by decoupling spatial fidelity from temporal dynamics in a two‑stage fine‑tuning pipeline.
These works inject relational and physics cues distilled from foundation models (VideoREPA) or jointly learn video generation with auxiliary physics prediction (RoboScape) to enforce physical plausibility.
These methods preserve pretrained spatiotemporal priors while adapting to a new domain: PYoCo replaces the image‑noise prior with a video‑noise prior, CREPA aligns hidden states across neighboring frames, and TIC‑FT inserts temporally structured conditioning buffers.
SimDA inserts lightweight spatial and temporal adapters into a pretrained image diffusion backbone, while LoRA‑style modules are added to cross‑attention layers to specialize visual style on small datasets.
ShotAdapter extends a single‑shot generator to multi‑shot video synthesis by introducing a transition token and a local attention mask that permits shot‑specific prompting.
These works provide explicit temporal and viewpoint control: MotionBooth fine‑tunes on subject images with motion and camera objectives; CoMo decomposes motions into reusable primitives; VideoComposer fuses textual, spatial, and motion vectors via a spatio‑temporal encoder; VD3D injects 3D camera embeddings into diffusion transformers.
FACTOR conditions generation on entity descriptions, bounding‑box trajectories, and reference images; DragEntity treats objects as latent entities with relational constraints; PuppetMaster adds part‑level drag tokens for articulation control.
VideoStudio translates prompts into structured multi‑scene scripts executed by a diffusion model; VideoAgent treats generated videos as intermediate plans, iteratively refining and selecting them before final execution.
Apollo employs a progressive pretrain‑post‑train curriculum that first learns unimodal generation then aligns audio‑video pairs through joint training and constrained multi‑task optimization.
Personalization and Style Adaptation
Personalization and data pipelines reshape video generation toward identity fidelity and scalable supervision.
Across personalization and data‑pipeline research, the dominant pattern is to preserve subject identity while injecting lightweight, post‑training adapters, and to replace costly dense annotation with structured, automatically generated supervision.
Personalization tailors a pretrained video generator to a specific subject or style, ensuring that the output consistently reflects the target identity across varied motions, viewpoints, and conditioning signals.
This camp focuses on preserving a subject’s visual identity while allowing free motion. Methods such as MagicMirror and DualReal introduce lightweight adapters or stage‑aware controllers that inject identity cues into the diffusion process without full model retraining.
This camp extends personalization beyond faces to full characters and multimodal inputs. Techniques such as HunyuanPortrait, FairyGen, UniAnimate‑DiT, and HunyuanCustom decouple motion from identity and inject modality‑specific controls (audio, text, images) via attention‑based adapters.
This camp tailors personalization to concrete downstream tasks such as virtual try‑on, product demonstration, and mobile‑friendly video creation. DreamVVT, DreamActor‑H1, and MobileVidFactory embed subject‑specific constraints into diffusion transformers to balance realism, efficiency, and stylistic consistency.
This camp enriches training data with intermediate semantic signals—object masks, pose tracks, audio‑visual alignments—to guide temporal modeling. Pipelines such as DreamVE, LoVoRA, MATRIX, Phantom, and HuMo generate these signals automatically, reducing manual annotation effort.
This camp generates supervision signals synthetically or via learned evaluators, sidestepping dense manual annotation. LinkTo‑Anime renders optical flow from controllable animation states, while VideoScore learns a feedback model that approximates human judgments for large‑scale supervision.
Self-training and Distillation
Self‑training and distillation refine video generators without full retraining.
Self‑training reuses model‑generated data for supervision, while distillation compresses teacher knowledge into cheaper students.
Self‑training lets a model improve itself by treating its own outputs—or those of a stronger teacher—as pseudo‑labels, thereby sidestepping the need for fresh human annotations.
How does self‑training differ from standard supervised fine‑tuning (SFT)?
Supervised fine‑tuning updates the model on human‑written instruction‑response pairs, whereas self‑training creates its own supervision from generated videos or latent trajectories, eliminating the need for new annotations.
Methods in this camp reuse the model’s own outputs, intermediate representations, or lightweight adapters as supervision. They operate either offline (training on self‑generated videos) or online (adapting a small set of parameters during inference). The primary benefit is improved temporal consistency and controllability without additional labeled data.
Distillation methods transfer behavior from a large, often diffusion‑based teacher to a more efficient student. Supervision can be distributional, trajectory‑level, or adversarial, and is typically applied to accelerate inference while preserving visual fidelity and temporal coherence.
Inference-Time Steering
Inference-time methods steer pretrained video generators using post‑trained signals without further training.
Inference‑time alignment operationalizes post‑trained signals by steering video generation during sampling, preserving the pretrained prior while enforcing user‑specified semantics, motion, and physical constraints.
The common thread across the surveyed approaches is inference‑time steering, which we now define.
During generation the model’s sampling process is guided by external signals, but the model weights remain frozen.
How does inference‑time steering differ from fine‑tuning the model?
Fine‑tuning changes the model’s weights and requires additional training data, whereas inference‑time steering leaves the weights untouched and manipulates the sampling trajectory using signals that are applied only at generation time.
This camp modifies the denoising trajectory directly, injecting instance‑aware or spatially localized guidance while the backbone remains unchanged. Techniques such as InstanceV use learned masks to preserve small objects, ALG applies low‑pass filtering to the conditioning image early in diffusion, and Masked Latent Adaptation confines guidance to task‑relevant latent regions.
This camp employs closed‑loop feedback during inference. DragVideo uses user‑provided drag signals to iteratively align motion, DFVEdit performs cyclic latent and flow updates for zero‑shot editing, FlashI2V gradually shifts the noise distribution to reduce conditional leakage, and MotionAgent adds a “rethinking” planning step to verify and adjust motion before it propagates.
Methods in this camp rely on auxiliary models to provide external semantic or physical feedback. CSVC optimizes text prompts under a causal graph using a vision‑language model, SynMotion decomposes textual descriptions into motion‑relevant components, and Diff Phy employs external physics rule checkers to filter trajectories that violate physical plausibility.
This camp structures inference as a sequence of stages, each with a distinct role. iDiT‑HOI uses a two‑stage diffusion transformer where the first stage captures coarse motion and the second refines hand‑object interactions. Raccoon adopts a similar multi‑stage pipeline for stable long‑horizon generation, and WMReward applies reward‑model guidance within a direct‑trajectory framework.
**Table 4.** Summary of inference-time methods via post-trained signals for video generation.
Cross-Family Comparison
Survey of post‑training families and how they combine in multi‑stage pipelines.
Architecture does not determine alignment efficacy; families complement each other and are often chained in multi‑stage pipelines.
Directly optimizes the generator on paired video data or structured prompts, aligning the model to specific user inputs. It excels when high‑quality labeled examples are available, enabling precise mappings from conditions such as pose or identity to target videos.
Improves alignment without new labeled data by leveraging a stronger teacher or the model’s own outputs to generate pseudo‑aligned targets. The goal is to scale alignment, boost robustness, or compress the model for deployment.
Optimizes the generator using comparative or scalar feedback that captures human judgments or task‑specific metrics, allowing alignment of complex, multi‑objective behaviors such as semantic fidelity, temporal coherence, and identity consistency.
Modify the generation process at test time, providing flexible control without altering model weights. These methods are attractive when retraining is impractical due to compute, data, or deployment constraints.
Datasets and Benchmarks
Mapping datasets, benchmarks, and evaluation pipelines that define video‑generation alignment.
Post‑training datasets and benchmarks expose a three‑way split of alignment—instruction, temporal consistency, and physical plausibility—yet no single resource covers all three.
Methods that refine a pretrained video generator on datasets with explicit instruction, temporal, or identity supervision. They typically use a modest‑size backbone (e.g., CogVideo 2B) and report strong gains on metrics that match their supervision, such as subject consistency and controllability.
Approaches that generate pseudo‑labels from a teacher model and distill them into a student, aiming to improve temporal stability and reduce inference cost. They operate on larger video corpora (e.g., OpenHumanVid) and retain the original backbone size.
Methods that learn a reward model from human or synthetic preference data and fine‑tune the generator to maximize that reward. They typically require larger backbones (13 B–14 B) and report the highest scores on human‑centric dimensions.
Techniques that steer generation at sampling time using external critics or control signals, without updating model weights. They are lightweight (often 2 B backbones) and focus on controllability and efficiency.
The field still disagrees on which alignment dimension should dominate benchmark design; some works prioritize instruction fidelity, while others stress temporal or physical realism.
**Table 5.** Datasets used for training in video generation post-training and alignment.
**Table 6.** Representative benchmarks used for video generation post-training and alignment evaluation.
**Table 7.** Reported quantitative results of representative post-training methods on VBench. TF: Temporal Flickering, AQ: Aesthetic Quality, SC: Subject Consistency, IQ: Imaging Quality, MS: Motion Smoothness, BC: Background Consistency, DD: Dynamic Degree. Bold values indicate the best reported score within each post-training family. “-” indicates that the corresponding information is not reported in the original paper.
**Table 9.** Reported quantitative results on physical-plausibility benchmarks. Bold values indicate the best reported score within each post-training family. "-" indicates that the corresponding information is not reported in the original paper.
**Table.** Taxonomy-aware interpretation of benchmark patterns.
Challenges and Future Directions
Post‑training aligns powerful video generators to user intent without full retraining.
Pretrained video generators are strong but often misaligned with user goals; post‑training and alignment bridge this gap without full retraining.
Supervised fine‑tuning (SFT) tightly couples appearance and motion, limiting the ability to scale identity preservation, motion adaptation, and multi‑stage instruction alignment.
Decoupling appearance from motion via disentangled or dual‑branch pathways trades off identity preservation against learning new dynamics, and full disentanglement remains difficult in practice.
Current SFT methods are optimized for short clips, so long‑horizon or multi‑stage instructions cause drift, blur, and unnatural background shifts in extended videos.
Self‑training and distillation accelerate generation but introduce risks of error accumulation and temporal inconsistency.
Repeated training on model‑generated videos can cause model collapse, where early errors amplify and dominate later frames.
Distilling diffusion video generators to few inference steps often sacrifices temporal smoothness, producing flickering artifacts or abrupt motion changes.
Preference‑based and reinforcement learning aim to align generation with human preferences but suffer from coarse reward signals and unstable optimization.
Reward designs that prioritize visual sharpness and stability tend to suppress dynamic motion and miss physical errors such as interpenetration.
Video‑level reinforcement learning is sample‑inefficient and unstable because high‑dimensional trajectories cause high variance, making large‑scale training prohibitive.
Inference‑time guidance steers generation but often relies on task‑specific gradient evaluators, limiting flexibility and semantic reasoning.
Gradient‑based methods require trained evaluators, hindering the incorporation of high‑level semantic or physical constraints without additional fine‑tuning.
Combining multiple guidance signals can lead to conflicts, unstable outputs, and high computational overhead, limiting real‑time applicability.
Evaluation lacks systematic cross‑paradigm comparison, obscuring the trade‑offs introduced by different alignment objectives.
Standardized protocols are needed to compare supervised, self‑training, preference, and inference methods under matched conditions.
Over‑regularized motion or identity constraints reduce diversity, while reward‑driven optimization can cause temporal drift and reward exploitation.
Safety requires safeguards beyond quality metrics, including multimodal risk detection, distortion‑free watermarking, and explainable forensic tools.
The path toward long‑horizon, physically plausible video generation.
Preference and Reward Alignment
Preference‑based and reward‑based methods dominate recent video alignment, yet unified evaluation and scalable rewards remain open challenges.
Preference‑based and reward‑based methods now form the core of video generation alignment, unifying reinforcement learning, direct preference optimization, and multi‑dimensional reward modeling.
Instead of fixing model weights with static supervised targets, the model is steered by evaluative signals that capture the desired whole‑video behavior.
How is Preference‑based Alignment different from Supervised Fine‑tuning (SFT)?
SFT optimizes against fixed input‑output pairs, while Preference‑based Alignment optimizes against relative or trajectory‑level feedback, enabling the model to learn desired behavior without an explicit target video.
RL treats video generation as a long‑horizon decision process, optimizing a policy with respect to trajectory‑level rewards that encode temporal consistency, physical plausibility, and structured constraints. Methods range from system‑level end‑to‑end alignment (e.g., VANS, Seedance 1.0) to diffusion‑step‑level control (e.g., Phys‑AR, CamVerse). RL can also incorporate curriculum or competence‑aware schedules to improve stability.
Preference‑based optimization directly optimizes relative‑preference objectives, avoiding explicit reward modeling. It builds reliable preference pairs through automatic scoring (OmniScore), real‑video winners, or checkpoint‑based references, and structures supervision across temporal and semantic dimensions to reduce motion bias and improve stability.
Video reward models decompose human preference into multi‑dimensional signals that capture visual quality, motion consistency, identity preservation, and physical plausibility. They address reward misspecification, provide dense step‑wise feedback, and tailor evaluation to specific domains such as anime or physics‑driven scenes.
**Table 3.** Summary of preference-based and reinforcement learning methods for video generation. Entries are sorted chronologically by publication date. “-” indicates that the corresponding information is not reported or not clearly specified in the original paper.