DuoMatching: Joint-Marginal Distribution Matching for Few-Step Video Generation
Jiahao Zhan, Yan Wang, Yongrui Ma, Qunliang Xing, Ruchang Yao, Runtao Liu, Shijie Zhao, Tianfan Xue
DuoMatching improves few-step video generation by augmenting joint distribution matching with explicit frame-level marginal supervision.
How can we improve few-step video generation quality by simultaneously matching both the joint video distribution and the marginal frame distribution?
Few-step video generators often struggle with fine-grained visual quality and semantic alignment because they rely solely on joint distribution matching, which prioritizes cross-frame consistency over individual frame accuracy. DuoMatching introduces a joint-marginal framework that adds a dedicated marginal objective, using image generation models to provide direct frame-level supervision. A lightweight module called LatentBridge resolves the latent space mismatch between video and image models, while Latent Variation Sampling distributes this supervision across temporal segments to preserve motion dynamics. This approach consistently outperforms existing distribution matching distillation methods, achieving human preference rates above 80% across all evaluated baselines.
Paper Primer
The core move is a unified joint-marginal objective: the model optimizes for both the video teacher’s joint distribution (to maintain temporal coherence) and an image teacher’s marginal distribution (to sharpen frame-level details). DuoMatching is like a film editor who checks both the flow of the entire scene and the clarity of every individual frame, using a specialized bridge to translate the compressed video latents into the image teacher's high-resolution space.
DuoMatching significantly improves visual quality and semantic alignment in few-step video generation.
Human preference study across 400 challenging prompts. Over 80% preference rate against all evaluated baselines, including CausVid and Causal Forcing++.
The joint-marginal formulation preserves temporal dynamics while enhancing static frame quality.
VBench metrics and human evaluation of temporal/motion quality. Temporal and motion preference rates remain at or near parity with baseline models.
Why is joint distribution matching alone insufficient for high-quality video generation?
Joint matching prioritizes cross-frame dependencies, meaning improvements to an individual frame's detail might be rejected if they conflict with surrounding frames, leading to deficiencies like blurry textures or missing objects.
What is the role of LatentBridge in this framework?
LatentBridge is a lightweight module that maps temporally compressed video latents to frame-specific representations compatible with image generation models, enabling direct marginal supervision without the memory cost of decoding and re-encoding frames.
Introduction and Motivation
We expose why joint DMD fails at frame fidelity and introduce DuoMatching to add marginal supervision.
Few‑step video generators rely on joint Distribution Matching Distillation (DMD) to keep frames temporally coherent, but this joint focus leaves individual frames blurry and semantically misaligned. Adding a dedicated marginal‑matching objective can close the visual‑quality gap without sacrificing motion consistency.
DMD compresses a multi‑step video teacher into a few‑step student by aligning their joint frame distribution.
Because joint DMD only supervises the whole video, we supplement it with marginal matching that directly supervises each frame using a strong image generator.
**Figure 1.** Motivation and results of DuoMatching. Videos generated by autoregressive models often have degraded quality for later rollout frames (see Q-Align [38] curves). Joint DMD mitigates this drift, yet the student still retains a visual quality gap relative to the video teacher. The example reveals a missing feather specified in the prompt. To solve this problem, we reformulate distribution matching from a coupled joint-marginal perspective to better approximate the real video distribution, as illustrated by improved visual quality and the recovered feather, while maintaining temporal coherence.
The limitation of joint‑only DMD is its inability to capture frame‑level fidelity, which DuoMatching resolves by adding explicit marginal supervision.
Related Work
Survey of prior video generation methods and distribution‑matching techniques.
Recent few‑step video generation models achieve strong visual quality but rely on costly iterative denoising, prompting research into one‑step and distilled approaches such as VideoLCM, CausVid, and DuoMatching. Distribution‑matching techniques like DMD, Phased DMD, One‑Forcing, Reward Forcing, DMDR, Data‑Forcing, and SA‑DMD aim to align generated and real data distributions, yet they still face optimization instability and frame‑level fidelity issues. Complementary strategies incorporate image priors—e.g., AnimateDiff, MCM, and AVDM2—to boost video quality, while the present work revisits joint‑marginal matching and introduces LatentBridge and Latent Variation Sampling to improve marginal fidelity.
Joint-Marginal Distribution Matching
We introduce Joint‑Marginal Formulation, adding marginal supervision to improve frame fidelity.
Frame‑level visual quality suffers when a model only matches the joint video distribution, because the per‑frame details receive no direct supervision. Adding explicit marginal matching supplies the missing frame‑wise signal while preserving temporal coherence.
We keep the original joint video‑teacher loss but also force the generator’s per‑frame distribution to align with an image‑teacher distribution, balancing temporal consistency and frame fidelity.
Apply M to the video teacher distribution $P_v$, which assigns equal mass 1/3 to each frame, yielding $m_v$ = M $P_v$ = 1 for F₂ and 0 for F₁, F₃.
Apply M to the real‑data distribution $P$* (identical to $P_v$ in this toy case), producing m* = 1 for F₂ and 0 elsewhere.
Generate a video with the student model Q_$\theta$ that places mass 0.5 on F₂ and 0.25 on each of F₁ and F₃.
Compute the marginal m_$\theta$ = M Q_$\theta$ = 0.5 for F₂ (and 0 for the others).
Evaluate the KL terms: `D_KL`(m*‖$P_i$) = 0 if $P_i$ matches m*; `D_KL`(m*‖$m_v$) = 0 as well, illustrating the inequality.
Even with a simplistic sampling policy, the marginal term forces the generator to allocate probability mass to the selected frame, directly improving per‑frame fidelity.
How does Joint‑Marginal Formulation differ from standard joint distribution matching?
Standard joint matching only penalizes the full‑video KL `D_KL`(Q_$\theta$‖$P_v$), which can ignore per‑frame errors. Joint‑Marginal adds a second KL `D_KL`(m_$\theta$‖$P_i$) that explicitly aligns each generated frame’s marginal with an image‑teacher distribution, guaranteeing frame‑level supervision while still respecting temporal dependencies.
**Figure 2.** Spatial structure of score differences in joint and marginal distribution matching. Marginal matching exhibits more spatially coherent responses, whereas joint matching produces more scattered responses in this example. Each heatmap is independently normalized to highlight spatial structure rather than absolute magnitude.
LatentBridge Architecture
Introduce LatentBridge and Latent Variation Sampling to align video and image latents.
LatentBridge converts a temporally compressed video latent slice into a frame‑specific latent that lives in the image teacher’s latent space, letting the video model inherit the image teacher’s high‑quality frame distribution.
LatentBridge applies a simple linear map $B_{\phi}$ (for illustration) $B_{\phi}(z_{l-1},z_l,i)= [0.5,\,0.3]$, producing $u_{l,(2)}$.
The image teacher encodes the true frame $x_{l,2}$ to $E_{\text{img}}(x_{l,2}) = [0.52,\,0.28]$.
The L1 loss for this sample is $|0.5-0.52| + |0.3-0.28| = 0.04$.
During training the gradient updates $B_{\phi}$ to reduce this discrepancy, learning a per‑frame mapping despite the shared slice.
LatentBridge learns to disambiguate multiple frames from a single compressed latent, enabling frame‑level fidelity without decoding the whole video first.
Why not simply decode the video latent to RGB frames and then re‑encode with the image VAE?
That approach forces gradients to pass through two full VAE decoders/encoders, exploding memory usage. LatentBridge learns a direct, lightweight mapping, so back‑propagation stays within a single small network and the video model can be trained on commodity hardware.
Latent Variation Sampling selects latent slices that show the greatest change between consecutive frames, ensuring the limited frame‑level supervision focuses on dynamic regions rather than static ones.
Slice values: $z_1=[0,0]$, $z_2=[0.1,0.2]$, $z_3=[0.5,0.6]$, $z_4=[0.55,0.58]$, $z_5=[0.9,1.0]$.
Compute $d_1=0.025$, $d_2=0.18$, $d_3=0.001$, $d_4=0.045$ (using $CHW=2$).
Select the two largest $d_l$ values: $d_2$ and $d_4$, giving split positions $b_1=2$, $b_2=4$.
Segments: $[1,2]$, $[3,4]$, $[5,5]$.
Uniformly sample one slice from each segment: $l_1=1$, $l_2=4$, $l_3=5$.
Apply marginal DMD to the image‑aligned latents of slices $1$, $4$, and $5$.
By forcing a sample in each high‑variation segment, the supervision covers both rapid motion (slice 2) and subtle changes (slice 4), avoiding over‑focus on static regions.
Why sample uniformly within a segment instead of always picking the slice with the highest $d_l$?
Uniform sampling prevents the model from over‑fitting to the few peaks of motion; it encourages the generator to handle a range of dynamics within each segment, leading to more robust temporal consistency.
We train causal generators from the Causal Forcing++ checkpoint for 1,000 steps and bidirectional generators from the Wan2.1‑T2V‑1.3B checkpoint for 1,200 steps, always using the frozen Qwen‑Image as the image teacher. All experiments run on eight 80 GB GPUs.
We compare against autoregressive baselines (Self Forcing, Reward Forcing, Causal Forcing++, One‑Forcing) for causal generation and against the bidirectional CausVid DMD variant for bidirectional generation. Evaluation uses VBench’s official suite plus a curated 400‑prompt test set measuring Semantic Score, Aesthetic Quality, Imaging Quality, Dynamic Degree, and Motion Smoothness.
Performance Evaluation
We quantify DuoMatching’s gains on VBench and human preference.
Recall that DuoMatching adds explicit marginal matching to improve frame‑level fidelity while preserving temporal coherence.
Self Forcing forces each generated frame to match its own teacher distribution, encouraging consistency with the source video.
One-Forcing conditions the generation of a frame on a single preceding frame, tightening short‑range temporal dependencies.
Reward Forcing augments the loss with a reward term that favors frames aligning with high‑level semantic cues.
Causal Forcing++ extends causal autoregressive generation with joint‑marginal distribution matching, improving temporal coherence.
DuoMatching’s total VBench score surpasses Causal Forcing++ by 16%.
Ours achieves a total of 73.61 versus 70.56 for Causal Forcing++ (same NFE budget).
**Figure 4.** Qualitative comparison between DuoMatching and Causal Forcing++ across four aspects: fine-grained visual quality, semantic alignment, composition, and dynamics. DuoMatching improves visual details, prompt adherence, and composition while preserving video dynamics.
DuoMatching consistently outperforms baselines on VBench metrics and human preference.
Ablation Study
We quantify how each component affects video quality and resource usage.
This section isolates the contribution of each design choice by removing it and measuring the impact on video quality and memory consumption. The tables below report the quantitative outcomes of these ablations.
**Table 3.** Ablation study on the proposed components.
Using the Wan2.1‑14B image teacher raises the total video‑student score from 68.50 to 94.72.
Table 3 shows a total of 94.72 for Wan2.1‑14B versus 68.50 for the baseline.
**Table 4.** Ablation of marginal DMD and latent mapping strategies. Direct applies marginal DMD directly to temporally compressed video latent slices. Decode-Encode decodes video latents into RGB frames and re-encodes sampled frames with the image VAE, retaining gradients through both operations. Peak memory denotes the maximum GPU memory allocated to tensors during a complete training iteration. OOM indicates an out-of-memory failure.
LatentBridge improves the semantic metric by +5.20 points over the Direct baseline.
Table 4 reports 76.94 for LatentBridge versus 71.74 for Direct.
**Table 5.** Ablation of the number of sampled latent slices $K$ and sampling strategies. Temporally stratified sampling randomly selects one latent slice from each of $K$ segments of equal length.
LVS achieves the highest Total score at $K\!=\!4$, improving it by +0.05 over uniform random sampling.
Table 5 shows Total = 81.04 for LVS versus 80.99 for uniform random when $K\!=\!4$.
**Table 6.** Ablation of the marginal DMD loss weight $\omega$.
Qualitative Comparisons
Appendix C showcases DuoMatching’s visual gains and proves its marginal‑fitting advantage.
**Figure 5** Additional qualitative comparisons of DuoMatching with Reward Forcing, One-Forcing, and CausVid. DuoMatching improves visual details, semantic alignment, and composition while preserving video dynamics.
Compared with Reward Forcing, DuoMatching renders the ice dragon with clearer structure and preserves the boxer’s appearance during rapid motion. In the pianist scene, DuoMatching yields a more plausible spatial relationship among pianist, keyboard, and piano. Against One‑Forcing and CausVid, DuoMatching reduces over‑exposure and produces richer fur, feather, and clothing details, especially when the number of function evaluations (NFE) is set to 1, where motion stability improves markedly.
The proof shows that, under matched marginal capacity, fitting only the frame‑level marginal cannot incur larger KL divergence than fitting the full joint video distribution; the extra error is bounded by the conditional KL term. Consequently, marginal‑only fitting achieves no greater marginal error than joint fitting, with any advantage limited by the conditional discrepancy. This is analogous to first drawing a map’s outline before adding detailed landmarks—getting the outline right already guarantees the map’s overall fidelity.
Theoretical Analysis
Adding marginal matching improves the joint‑marginal KL, showing the optimum beats the video teacher.
Recall that few‑step video generation suffers from poor frame‑level fidelity because the joint‑marginal objective (DMD) emphasizes temporal coherence over individual frames.
We assume all involved densities are positive on a common support, KL values are finite, and $Q_{\omega}$ exists and is right‑differentiable at $\omega=0$.
Thus, a positive marginal weight $\omega$ guarantees that the joint‑marginal optimum $Q_{\omega}$ is strictly closer to the ideal distribution than the original video teacher, confirming the benefit of explicit marginal matching.
LatentBridge Efficiency
We assess LatentBridge’s efficiency, reconstruction fidelity, and loss‑transfer guarantees.
LatentBridge is a lightweight mapper that translates video latents into image latents, enabling fast cross‑modal generation.
**Figure 6** Qualitative evaluation of LatentBridge across image latent spaces. Top: target latents obtained by encoding the corresponding RGB frames with each image VAE. Bottom: reconstructions from latents predicted by LatentBridge from video latents of Wan2.1. Both rows use the corresponding image VAE decoder.
**Figure 7.** Latent reconstruction error of LatentBridge at different stages of generator training. Given a generated video latent slice $z^l$, with $z^{l-1}$ and the local frame index $i$ as conditioning, LatentBridge predicts the corresponding image latent. We measure its $\ell_1$ error against the target obtained through Decode-Encode over 100 samples.
Using the predicted image‑latent $u_{\\theta}=B_{\\phi}(z_{l-1}^{\\theta},z_{l}^{\\theta},i)$ and the video‑latent $v_{\\theta}=E_{\\text{img}}[D_{\\text{video}}(z_{\\theta})]_{t(l,i)}$, the paper shows that a Lipschitz bound on the frame‑level loss transfers reductions from $J_{B}$ to $J_{R}$ when the mapping error is sufficiently small.
If the observed loss drop $\\delta=J_{B}(\\theta_{0})-J_{B}(\\theta_{1})$ exceeds $L_{\\ell}(\\varepsilon_{0}+\\varepsilon_{1})$, then the same frame‑level loss evaluated on decoded video frames also decreases, guaranteeing that improvements measured through LatentBridge reflect true video quality gains.
Training Procedure
Algorithm 1 details the full training loop for DuoMatching.
The training procedure for DuoMatching alternates between updating the generator with joint and marginal DMD losses and refreshing the fake estimators on newly sampled data.
Algorithm 1: Training of DuoMatching
**Algorithm 1** Training of DuoMatching **Require:** Generator $G_\theta$; frozen LatentBridge $B_\phi$ **Require:** Frozen teachers $\mu_{\text{real}}^v, \mu_{\text{real}}^{\text{img}}$ **Require:** Trainable fake estimators $\mu_{\text{fake}}^v, \mu_{\text{fake}}^{\text{img}}$ **Require:** Marginal weight $\omega$; sampling budget $K$; update interval $R$; warmup length $T_{\text{warm}} > 0$; training iterations $N$ 1: **for** $n = 0, \dots, N - 1$ **do** 2: **if** $n \mod R = 0$ **then** **Generator update** 3: Sample text condition $c$ and noise $\xi \sim \mathcal{N}(0, I)$ 4: $z \leftarrow G_\theta(\xi, c)$ 5: $\{l_k\}_{k=1}^K \leftarrow \text{LVS}(z, K)$ 6: **for** $k = 1, \dots, K$ **do** 7: Sample local frame index $i_k$ 8: $u_k \leftarrow B_\phi(z^{l_k-1}, z^{l_k}, i_k)$ $\quad \triangleright u_k = u^{l_k, (i_k)}$ 9: **end for** 10: $U \leftarrow \text{Stack}(u_1, \dots, u_K)$ $\quad \triangleright$ Retain gradients through $B_\phi$ 11: Sample image timestep $\tau$ and $\epsilon \sim \mathcal{N}(0, I)$ 12: $U_\tau \leftarrow \alpha_\tau^{\text{img}} \text{sg}(U) + \sigma_\tau^{\text{img}} \epsilon$ 13: $s_{\text{real}}^{\text{img}} \leftarrow \text{Score}(\mu_{\text{real}}^{\text{img}}; U_\tau, \tau, c)$ $\quad \triangleright$ Image teacher; no autograd 14: $s_{\text{fake}}^{\text{img}} \leftarrow \text{Score}(\mu_{\text{fake}}^{\text{img}}; U_\tau, \tau, c)$ $\quad \triangleright$ Fake estimator; no autograd 15: $\Delta s \leftarrow s_{\text{fake}}^{\text{img}} - s_{\text{real}}^{\text{img}}$ $\quad \triangleright$ Score difference 16: $g \leftarrow \alpha_\tau^{\text{img}} w_{\text{img}}(\tau) \Delta s$ $\quad \triangleright$ DMD gradient with respect to $U$ 17: $\tilde{\mathcal{L}}_{\text{marginal}} \leftarrow \frac{1}{2} \text{MSE}(U, \text{sg}(U - g))$ $\quad \triangleright$ Marginal DMD surrogate loss 18: $\mathcal{L}_{\text{joint}} \leftarrow \text{JointDMD}(z, c, \mu_{\text{real}}^v, \mu_{\text{fake}}^v)$ $\quad \triangleright$ Joint DMD loss 19: $\omega_n \leftarrow \omega \min(1, n/T_{\text{warm}})$ 20: $\tilde{\mathcal{L}}_{\text{DuoMatching}} \leftarrow \mathcal{L}_{\text{joint}} + \omega_n \tilde{\mathcal{L}}_{\text{marginal}}$ $\quad \triangleright$ Joint DMD + marginal DMD 21: Update $\theta$ using $\nabla_\theta \tilde{\mathcal{L}}_{\text{DuoMatching}}$ 22: **end if** **Fake estimator updates** 23: Sample fresh text condition $c'$ and noise $\xi' \sim \mathcal{N}(0, I)$ 24: $\bar{z} \leftarrow \text{sg}(G_\theta(\xi', c'))$ 25: $\{\bar{l}_k\}_{k=1}^K \leftarrow \text{LVS}(\bar{z}, K)$ 26: **for** $k = 1, \dots, K$ **do** 27: Sample local frame index $\bar{i}_k$ 28: $\bar{u}_k \leftarrow \text{sg}(B_\phi(\bar{z}^{\bar{l}_k-1}, \bar{z}^{\bar{l}_k}, \bar{i}_k))$ 29: **end for** 30: $\bar{U} \leftarrow \text{Stack}(\bar{u}_1, \dots, \bar{u}_K)$ 31: Sample fresh image timestep $\tau$ and $\epsilon \sim \mathcal{N}(0, I)$ 32: $\bar{U}_\tau \leftarrow \alpha_\tau^{\text{img}} \bar{U} + \sigma_\tau^{\text{img}} \epsilon$ 33: Update $\mu_{\text{fake}}^{\text{img}}$ with flow-matching target $\epsilon - \bar{U}$ $\quad \triangleright$ Fake estimator for marginal DMD 34: Update $\mu_{\text{fake}}^v$ using the video flow-matching loss on $(\bar{z}, c')$ $\quad \triangleright$ Fake estimator for joint DMD 35: **end for**
Conclusion
Appendix A provides concluding remarks, acknowledgments, references, and hyperparameter ablation details.
We presented DuoMatching, a unified joint‑marginal distribution matching framework for few‑step video generation. By adding explicit marginal DMD, the model improves frame‑level fidelity while preserving motion dynamics.
We thank Shuai Ma for naming DuoMatching and Kunyu Feng and Zhenhua Yang for their helpful suggestions on the manuscript.
The bibliography lists prior work on video diffusion, distribution‑matching distillation, and related generative modeling techniques.
Increasing the marginal DMD loss weight to $\omega$ = 0.4 yields the best overall scores, while $\omega$ = 0.8 degrades dynamics, confirming that excessive marginal supervision harms temporal consistency.