Perceptual Flow Matching for Few-Step Generative Modeling

Chuyang Zhao, Yifei Song, Hongfa Wang, Jianlong Yuan, Yuan Zhang, Siming Fu, Zhineng Chen, Huilin Deng, Haoyang Huang, Nan Duan

Perceptual Flow Matching replaces latent-space velocity regression with perceptual feature supervision to enable high-quality few-step generation.

How can we improve few-step generation in flow-matching models by changing the space in which the model learns to predict the target image?

Flow matching models typically require 35–50 sampling steps to produce high-quality images, as standard Euclidean regression in latent space forces the model to predict blurry, off-manifold averages when noise is high. Perceptual Flow Matching (PFM) shifts the training objective from latent velocity regression to a perceptual feature loss on decoded images, using pretrained models to bias predictions toward semantically valid modes. This change enables high-fidelity generation in just 4–8 steps without requiring teacher models, distillation, or auxiliary score networks.

Paper Primer

PFM treats the supervision space as the primary lever for few-step performance. By comparing decoded predictions against ground truth in a pretrained perceptual feature space, the model learns to penalize blurry, off-manifold interpolations that standard Euclidean objectives treat as acceptable.

PFM is a drop-in training modification: it decodes the model's clean-sample estimate $\hat{x}_0$ into pixel space and computes a perceptual distance $d(\phi(\hat{y}_0), \phi(y_0))$. This forces the model to prioritize perceptual coherence over latent-space mean-seeking, effectively acting as a "mode-seeking" objective that remains robust under coarse, few-step integration.

PFM achieves state-of-the-art few-step generation quality on the COCO 2014 benchmark.

In 8-step generation, PFM attains an FID of 33.93, a CLIP score of 31.70, and an HPSv3 score of 11.42, outperforming distillation-based baselines like LCM and DMD2. Reduces required sampling steps from 35–50 to 4–8 while maintaining generation quality comparable to multi-step models.

The authors demonstrate that PFM implicitly internalizes classifier-free guidance (CFG) effects, often eliminating the need for inference-time CFG. When explicit guidance is required, the model supports "baking" CFG into the training objective, allowing for guidance-enhanced generation in a single forward pass.

Why does perceptual supervision outperform standard latent-space regression for few-step generation?

Standard regression is "mean-seeking," producing blurry averages when the posterior is multimodal at high noise levels. Perceptual supervision uses pretrained features to assign disproportionately high costs to these off-manifold blurs, forcing the model to predict sharper, more coherent samples that are easier to integrate in few steps.

Does this method require a specialized architecture or teacher model?

No. PFM retains the standard flow-matching framework and requires no teacher models, distillation, or auxiliary score networks, making it significantly easier to implement and scale than existing acceleration methods.

Researchers can achieve efficient few-step generation by simply swapping the supervision space of existing flow-matching pipelines, shifting the focus from complex distillation to representation-aware training objectives.

Introduction to Perceptual Flow Matching

We expose why standard flow matching fails for few‑step generation and introduce Perceptual Flow Matching.

Flow matching produces high‑quality samples but typically requires dozens of integration steps (e.g., 35–50), leading to prohibitive latency and compute cost. Existing few‑step accelerations add auxiliary networks or distillation pipelines, which increase complexity and can introduce overfitting or mode collapse, manifesting as synthetic artifacts.

PFM moves the supervision from the VAE latent space to a pretrained perceptual feature space, which aligns better with human visual quality and thus enables accurate predictions with far fewer sampling steps.

How does PFM differ from standard flow matching?

Standard flow matching regresses velocities directly in the VAE latent space, which treats all directions equally and thus averages over multiple plausible image modes. PFM instead measures error in a perceptual feature space; this space stretches perceptual differences and compresses irrelevant variations, so the regression target aligns with the nearest perceptual mode rather than a mean of modes, enabling accurate few‑step predictions.

PFM shifts supervision from VAE latent space to perceptual feature space, dramatically improving few‑step generation.

Flow Matching Foundations

Defines flow matching and its standard velocity target.

Flow matching provides a continuous transport map that moves a noisy latent $x_t$ back toward the clean data $x_0$, enabling diffusion‑style generation without discrete denoising steps. Sampling efficiency is typically reported in terms of the Number of Function Evaluations (NFE), i.e. how many velocity predictions are required to produce an image.

Flow matching learns a vector field that continuously pushes a noisy sample toward its original data point—like a particle drifting downstream until it reaches the source of the river.

Classifier-Free Guidance

Shows how to bake classifier‑free guidance into training to cut inference cost.

Classifier‑Free Guidance (CFG) improves generation quality by interpolating between conditional and unconditional predictions, but it incurs an extra unconditional network evaluation at every sampling step, raising inference cost.

CFG steers a generative model toward the desired condition by adding a scaled difference between conditional and unconditional predictions, effectively “pulling” the output in the right direction.

Empirically we observe that Perceptual Flow Matching already cuts the reliance on CFG at inference, so models trained with the perceptual loss can generate high‑quality samples without any additional guidance.

Why Perceptual Space Works

We show how supervising flow matching in a pretrained perceptual space eliminates blur in few‑step generation.

Standard flow matching regresses toward the Euclidean posterior mean $\hat{x}^{\star}_{0}= \mathbb{E}[x_{0}\mid x_{t}]$. When the posterior is multimodal, this mean falls between modes, producing blurry, off‑manifold predictions that cannot be corrected in only a few sampling steps.

A feature space learned by a self‑supervised denoising autoencoder that groups natural images together, so distances reflect human‑perceived similarity rather than raw pixel differences.

Instead of forcing the model to predict the Euclidean mean of possible images, we supervise it to match the feature‑space barycenter, which pushes predictions toward perceptually valid modes and away from blurry averages.

How does perceptual supervision differ from simply using a stronger Euclidean loss or an untrained DAE?

A stronger Euclidean loss still measures raw pixel distance, so the midpoint remains cheap ($R_{\phi}=1$). An untrained DAE provides a nonlinear mapping but lacks semantic structure, resulting in $R_{\phi}\approx1$ as well. Only a pretrained perceptual encoder has learned to push off‑manifold averages away, yielding $R_{\phi}>1$ and thus a genuinely different target for the flow‑matching model.

Compute $\phi(a)=|0-255|=255$, $\phi(b)=|255-0|=255$, $\phi(m)=|127.5-127.5|=0$.

Perceptual distance $d_{\phi}(m,a)=|0-255|=255$, $d_{\phi}(b,a)=|255-255|=0$ (but we add a tiny epsilon $10^{-6}$ to avoid division by zero).

Plug into $R_{\phi}$: $R_{\phi}= \frac{2\times255}{10^{-6}} \approx 5.1\times10^{8}$, a huge penalty indicating the midpoint is far from the perceptual manifold.

This toy example shows that a perceptual encoder that captures a simple semantic cue (channel contrast) can assign an enormous penalty to a bland average, illustrating why pretrained features dramatically discourage blur.

**Figure 1.** (a) Flow matching (FM) on 2D spiral data, supervised in three spaces: (i) velocity regression in Euclidean space (standard FM), (ii) $x_0$ regression in a random denoising-autoencoder (DAE) perceptual space, and (iii) $x_0$ regression in a pretrained DAE perceptual space. In Euclidean space, the prediction collapses toward the data mean at small NFE, whereas supervision in the pretrained DAE space places samples on the real data manifold even at small NFE. (b) Off-manifold distance (at NFE=2) as the off-manifold penalty $R_\phi$ increases: larger $R_\phi$ yields lower off-manifold distance, i.e. better few-step generation. (c) Average of 1,000 similar face images from FFHQ in pixel, VAE latent, and DINO feature space. The pixel and VAE-latent averages are blurry (off-manifold), whereas averaging in DINO feature space preserves sharp facial structure.

Across supervision spaces, higher $R_{\phi}$ consistently yields lower off‑manifold distance, confirming that perceptual penalties directly improve few‑step generation. Increasing the perceptual feature dimension $D$ further raises $R_{\phi}$ and tightens the generated samples around the data manifold.

Experimental Setup

Key quantitative results of PFM on text‑to‑image generation.

PFM improves CLIP score by 1.27 points over the next best 4‑step method.

Table 3 shows CLIP 31.89 for PFM vs 30.62 for LCM at 4 NFE.

Table 3 compares SD3‑Medium, LCM, DMD2, and PFM across two sampling budgets (4 and 8 steps). PFM consistently attains the highest CLIP and HPSv3 scores while maintaining competitive FID, demonstrating that perceptual supervision yields higher fidelity and aesthetic quality even with few steps.

**Table 3.** Text-to-image generation results on COCO 2014 val set. We manually implemented LCM and DMD2 on SD3-Medium based on their original codebase and trained on the same dataset.

Main Results

PFM delivers top‑quality images and video with only eight sampling steps.

Standard flow matching fails when only a few sampling steps are allowed because velocity regression in latent space cannot capture high‑frequency perceptual details. Perceptual Flow Matching (PFM) fixes this by supervising the model in a pre‑trained perceptual feature space.

LCM distills a large diffusion model into a lightweight sampler by matching its latent‑space predictions.

DMD2 improves over LCM by adding a perceptual consistency term that forces the student’s intermediate features to stay close to the teacher’s.

PFM reaches a CLIP‑I score of 0.9187 with only eight sampling steps, outperforming the 8‑step DMD2 baseline by +0.0875.

Table 4 reports CLIP‑I 0.9187 for PFM versus 0.8312 for DMD2 (8 steps).

**Figure 2.** Qualitative comparison on SD3-Medium at different sampling steps.

**Figure 3.** Qualitative results of image editing. All images are generated with the same prompts. The Qwen-Image-Edit baselines use a CFG scale of 4.0, whereas PFM does not use CFG at inference. PFM closely matches the 40-step baseline with only 8 sampling steps.

PFM (8 steps) attains a total VBench score of 0.792, surpassing the 8‑step Wan2.1 baseline (0.735) and slightly exceeding the 35‑step baseline (0.774).

Table 5 lists total scores: 0.792 for PFM, 0.735 for Wan2.1 8×2, 0.774 for Wan2.1 35×2.

**Figure 4.** Qualitative results of text-to-video generation.

Ablation Studies

Ablations pinpoint which components drive PFM’s sharp few‑step generation.

We briefly restate the core premise: standard flow matching regresses velocities in VAE latent space, which fails to capture high‑frequency perceptual details in few‑step generation, whereas supervising in a pretrained perceptual feature space (PFM) restores sharpness.

**Figure 5.** **Ablation of supervision spaces.** All models are fine-tuned from SD3-Medium using the same training data and the same number of training iterations, and are evaluated with 8 sampling steps. PFM is evaluated without classifier-free guidance (CFG), while all other models are evaluated with a CFG scale of 4.0.

**Figure 7.** **Ablation of different perceptual models.** All perceptual models yield clear results in 8 steps, while a randomly initialized ViT (RandViT) produces blurry results, similar to standard flow matching.

**Figure 6.** **Ablation of classifier-free guidance.** PFM without CFG baking generates images with good fidelity, and prediction-side CFG baking yields results similar to inference-time CFG.

**Figure 9.** **Ablation of inference steps.** PFM produces high-fidelity results at 4 and 8 NFE and remains sharp even at 1 NFE, whereas the SD3 baseline is blurry at 1 NFE.

**Figure 8.** Ablation on layer depth in the perceptual model. We compute the perceptual loss using features from different layers of DINOv2. Shallow layers, which remain closer to pixel-level representations, produce blurrier results, while the deepest layers tend to collapse to fixed modes.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers