Wan-Streamer v0.2: Higher Resolution, Same Latency

Lianghua Huang, Zhi-Fan Wu, Yupeng Shi, Wei Wang, Mengyang Feng, Junjie He, Chen-Wei Xie, Yu Liu, Jingren Zhou, Ang Wang, Bang Zhang, Baole Ai, Chen Liang, Cheng Yu, Chongyang Zhong, Jinwei Qi, Kai Zhu, Pandeng Li, Peng Zhang, Wenyuan Zhang, Xinhua Cheng, Yitong Huang, Yun Zheng, Yuxiang Bao, Yuzheng Wang, Zoubin Bi

Wan-Streamer v0.2 scales interactive video output to 640p while maintaining a 200ms response latency.

How can we increase the output resolution of a real-time audio-visual streaming model without increasing its interactive latency?

Real-time interactive agents are often limited to close-up "talking head" shots because higher-resolution video generation is too computationally expensive to maintain a low-latency conversational loop. The authors upgrade the system to 640×368 resolution by splitting the model into a single-GPU "Thinker" for low-latency control and a multi-GPU "Performer" group that uses context-parallel sequence sharding to handle the heavier visual generation. This topology preserves the original 200ms model-side latency, enabling agents to remain visibly grounded in their surroundings during real-time dialogue.

Paper Primer

The core challenge is the "Thinker-Performer" boundary: the system must process user input, update the interaction state, and generate synchronized audio-visual responses within a strict 160ms unit window. To scale resolution without increasing latency, the authors move the expensive latent generation path into a Ulysses-style context-parallel group, while keeping the language and state updates on a single, low-latency GPU.

The v0.2 upgrade triples the output resolution while preserving the 200ms model-side latency target.

The system maintains a 200ms signal-to-signal latency at 25 FPS, consistent with the v0.1 baseline, despite increasing the output stream from 192×336 to 640×368. Resolution increase of ~3.3x (pixel count) with zero increase in model-side response latency.

The serving topology successfully isolates visual generation costs from the latency-critical control path.

By using pre-sharded K/V caches and Ulysses-style sequence parallelism for video latents, the system avoids the communication overhead that would otherwise break the 160ms processing cadence.

Why is this specific split between "Thinker" and "Performer" necessary?

The Thinker must remain a single-GPU path to ensure low-latency perception and state updates, while the Performer group absorbs the high-resolution video generation cost through parallelization, preventing the visual workload from delaying the conversational loop.

Does this approach apply to all modalities equally?

No; the system treats audio and video differently. Because audio latents are much shorter, they are generated without sequence sharding to avoid the overhead of communication, whereas the high-resolution video latents are split across ranks.

Introduction and Motivation

We expose the resolution‑latency trade‑off that drives Wan‑Streamer v0.2.

Real‑time audio‑visual interaction demands sub‑second response while showing enough visual detail for scene‑grounded conversation. The original Wan‑Streamer v0.1 kept a causal, native‑streaming formulation but was limited to a 192×336 video stream, which compresses posture, objects, and layout out of view.

We must raise visual resolution without breaking the strict latency envelope imposed by streaming causality.

The core challenge is balancing higher visual resolution against the strict interactive latency budget.

Model Formulation and Design

Upgrade the streaming model while preserving latency by offloading high‑resolution generation.

The upgrade must raise visual fidelity while keeping the strict latency budget that real‑time interaction demands. The challenge is to add higher‑resolution generation without breaking the causal streaming pipeline.

The model treats all modalities as a single causal stream, updating a shared token history so that each new output can depend on everything that has happened so far.

How does this differ from a conventional encoder‑decoder pipeline?

In a standard encoder‑decoder, the encoder processes the entire input before the decoder starts, breaking causality. Wan‑Streamer instead interleaves encoding and decoding token‑by‑token on a shared history, so the decoder can immediately attend to the most recent user utterance and to its own previously generated latents.

t=0: history = [user‑text(“hello”)]

t=1: model produces audio‑latent A and appends it → history = [user‑text, audio‑latent A]

t=2: model produces video‑latent V and appends it → history = [user‑text, audio‑latent A, video‑latent V]

Next user utterance can attend to both A and V because they are already in history.

The example shows that each generated latent becomes instantly available to subsequent steps, eliminating any separate “post‑processing” stage.

**Figure 1** Wan-Streamer remains one native-streaming model: language, audio, and video inputs and outputs are represented on a shared causal timeline and coordinated by block-causal attention. v0.2 keeps this formulation while increasing the output resolution and changing the deployment strategy described in Fig. 2.

Latency-Preserving Serving Architecture

Key latency numbers show v0.2 adds high‑resolution video without slowing the interactive loop.

Wan‑Streamer v0.2 keeps model‑side latency at ≈200 ms while delivering 640×368 video at 25 FPS.

Measured under the same response boundary as v0.1; the additional latent‑generation work is confined to the Performer group.

The system separates latency‑critical inference (Thinker) from heavy latent generation (Performer) so the user‑facing loop stays fast.

How does this differ from a naïve multi‑GPU inference pipeline where all stages run in parallel?

In a naïve pipeline the latency‑critical path would still depend on the slowest stage, so adding GPUs could increase synchronization overhead. The Thinker‑Performer split isolates the critical path on a single GPU and only overlaps the heavy work in a separate group, guaranteeing the 200 ms bound.

Performer ranks split the long video latent sequence across GPUs and exchange partial results via all‑to‑all collectives, keeping each rank’s workload balanced.

Why not simply increase the batch size of a single GPU instead of sharding the sequence?

Increasing batch size would raise memory usage and still require the same sequential compute for the long latent sequence, so the per‑unit latency would grow. Sharding distributes both memory and compute, allowing each rank to finish its chunk within the 160 ms window.

Serving schedule for one streaming unit (k → k+3)

**Figure 2** v0.2 latency-preserving serving. The thinker stays on one GPU and owns perception, language/state update, K/V construction, and final decoding. The performer uses Ulysses-style context parallelism for the 640×368 latent generation path: K/V slices are written into pre-sharded performer caches, the high-resolution latent video sequence is split across ranks for denoising, and Ulysses all-to-all/gather communication stays inside the performer group. Audio latents are short and are not split across ranks.

The Thinker‑Performer split enables high‑resolution generation without increasing interactive latency.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers