InfinityEdit: Infinite Video Editing with a Lightweight Edit-Ignition Adapter

Yunze Tong, Mushui Liu, Canyu Zhao, Shiyi Zhang, Didi Zhu, Peng Zhang, Wanggui He, Jinlong Liu, Ying Chen, Hao Jiang, Pipei Huang, Bo Zheng

InfinityEdit enables continuous, instruction-based video editing by injecting edits into a frozen streaming generator.

How can we perform continuous, sequential video editing on long streams without the temporal drift or quality degradation typical of in-place editing methods?

Existing video editors operate on fixed-length clips, treating edits as a one-time rewrite. This approach fails for open-ended streams where edits must persist and evolve as new content arrives. InfinityEdit introduces a lightweight adapter that "ignites" an edit in the current chunk, allowing the frozen backbone to carry the edited content forward indefinitely. The system maintains high visual quality and edit faithfulness over long, multi-round sequences, outperforming existing in-place and prompt-switching baselines.

Paper Primer

Infinite video editing requires a model to generate a temporally subsequent segment that satisfies an edit request while maintaining continuity with the preceding stream. Unlike in-place editing, which rewrites a static clip, this task demands that the model preserve scene identity across an unbounded sequence of edits without accumulating drift or quality degradation.

InfinityEdit attaches a lightweight adapter to a frozen streaming diffusion transformer. The adapter uses three attention modules: history cross-attention to anchor the new chunk to the past, temporal causal self-attention to propagate cues forward, and edit cross-attention to inject the instruction. The system acts like a relay runner: the adapter "ignites" the edit in the first chunk, and the frozen backbone carries the baton forward until the next instruction arrives.

InfinityEdit maintains superior edit faithfulness and temporal stability across sequential editing rounds.

In a 3-round sequential editing benchmark, the method achieved the highest scores in Edit Faithfulness (3.82) and Cross-Edit Coherence (3.84) compared to baseline methods. The method demonstrated robust stability with a standard deviation of only 0.023 across editing rounds, significantly outperforming prompt-switching baselines (std > 0.69).

Why can't we just use existing streaming generators and change the prompt when an edit arrives?

Simple prompt switching often conflicts with the generator's internal stability mechanisms, such as anchor frames, which tend to pull the generation back toward the original, unedited content. This causes the model to ignore or weaken the edit.

How does the system prevent the video from degrading after many edits?

The adapter is trained with history corruption, exposing it to imperfect generated frames during training. At inference, the system uses a sliding history window and resets the anchor frame to the first edited frame of each new segment, preventing the model from reverting to the source.

The Problem of Infinite Editing

We expose the in-place editing bottleneck and define infinite video editing.

Existing video editors treat editing as an “in‑place” operation: they align each output frame with the corresponding frame of a fixed source clip. This assumption works for short, bounded clips but collapses when the video stream continues beyond the original window, causing temporal drift and quality loss.

Current editors assume the edit can be applied directly onto a static source clip, aligning every output frame with the matching input frame.

We therefore define *infinite video editing*: given a preceding segment and an edit request, the model must generate the next segment that continues the stream while applying the edit, repeating this process for an unbounded sequence of instructions. This setting introduces two core challenges—first, the edited segment must be a faithful continuation rather than a simple frame‑wise rewrite, and second, quality must remain stable as edits accumulate, because each generated segment becomes the history for the next.

Current methods fail on long streams because their in‑place editing assumption cannot propagate edits beyond a fixed window, leading to drift and instability.

Background and Prior Methods

We situate our task among video generation, editing, and streaming literature.

Recent diffusion research has expanded from static images to video, spawning both generative backbones and editing pipelines.

A transformer‑based diffusion backbone that models spatio‑temporal attention across video frames.

Open‑source models built on DiTs, such as CogVideoX, Wan, and HunyuanVideo.

Models that push visual quality, motion realism, and video length, e.g., SkyReels‑V4, Sora 2, Seedance 2.0.

Instruction‑driven approaches (InsViE, Ditto, OpenVE, Kiwi‑Edit) and semantic control techniques (RefVFX, Video‑As‑Prompt) that edit a source clip within its original temporal window.

Autoregressive continuation methods (CausVid) and low‑latency streaming techniques (step distillation, sparse context retrieval, hierarchical caching) that enable generation of arbitrarily long videos.

Systems that edit live streams frame‑by‑frame (SANA‑Streaming, LiveEdit) and those that continue a stream under changing prompts (LongLive, Anchor Forcing, CausalCine).

A model that produces video chunk by chunk, keeping a bounded history as conditioning while allowing new frames to be appended indefinitely.

The goal is to apply a user edit to an ongoing video stream and then keep generating future frames that respect the edit.

Helios is a 14 B autoregressive diffusion transformer that generates video chunk by chunk, keeping a clean history and compressing it into a constant‑size memory.

Data Collection Pipeline

We detail the pipeline that builds high-quality edit triplets for training.

Training a streaming editor requires reliable triplets (source video, edit instruction, target video). Existing pipelines that assume “in‑place” editing struggle to produce consistent, long‑range data, so we construct a dedicated collection process that mitigates drift and quality collapse.

**Figure 1.** Our data collection pipeline.

We turn abstract edit types into concrete, video‑specific instructions by first matching source videos to their dominant entities and then prompting a large language model to elaborate.

Group Clip A under entity “dog” and Clip B under entity “car”.

Prompt Gemini 3 Flash with (“dog”, “change color”) → “make the dog’s fur golden”.

Prompt Gemini 3 Flash with (“car”, “change color”) → “paint the car teal”.

Resulting instructions $c_{edit}$ are “turn the dog’s fur to gold” and “repaint the car to teal”.

Grounding abstract edits to concrete entities prevents nonsensical instructions like “change color” applied to a background sky.

Why do we group videos by caption entities before expanding edit types?

Grouping ensures the language model receives a concrete object reference, so the expanded instruction can specify exactly which visual element to modify rather than producing a vague, ungrounded edit.

We synthesize the target video $V_{tgt}$ so that its first frame aligns with the source’s last frame, then apply the appropriate edit to that connection frame before generating the rest of the clip.

Extract $X_{con}$ (the last frame of Clip A).

Pass $X_{con}$ and “make the dog’s fur golden” to Qwen‑Image‑Edit‑2511 → $X'_{con}$ showing a golden‑fur dog.

Feed $X'_{con}$ to Wan2.2‑I2V‑A14B, which generates a 2‑second video where the dog runs while its fur remains golden.

Using the edited connection frame as the seed guarantees that the visual change appears from the very first generated frame, eliminating a flicker that would occur if the edit were applied later.

Why don’t we edit the entire source clip instead of just the connection frame?

Altering only the connection frame is sufficient because the image‑to‑video model propagates that visual change throughout the subsequent frames, while preserving the original motion dynamics.

We adapt the raw triplets to the backbone’s input format and filter them for quality using a four‑criterion human rating.

Compute the average score: (3 + 4 + 2 + 3) / 4 = 3.0.

Since the average exceeds the threshold of 2.5, the triplet is kept for training.

If any individual criterion fell below 2, the annotators would flag the triplet for manual review.

Aggregating multiple criteria balances different aspects of quality, preventing a triplet with a perfect alignment but a severe artifact from entering the dataset.

The Edit-Ignition Adapter

The Edit‑Ignition Adapter injects edit instructions into a frozen generator while preserving temporal consistency.

In‑place editing forces every frame to stay tied to a fixed source clip, which leads to temporal drift when the video stream grows. The Edit‑Ignition Adapter solves this by adding a lightweight, trainable bridge that injects edit cues without disturbing the frozen diffusion transformer.

A tiny trainable block placed after each transformer layer that routes edit instructions into the latent stream while the backbone stays frozen.

HistCA: queries from the first 2 frames attend to the 3 history tokens, producing an update $\Delta_{\text{hist}}$ that is added to those 2 frames.

TempSA: the updated 2 frames plus the remaining 2 frames undergo causal self‑attention, yielding a forward‑propagated signal $\Delta_{\text{temp}}$ that respects the temporal order.

EditCA: every frame now queries the edit embedding $c_{\text{edit}}$, adding $\Delta_{\text{edit}}$ to all 4 frames.

Modulation: each update is scaled by $(1+s_{\sigma})$ and shifted by $b_{\sigma}$ (e.g., $s_{\sigma}=0.1$, $b_{\sigma}=0.05$) before being added back, leaving the final hidden state $H'$.

The three stages keep the chunk anchored to past content, propagate that anchor forward, and finally let the edit instruction affect the whole chunk without ever altering the frozen backbone.

How does the Edit‑Ignition Adapter differ from naïve fine‑tuning of the whole generator?

Fine‑tuning updates every weight in the diffusion transformer, risking loss of the pretrained generation quality and incurring huge compute. The adapter, by contrast, freezes the backbone and only learns a few lightweight attention blocks; its zero‑initialized projections start as identity maps, so the original prior is preserved and editing is learned as a residual update.

**Figure 2.** Our adapter's architecture. It is composed of three attention modules.

Training the Adapter

Training the adapter via flow-matching while shaping noise and history for robust editing.

In‑place editing forces the model to stay aligned with a fixed source clip, which quickly leads to temporal drift when the video stream grows. The training recipe below equips the lightweight edit‑ignition adapter with a loss and data schedule that keep it stable over arbitrarily long sequences.

Instead of asking the adapter to directly reconstruct a clean frame, we ask it to predict the velocity that would move a noisy interpolated state toward the target, turning the problem into a regression of a motion field.

How does flow‑matching differ from the standard diffusion denoising loss?

Standard diffusion loss asks the model to predict the original clean sample from a noisy one, i.e. a denoising direction. Flow‑matching instead asks for the velocity that would move the interpolated state toward the clean target, which directly encodes the direction of change and yields a regression that is linear in the noise level, making it easier for a lightweight adapter to learn the edit dynamics.

During training we deliberately corrupt the history latent so the adapter learns to operate on imperfect past frames, mirroring the errors it will encounter at inference time.

Frame 1: with probability 0.5 we corrupt; assume corruption occurs, so $\tilde{x}_1 = 0.3\,\epsilon_1 + 0.7\,x_1$.

Frame 2: corruption fails (probability 0.5), so $\tilde{x}_2 = x_2$ remains clean.

Frame 3: corruption occurs, yielding $\tilde{x}_3 = 0.35\,\epsilon_3 + 0.65\,x_3$.

The adapter receives the corrupted short history $(\tilde{x}_1,\tilde{x}_2,\tilde{x}_3)$ and must still predict the correct edit velocity.

Even a modest corruption probability forces the adapter to rely on robust cues rather than assuming perfect past frames, which dramatically reduces exposure bias at inference.

We replace the backbone’s multi‑stage, resolution‑dependent sigma schedule with a single‑stage mixture of Gaussians, giving the adapter a consistent representation space while still covering the full range of edit‑relevant noise levels.

Why not keep the original pyramid sigma schedule used by the backbone?

The pyramid schedule ties high noise to low‑resolution stages and low noise to high‑resolution stages. For editing, an edit may affect fine details without changing resolution, so coupling noise to scale forces the adapter to learn unrelated variations. The mixture‑Gaussian sampler decouples noise from resolution, letting the adapter focus on the edit itself.

Sample a component index $k$ according to $\pi$: suppose $k=2$ (probability 0.4).

Draw $\sigma$ from $\mathcal{N}(\mu_2=0.4,\delta_2^{2}=0.05^{2})$, e.g. $\sigma=0.38$.

Use $\sigma=0.38$ in the flow‑matching loss for this training step.

Repeating many times yields a distribution concentrated around the three target noise levels, with more samples near the middle and high noise levels.

By adjusting $\pi$, we can bias training toward the noise regimes that matter most for a given editing stage, without altering the backbone’s resolution schedule.

Training proceeds in two stages: first we expose the adapter uniformly to all noise levels, then we concentrate on low‑noise, detail‑rich steps and later frames that are hardest to preserve.

Phase 1, frame 1: sample $\sigma$ from the mixture (uniform), compute loss with weight $1$.

Phase 1, frame 5: same procedure, loss weight still $1$.

Phase 2, frame 1: sample $\sigma$ with biased $\pi$, loss weight $1.0$.

Phase 2, frame 5: same $\sigma$ sampling, loss weight $2.0$, emphasizing the later frame.

Overall, Phase 2 concentrates training effort on low‑noise samples and later frames, sharpening detail preservation.

Curriculum learning lets the adapter first acquire a generic edit skill set, then specialize where the editing problem is most challenging.

Together, the flow‑matching loss, history corruption, mixture‑Gaussian sigma sampler, and two‑phase curriculum give the edit‑ignition adapter a stable, focused training signal that scales to infinite‑length streaming edits.

Inference and Streaming Continuation

Inference runs the adapter once per edit, then lets the frozen backbone continue the stream.

Repeated edits on a streaming video quickly accumulate drift if every chunk is conditioned on the original source. The pipeline isolates the edit to a single ignition chunk, then relies on the backbone to propagate the change forward.

The adapter fires only on the first chunk after an edit instruction, injecting the edit into the latent before the backbone takes over.

Step 1: Adapter computes the Euler update, producing an edited latent for frames 1–2.

Step 2: The edited latent is inserted into the history window, which now contains the edited frames.

Step 3: The adapter is turned off; the next chunk will be generated by the backbone.

Only the first chunk incurs the adapter’s cost, yet the edit is retained because the edited frames remain in the sliding history.

The frozen backbone generates subsequent chunks by conditioning on the sliding history, automatically carrying forward the edit without further adapter involvement.

Step 1: History window contains edited frames from the ignition chunk.

Step 2: Backbone generates C₁ using the pyramid scheduler, sees edited frames in its history.

Step 3: History slides forward, now holds the last two frames of C₁.

Step 4: Backbone generates C₂, again conditioned on the edited history.

Step 5: History slides, retaining edited content for C₃.

The edit persists as long as edited frames stay within the fixed‑size history window, eliminating the need for repeated adapter calls.

Receive an edit instruction.

Generate the ignition chunk with the adapter using a single‑stage Euler schedule at near‑zero $\sigma$.

Insert the ignition chunk into the history window and reset the anchor frame $x_0$ to its first edited frame.

For each subsequent chunk, run the frozen backbone with its pyramid scheduler while keeping the adapter disabled.

Slide the history window forward, discarding the oldest frames but retaining edited content.

Maintain the anchor $x_0$ fixed until the next edit instruction arrives.

**Figure 3.** Inference pipeline of our method. Each edit round starts with an ignition chunk, after which subsequent chunks continue the stream.

Quantitative Results

Quantitative results show our edit‑ignition adapter outperforms baselines across metrics.

Recall that in‑place editing forces the model to stay aligned with a fixed source clip, which leads to drift in long streams. Our edit‑ignition adapter decouples the initial edit from the streaming generator, enabling stable infinite‑length editing.

Our method improves Camera Motion by +0.22 over the next‑best baseline.

Achieves 0.7654 versus 0.5494 for the strongest pure‑backbone competitor.

The provided image contains a table comparing various methods across four metrics: Camera Motion, Motion Smoothness, Temporal Flickering, and Dynamic Degree. The methods are categorized into "Pure Backbone," "In-Place Editing," "Prompt Switching," and "Ours."

**Table 2.** VLM-as-Judge evaluation results.

**Table 3.** Edit faithfulness across sequential editing rounds. Our method maintains stable performance ($\Delta < 0.06$ between rounds), demonstrating robust sequential editing capability without degradation.

**Figure 4** The aesthetic quality score recorded with different editing round.

Qualitative Benchmarks

Visual comparison of editing methods on streaming video tasks.

We compare six video‑editing methods on two benchmark scenarios, each receiving identical source footage and three successive edit instructions.

**Figure 5.** Qualitative comparison with various methods on the streaming video editing task, where a sequence of editing instructions (here camera-movement and style edits) is applied continuously to a scenery source video. All methods share the same source video and editing instructions. Methods marked with † take no source video as input, while all other conditions are kept identical for fairness.

**Figure 6.** Qualitative comparison with various methods on the streaming video editing task, where a sequence of editing instructions (here entity-transformation edits) is applied continuously to a human-centric source video. All methods share the same source video and editing instructions. Methods marked with † take no source video as input, while all other conditions are kept identical for fairness.

All competing baselines either fail to realize the requested edit, drift away from the original subject, or lose temporal consistency across edits, whereas our approach uniquely preserves edit fidelity, source content, and a continuous stream.

**Figure 7** Long-video streaming editing. We lengthen each editing segment so that the full sequence exceeds 1000 frames. Our method stays visually stable and keeps realizing each instruction. The edited attributes also persist across segments.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers