Editing Images into Reusable Layers via Compositional Edit‑Prior Distillation

Wen Xue, Yong Liu, Xueheng Li, Si Wu, Yong Xu, Yipeng Sun, Yan Li, Simiu Gu

LayerEditor jointly performs instruction-driven image editing and RGBA layer decomposition using on-policy distillation.

How can we modify an image while simultaneously decomposing it into reusable, editable RGBA layers?

Existing image editing models produce flattened outputs, forcing users to perform separate, error-prone decomposition steps to isolate objects or text for reuse. LayerEditor introduces a unified framework that generates an edited canvas and an ordered stack of RGBA layers simultaneously, using a shared token sequence and a dedicated Alpha decoder to manage transparency. On the Crello-Testset, this approach reduces composite FID by 38.8% compared to joint-generation baselines while maintaining high editing fidelity.

Paper Primer

The core mechanism, In-Context RGB–Alpha Token Modeling, represents the edited canvas and the layer stack as a single sequence of tokens. To prevent the model from losing its original editing capability during this adaptation, the authors use Compositional Edit-Prior Distillation (C-EPD): a teacher-student setup where a frozen, pretrained editor guides the student's layered output by supervising both the direct canvas and the recomposed layer stack.

LayerEditor improves layered editing fidelity and recomposition quality over existing joint-generation methods.

Composite SSIM scores on the Crello-Testset and ScaleEdit-Testset benchmarks. A 5.7% increase in Composite SSIM and a 38.8% reduction in Composite FID compared to the Seedream5-Pro joint-generation baseline.

The model effectively transfers editing priors to layered outputs without degrading visual quality.

Ablation studies comparing the full model against variants without C-EPD. Removing C-EPD drops edit consistency scores from 6.14 to 4.67 and layer segmentation scores from 6.83 to 5.01.

Why is a unified model necessary instead of just running an editor followed by a decomposition tool?

Cascaded workflows often fail to coordinate the edit with the target layer structure, leading to incomplete layers or artifacts that require manual refinement. A unified model optimizes both objectives simultaneously, ensuring the generated layers are semantically meaningful and reusable.

What is the scope of the layer-count control?

The model supports variable layer counts (2–10 layers) and user-specified content for individual layers, allowing the same edited image to be reorganized into different granularities based on the user's downstream needs.

Introduction: The Need for Layered Editing

LayerEditor unifies editing and layer generation, enabling controllable RGBA outputs.

Existing image‑editing models emit a single flattened pixel canvas, which makes it hard to edit fine‑grained regions or to reuse parts of the result for later edits.

Separately applying a decomposition step after editing often yields poorly organized layers, forcing users to run the model multiple times to correct layer‑level errors.

LayerEditor generates both the edited image and an ordered stack of reusable RGBA layers in a single forward pass, letting users tweak individual layers afterward.

How does LayerEditor differ from a standard editing model followed by a separate decomposition step?

Standard models treat editing and layer recovery as independent stages, so the edit is optimized without regard to layer structure. LayerEditor optimizes them jointly: the token sequence encodes both the final canvas and the layer stack, and C‑EPD ensures the pretrained edit quality is retained while the layers are learned simultaneously.

To train and evaluate this capability we built LayerEditor‑100K, a large‑scale dataset containing source images, edit instructions, target results, and ordered RGBA layer annotations.

We also introduce Crello‑Testset and ScaleEdit‑Testset, benchmarks that measure editing fidelity, layer quality, and controllable layer manipulation such as layer‑count control and user‑specified content.

**Figure 1.** LayerEditor edits images into reusable RGBA layers with user-specified layer content. The predicted layers can be independently manipulated for subsequent edits.

The shift from pixel‑based editing to layer‑based editing unlocks precise local control and reusable components for downstream design tasks.

Related Work

We survey prior conditional editing, layered generation, and distillation approaches.

InstructPix2Pix introduced paired image–instruction supervision for conditional editing, and later works such as Qwen-Image and FireRed-Image-Edit improved fidelity and instruction adherence with stronger generative priors. Subsequent studies explored selective reconstruction, interactive control, and agentic pipelines (Visual ChatGPT, PhotoAgent, Agent Banana), yet they either output flattened images or rely on cascaded workflows to retrieve editable layers.

Layered visual modeling has produced a variety of generation and decomposition methods, including LayerDiff, PSDiffusion, LayeringDiff, LASAGNA, COLE, CreatiPoster, Editable Visual Design, MiLDEdit, MRT, LaDe, and UniWorld‑Design for coordinated layer creation. Decomposition approaches such as Qwen-Image-Layered, LiWi, and earlier transparency‑aware techniques (Rhemann et al., 2009; Zhang & Agrawala, 2024; Yu et al., 2026) recover editable components after rendering, but they are typically optimized separately from instruction‑driven editing.

On‑policy distillation (OPD) aligns a student model with a teacher by supervising on states actually visited by the student, reducing distribution mismatch. Recent extensions such as Flow‑OPD, DiffusionOPD, and STEP‑OPD apply this principle to flow‑based and diffusion generative models, while progressive distillation and consistency models focus on reducing sampling steps rather than preserving layered editing capabilities.

LayerEditor Architecture and C-EPD

Jointly generate an edited canvas and reusable RGBA layers via a new distillation trick.

Standard image‑editing models emit a single flattened pixel canvas, which makes localized edits impossible; the model must instead produce a reusable stack of RGBA layers to enable per‑object manipulation.

An RGBA Stack is an ordered list of layer pairs ($R_i$, $A_i$) where each $R_i$ holds RGB colors and each $A_i$ holds opacity, and the stack is composited back‑to‑front to form the final image.

Why not just predict a single RGBA image instead of a stack?

Predicting a single RGBA image collapses all objects into one layer, losing the ability to edit or replace individual elements later. The stack preserves per‑object semantics, enabling downstream compositional edits.

The model treats the canvas and every layer as a sequence of tokens, each token carrying a field identifier (canvas, RGB, or Alpha) and a spatial coordinate, so a single transformer can attend across all fields jointly.

How does this token scheme differ from naïvely concatenating separate RGB and Alpha streams?

Naïve concatenation would treat each field as an independent sequence, losing the ability for the transformer to jointly reason about cross‑field interactions. By embedding the field identifier into the rotary coordinate, the model can attend across canvas, RGB, and Alpha tokens in a single pass, enabling coherent compositional reasoning.

LayerEditor extends a pretrained multimodal diffusion transformer (MMDiT) to output both a direct edited canvas D and an ordered RGBA stack S in one forward pass, then uses on‑policy distillation (C‑EPD) to keep the original editing behavior.

Why not train a separate model for layers after the base model is fine‑tuned?

Training a separate model would discard the pretrained editing prior, leading to the text corruption and structural distortions observed in ablations. Joint generation plus on‑policy distillation preserves the original prior while adding layer capability.

C‑EPD uses the frozen pretrained editor as a teacher that supplies on‑policy targets for both the direct canvas prediction and the composited image reconstructed from the predicted layers, ensuring the layered model retains the original editing behavior.

How does C‑EPD differ from naïve layer‑wise supervision?

Naïve supervision would train each layer independently, ignoring the interaction that produces the final composited image. C‑EPD instead supervises the whole composition against the teacher’s canvas, ensuring that the layers jointly reproduce the intended edit.

Layer 0 RGB $R_0$ is set to a red patch: all pixels = $(1,0,0)$. Alpha $A_0$ is $0.6$ everywhere.

Layer 1 RGB $R_1$ is a blue patch: all pixels = $(0,0,1)$. Alpha $A_1$ is $0.4$ everywhere.

Composite starts from white background $b_{C_{-1}} = (1,1,1)$. After layer 0: $b_{C_0}=0.6\cdot(1,0,0)+(0.4)\cdot(1,1,1)=(0.8,0.4,0.4)$.

After layer 1: $b_{C_1}=0.4\cdot(0,0,1)+(0.6)\cdot(0.8,0.4,0.4)=(0.48,0.24,0.64)$.

The final composited image $C$ thus blends red and blue with the background, demonstrating how the Alpha masks control contribution.

The example shows that even with overlapping colors, the per‑pixel contribution is a weighted blend determined solely by the Alpha values; changing an Alpha mask directly changes the visual influence of its layer without altering the RGB content.

LayerEditor‑100K provides 100 K editing pairs with full RGBA stacks, covering 48 329 text edits, 36 858 object edits, and 14 813 poster‑level appearance edits, enabling thorough evaluation of both instruction fidelity and layer isolation.

**Figure 2.** Overview of LayerEditor. LayerEditor jointly generates the edited canvas $D$ and ordered RGBA stack $S$. C-EPD uses the frozen teacher to supervise both the Direct prediction and the decoded composition reconstructed from the predicted layers.

Training Data and Benchmarks

Describes how the model is trained and the layered benchmarks used for evaluation.

Layered editing models need supervision not only for the final image but also for the intermediate RGBA layers. Existing image‑editing benchmarks provide only flattened outputs, making it impossible to measure layer controllability. This section introduces the data and training pipeline that enable supervision of both canvas and layers.

**Figure 1.** Overview of the data construction pipeline. The pipeline consists of two main routes: (A) Route A, which processes native layered designs, and (B) Route B, which generates flattened editing pairs. Both routes converge at (C) Unified checks and validation to produce the final LayerEditor-100K dataset.

**Figure 3.** Overview of LayerEditor-100K. Paired editing examples and ordered RGBA annotations cover text, object, and poster-level appearance edits with 2–10 layers, providing supervision across diverse editing tasks and layer structures.

Two disjoint benchmark suites evaluate how well a model can generate and manipulate reusable RGBA layers: Crello‑Testset uses native layered designs, while ScaleEdit‑Testset starts from flattened edits and recovers the underlying layers.

Layer 1 (Background) remains unchanged – pixel values identical to the source.

Layer 2 (Text) renders the word “SALE” at coordinates (x=30, y=20) with opacity 0.9.

Layer 3 (Object) places a red star at (x=70, y=50) with opacity 1.0.

Compositing the three layers with standard “over” alpha blending yields the target image.

Metric computation: SSIM between target and composited image = 0.99, RGBA L1 = 0.04, Alpha IoU = 0.92.

This concrete example shows how the benchmark quantifies both visual fidelity (SSIM) and the correctness of individual layer masks (Alpha IoU), which are invisible in flattened‑only evaluations.

How do these benchmarks differ from standard image‑editing benchmarks?

Standard benchmarks score only the final rendered image (e.g., using PSNR or FID). Crello/ScaleEdit benchmarks additionally require the model to output a reusable RGBA stack and evaluate each layer separately (RGBA L1, Alpha IoU), thus measuring controllability and compositional correctness.

Sample a source image $I_{\text{src}}$ and an edit instruction $u$.

Generate a canvas $D$ and an ordered RGBA stack $S$ using the model $G$ (Equation [F1]).

Render $D$ from $S$ via standard “over” compositing to obtain a predicted edited image.

Compute a reconstruction loss between the predicted image and the teacher model’s target.

Apply the C‑EPD loss, which compares the reconstructed composition against the teacher’s layered output.

Back‑propagate the combined loss and update model parameters with Adam.

Experimental Results

LayerEditor outperforms baselines on fidelity, layer quality, and controllability across benchmarks.

LayerEditor jointly generates an edited canvas and a reusable RGBA stack, enabling precise post‑edit revisions.

LayerEditor attains the highest Composite SSIM and Alpha IoU on both Crello‑Testset and ScaleEdit‑Testset.

Table 1 shows a 5.7 % SSIM gain over Seedream5‑Pro (Joint) and a 38.8 % FID reduction.

**Figure 11.** Rubric score examples. For the same editing request, progressively improved outputs receive higher scores across editing fidelity, layer separation, composition, and Alpha quality. The examples illustrate the rubric's discriminative behavior rather than agreement with human ratings.

**Figure 12.** Instruction-following editing and controllable layer decomposition on Crello-Testset. Prompts specify the edit and layer count. Large output images show recompositions, and checkerboard panels show individual RGBA layers.

**Figure 13.** Instruction-following editing and content-controlled layer decomposition on ScaleEdit-Testset. Prompts specify the edit, layer count, and desired content separation. Large output images show recompositions, and checkerboard panels show individual RGBA layers.

**Figure 4.** Qualitative comparison of layered editing. All method outputs are rendered from the predicted RGBA stacks rather than the independently generated canvases. LayerEditor follows the editing instructions while producing coherent recompositions and reusable layers.

**Figure 5.** Prompt-conditioned layer-content control. Both examples request six RGBA layers with specified layer content. LayerEditor performs the requested edits while assigning text rows and birds to the corresponding reusable layers.

**Figure 6.** Image-to-layer decomposition on Crello test set.

**Figure 7.** Effect of C-EPD on recomposed outputs.

Table 1 aggregates quantitative comparisons; LayerEditor leads on Composite SSIM/FID and Alpha IoU across both benchmarks, outperforming the nearest baseline by up to 5.7 % in SSIM.

Removing C‑EPD reduces edit consistency from 6.14 to 4.67, a 24 % decline.

Table 3 ablation study on Crello‑Testset.

Table 3 confirms that each C‑EPD sub‑component improves edit consistency and reduces RGBA L1, with the full model achieving the best overall scores.

Ablation Studies and Limitations

Ablation studies quantify how opacity training and instruction control affect LayerEditor’s layered editing performance.

Opacity reconstruction ablations reveal that training the opacity channel improves boundary fidelity but does not isolate the decoder architecture from the added supervision. No off‑policy query control is provided, so the effect of on‑policy selection remains entangled with teacher supervision.

Rubric‑based evaluation (Section C.3) scores edit consistency, visual quality, layer segmentation, layer composition, and Alpha accuracy on a 1–10 scale anchored at 1, 3, 5, 7, 10. The Direct canvas is excluded; all metrics are computed on the recomputed composite $C$.

Instruction controllability (Section D.1) tests whether LayerEditor adapts both the edited appearance and the generated RGBA stack when the instruction changes. Table 6 shows LayerEditor achieving the highest Direct/Composite SSIM and lowest RGBA L1 among eight methods.

Implementation Details

Implementation specifics, training setup, and Alpha decoder details.

LayerEditor builds on the FireRed-Image-Edit-1.1 codebase, adding a rank‑256 LoRA adapter to the DiT backbone while keeping the image and text encoders and the pretrained RGB decoder frozen.

Training begins with two epochs of flow‑matching, then proceeds with joint flow‑matching and C‑EPD optimization; at inference a single denoising trajectory conditioned on $(I_{\text{src}}, u, N)$ yields the edited canvas and ordered RGBA fields without any teacher calls.

**Table.** Training and inference hyperparameters.

The Alpha decoder follows a VAE architecture with a single‑channel Alpha‑logit head initialized at zero weights and bias $-3.5$, producing opacity maps via a sigmoid after decoding latent $z_A$.

**Table 5.** Alpha-decoder configuration. Values separated by slashes follow stage order.

During rendering, layers are composited back‑to‑front using the formula $C_i = A_i \odot R_i + (1 - A_i) \odot C_{i-1}$, starting from a white background canvas $B$.

Training Configuration

Training loop details for C‑EPD, loss schedule, and local decoding.

The on‑policy rollout samples ten nodes from the thirty‑step inference sigma grid, taking two nodes from each of five temporal segments. For every active C‑EPD update we draw two queries from high‑, medium‑, and low‑noise bands with probabilities 0.2, 0.5, 0.3 respectively, and detach the rollout states.

At each selected noisy state the student recomputes velocities for the Direct, RGB, and Alpha fields with gradients, while the frozen teacher predicts its image‑level velocity from the same canvas state, source image, instruction, and noise level (no gradients). The student’s local clean RGB–Alpha estimates are decoded and composited on a white background; the teacher’s Direct prediction provides a detached target for the composite pixel loss (Eq 4).

The first two epochs use flow‑matching only. After the warm‑up phase (two epochs) a local counter k increments on every active update; every five updates C‑EPD is enabled and weighted by $\lambda_{C}$(k)=0.1 min(k/500, 1). The composite pixel term receives a fixed weight $\lambda_{S}$=0.05. On active updates the weighted C‑EPD loss is added to the flow‑matching loss (Eq 6); otherwise training proceeds with flow‑matching alone.

Initialize LoRA parameters $\theta$, frozen editor $\theta$₀, and frozen Alpha decoder $\phi$; set local counter k←0 after the warm‑up epochs.

For each training sample (`I_src`, u, N, Z*) drawn from `D_train`:

Iterate over epochs e=1…E:

Sample a diffusion time t and noise $\epsilon$∼𝒩(0, I); construct the noisy latent $Z_t$=(1−$\sigma_{t}$)Z*+$\sigma_{t}$ $\epsilon$ and the target velocity v* = $\epsilon$−Z*.

Predict velocities {$v_X$}_{X∈$\Omega$} from the joint noisy state and (`I_src`, u, N); compute the joint flow‑matching loss (Eq 5) and set L_$\theta$←`L_flow`.

If e > `E_warm`, increment k; if k mod 5 = 0, perform a C‑EPD update:

Sample two queries from the detached student rollout, recompute student velocities with gradients, and query the frozen teacher on the corresponding Direct state.

Decode RGB and Alpha from the student’s latent estimates, compose the clean stack on white (Eq 3), and decode the teacher’s Direct canvas $I_T$.

Average the two query losses (Eq 4) to obtain $L_{C-EPD}$ and update L_$\theta$←`L_flow`+$\lambda_{C}$(k) $L_{C-EPD}$.

After all epochs, return the trained LoRA parameters $\theta$ and Alpha decoder $\phi$.

Local decoding (A.4) operates on a selected denoising state z_$\sigma$=(1−$\sigma$)z₀+$\sigma$ $\epsilon$, where the clean latent z₀ is recovered as z_$\sigma$−$\sigma$ v*. The student’s latent estimates for each RGB and Alpha layer are decoded by the frozen decoders, yielding continuous RGB values in [0,1] and Alpha opacities via a sigmoid. Composition follows the source‑over formula: the visible contribution of layer i is weighted by the product of (1−$\alpha_{j}$) for all higher layers j, and the uncovered background contributes the remaining opacity.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers