Editable Visual Design
Junyan Ye, Wei Liu, Dongzhi Jiang, Zichen Wen, HaoDong Li, Zhutao Lv, Jiaxin Lin, Jinhua Yu, Jun He, Zilong Huang, Rui Chen, Weijia Li
A design paradigm using a VLM-driven agent to synthesize editable HTML/CSS layouts from visual-model-generated priors.
How can we generate visual designs that are fully editable (layer-wise) rather than flattened bitmaps?
Current visual generation models produce high-quality images but output flattened bitmaps that are impossible to edit, while code-based agents generate structured layouts that lack aesthetic intuition and struggle to render complex visual assets. Editable Visual Design resolves this by splitting the workflow: a Vision Language Model (VLM) acts as a "creative brain" to plan and code, while an image generation model serves as an on-demand "visual world simulator" to provide aesthetic priors and standalone assets. This closed-loop system delivers production-grade artifacts with decoupled layers and real text, allowing users to perform independent layout and asset adjustments.
Paper Primer
The system operates on an "imagine first, then act" loop. The agent first generates an "imagined visual" to establish composition and color priors, then independently generates isolated visual assets and writes native HTML/CSS, using a render-and-reflect mechanism to iteratively patch layout flaws.
The paradigm successfully bridges the gap between aesthetic quality and engineering editability.
Validation across posters, infographics, and marketing materials demonstrates that the system produces artifacts where text, background, and graphic layers are fully decoupled and independently editable. The system moves design from "black-box" raster output to a transparent, traceable process via Agent Design Replay.
Why use an image model as a "simulator" instead of just generating the final design directly?
Pure code generation lacks the two-dimensional spatial sense and aesthetic intuition required for complex visuals, while pure image generation lacks the structural logic needed for editable, text-accurate engineering deliverables.
What is the role of the "Agent Design Replay" in this workflow?
It serializes the agent's entire decision trajectory—from intent planning and asset generation to code reflection—providing transparency that allows users to trace design intent and intervene manually.
The Problem of Flattened Bitmaps
We expose why diffusion outputs hinder editing and propose code-driven Editable Visual Design.
Diffusion models such as GPT‑Image‑2 and Nano‑Banana output flattened bitmaps that entangle text and graphics, making layer‑wise post‑editing impossible. In contrast, code‑based visual generation via Coding Agents yields precise layout control but suffers from a missing global visual intuition and difficulty encoding complex assets.
A design artifact expressed as native HTML/CSS with separate visual assets, enabling post‑generation edits of layout, text, and graphics.
**Figure 1.** Editable design artifacts generated by our proposed Editable Visual Design.
The shift from pixel‑generation to code‑generation unlocks editable, layer‑aware designs without sacrificing visual quality.
The Coding-Agent Workflow
From a static bitmap to an editable canvas via a VLM‑guided coding agent.
Flat, flattened bitmaps lock the design after generation, preventing any post‑hoc edits. The workflow below keeps the design alive by interleaving visual imagination with native code generation.
Instead of producing a final image, a Vision‑Language Model (VLM) plans the design, a visual simulator imagines a reference picture, and a coding agent emits native HTML/CSS/SVG so every element stays directly editable.
How does this differ from the usual “generate‑image‑then‑code” pipeline?
Typical pipelines treat the image as the final product and only later attempt to extract code, which entangles pixels with layout. Our workflow keeps the visual reference separate and generates assets on demand, so the code never inherits pixel‑level artifacts and remains fully editable.
1️⃣ Understanding & Design Planning – the VLM parses the brief and settles on content, size, and visual register.
2️⃣ Visual Simulation – the VLM calls the image model for an imagined visual that guides later stages.
3️⃣ Structural Coding & Generation – the coding agent derives canvas topology, requests clean assets, and writes native HTML/CSS.
4️⃣ Verification & Visual Self‑Healing – a headless browser renders the page, a VLM reviewer checks alignment and readability, and targeted patches are applied.
5️⃣ Editable Design & Agent Design Replay – the final DOM is delivered with editable layers and a serialized replay of the entire decision process.
Step 1: VLM decides “green background, white title, centered layout”.
Step 2: Image model returns a mockup showing the green canvas with placeholder text.
Step 3: Coding agent requests a clean background asset ($\alpha$‑channel) → receives a 200×300 PNG with pure green.
Step 4: Coding agent requests the title asset → receives a 150×30 PNG with white text on transparent background.
Step 5: Agent writes HTML/CSS that places the background as a
Step 6: Verification renders the page, VLM confirms alignment, no patch needed.
The assets are generated independently, so the final HTML never inherits any of the mockup’s pixels; editing the title later only requires swapping the img source.
**Figure 2.** Overview of the Editable Visual Design workflow. A VLM plans the design, calls the image model for an imagined visual and for standalone assets, then writes native HTML/CSS/SVG and delivers an editable canvas. The band along the bottom is the Design Replay, the recorded trajectory from prompt to artifact.
**Figure B.** Design a 3:4 vertical New Chinese light-luxury tea-launch poster in dark green, off-white...
By decoupling visual imagination from native code generation, the workflow turns an immutable bitmap into a fully editable design artifact.
Case Studies and Performance
Editable Visual Design delivers high-quality visuals with full layer editability.
Agent Design Replay can produce highly structured designs with up to 120 editable layers.
Figure 5 shows a field‑guide case with 120 layers organized into 13 groups.
The agent records each planning, generation, and code‑adjustment step, exposing a full trace that can be inspected or edited later.
How does Agent Design Replay differ from a simple generate‑then‑edit pipeline?
In a simple pipeline the model emits a final image or code without exposing intermediate decisions; Agent Design Replay logs each phase, allowing selective revision and transparent reasoning about why a particular layout was chosen.
**Figure 5.** Agent Design Replay on two real cases. Each step carries a timestamp and its own output, across three phases: understand and plan, generate and build, and review and fix. Top: an information-dense field guide—120 editable layers in 13 groups—where the review catches and repairs real layout defects. Bottom: a visually driven travel poster—6 layers in 1 group—where the review passes with no changes. Layer structure follows the density of the brief.
**Figure 4.** From prompt to editable canvas. Three briefs in different visual registers—City Atlas, Summer Music, and Memphis Future—each shown as the prompt, the coded result, and the same artifact opened up. The right-hand panels carry the point: headline lettering, illustrations, badges, and background all stay independently addressable.
The discussion highlights that using a visual simulator first gives the VLM a concrete aesthetic prior, which then guides precise code construction, bridging the gap between generative creativity and deterministic editability.
Related Work and Limitations
We recap that diffusion yields flat bitmaps, while a coding‑agent pipeline enables editable designs.
Visual code generation has surged with large‑language models such as GPT‑5.6 Sol, Claude Fable 5, and Kimi K3, which can synthesize interactive interfaces from natural‑language prompts.
Benchmarks like Design2Code, WebSight, Web2Code, Interaction2Code, and DesignBench evaluate layout fidelity, while newer systems (LaTCoder, UICopilot, LayoutCoder, UI2CodeN) inject DOM hierarchy priors to improve structural correctness.
Vector‑oriented works such as StarVector, OmniSVG, InternSVG, AutoPresent, PPTAgent, ChartMimic, and ChartCoder demonstrate that code‑based SVG or presentation generation yields editable artifacts, yet they still focus on syntactic compliance rather than global aesthetic control.
Image‑generation pipelines built on diffusion (Latent Diffusion, BAGEL, Qwen‑Image, Emu3.5) and specialized models (Z‑Image, RealGen, GPT Image 1/2, Nano Banana 2, Seedream 5.0 Pro, SenseNova‑U1) excel at texture and composition, but their raster outputs lack layer‑wise editability.
Recent research (TextDiffuser, COLE, OpenCOLE, Graphist, PosterCraft) adds layout‑aware text rendering and hierarchical design decomposition, yet the final products remain pixel‑based images.
Agentic design generation has moved toward planning‑first paradigms (Visual ChatGPT, GenArtist, Idea2Img, RPG) and chain‑of‑thought approaches (GoT, T2I‑R1, Uni‑CoT, Mind‑Brush, SCOPE, GEMS, Qwen‑Image‑Agent) that interleave reasoning with execution.
GenClaw highlighted the limitation of end‑to‑end image agents and advocated a code‑driven intermediate canvas; VideoCoCo extends this idea to video by treating executable programs as controllable generation processes.
GPT‑Image‑2 is a vision‑language model that produces high‑resolution raster images from textual prompts while preserving detailed layout cues.
Codex on GPT‑5.6 is a large‑language model fine‑tuned to generate structured UI code (HTML/CSS/JS) from natural‑language specifications.
Questions & answers
What is the main contribution of Editable Visual Design?
Editable Visual Design introduces a closed-loop agentic workflow that splits design generation into two roles: a VLM that plans and writes native HTML/CSS code, and an image generation model that supplies aesthetic priors and standalone visual assets on demand, yielding production-grade artifacts with decoupled layers and real, editable text.
What problem does Editable Visual Design address?
The paper addresses the fundamental limitation that current visual generation models (e.g., GPT-Image-2, Nano-Banana) output flattened bitmaps that entangle text and graphics, making layer-wise post-editing impossible, while code-based agents lack the spatial and aesthetic intuition needed to render complex visual assets.
Why can't existing image generation models like GPT-Image-2 or Nano-Banana solve this problem on their own?
These diffusion-based models produce raster bitmaps that lock the design after generation, preventing any post-hoc layer-wise edits and making it impossible to independently adjust text, graphics, or layout elements.
Why can't pure code-based agents solve this problem on their own?
Pure code generation lacks the two-dimensional spatial sense and aesthetic intuition required for complex visuals, and struggles to encode complex visual assets, resulting in designs that are structurally correct but aesthetically poor.
How does the Editable Visual Design workflow operate at a high level?
The system follows an 'imagine first, then act' loop: the agent first generates an 'imagined visual' to establish composition and color priors, then independently generates isolated visual assets and writes native HTML/CSS, using a render-and-reflect mechanism to iteratively patch layout flaws.
What is the role of the image generation model in this system?
The image generation model acts as an on-demand 'visual world simulator' that provides aesthetic priors and generates standalone visual assets; it is kept separate from the code so the final HTML/CSS never inherits pixel-level artifacts and remains fully editable.
How does this workflow differ from a typical 'generate-image-then-code' pipeline?
Typical pipelines treat the generated image as the final product and only later attempt to extract code, which entangles pixels with layout; Editable Visual Design keeps the visual reference separate and generates assets on demand, so the code is always clean and fully editable.
What is Agent Design Replay and what purpose does it serve?
Agent Design Replay serializes the agent's entire decision trajectory—from intent planning and asset generation to code reflection—providing transparency that allows users to trace design intent and intervene manually at any phase, unlike a simple pipeline that emits a final output without exposing intermediate decisions.
What kinds of edits does the system enable that prior approaches do not?
The system delivers artifacts with decoupled layers and real text, allowing users to perform independent layout and asset adjustments—capabilities that are impossible with flattened bitmap outputs from standard image generation models.
What benchmarks and related evaluation frameworks are mentioned in the paper?
The paper mentions Design2Code, WebSight, Web2Code, Interaction2Code, and DesignBench as benchmarks that evaluate layout fidelity for visual code generation; however, the paper does not report specific quantitative results for Editable Visual Design on these benchmarks.
What prior and related systems does the paper compare against or situate itself relative to?
The paper references code-generation systems (LaTCoder, UICopilot, LayoutCoder, UI2CodeN), vector-oriented works (StarVector, OmniSVG, AutoPresent, PPTAgent, ChartMimic), image-generation pipelines (Latent Diffusion, BAGEL, Qwen-Image, Seedream 5.0 Pro), layout-aware text rendering works (TextDiffuser, COLE, OpenCOLE, Graphist, PosterCraft), and agentic planning systems (Visual ChatGPT, GenArtist, Idea2Img, GenClaw), noting that all still produce either pixel-based outputs or lack global aesthetic control.
What is the key distinction between Editable Visual Design and vector-oriented works like StarVector or OmniSVG?
While vector-oriented works demonstrate that code-based SVG or presentation generation yields editable artifacts, the paper notes they still focus on syntactic compliance rather than global aesthetic control, which Editable Visual Design addresses by incorporating an image model as an aesthetic prior.
What are the limitations of the system as acknowledged in the paper?
The paper does not explicitly enumerate a dedicated limitations section; it acknowledges that pure code generation lacks spatial sense and aesthetic intuition, and that pure image generation lacks structural logic, but does not detail failure cases, scalability constraints, or quantitative performance boundaries of the proposed system.
What output format does the system produce?
The system produces native HTML/CSS code as its output format, yielding production-grade design artifacts with decoupled layers and real, editable text rather than raster images.
What large language models are mentioned as relevant to visual code generation in the related work?
The paper mentions GPT-5.6 Sol, Claude Fable 5, and Kimi K3 as large language models capable of synthesizing interactive interfaces from natural-language prompts in the context of visual code generation.
What is the 'render-and-reflect' mechanism?
The render-and-reflect mechanism is a component of the workflow in which the agent renders its generated code and then iteratively inspects and patches layout flaws, enabling self-correction without human intervention.
Who are the authors of this paper, and where and when was it published?
The paper does not state the authors' names, the publication venue, or the exact publication date in the provided text; the arXiv identifier in the source URL is 2609.04034.
Key terms
- VLM (Vision Language Model)
- A multimodal AI model that processes both visual and textual inputs and is used here as the 'creative brain' to plan design intent and generate HTML/CSS code.
- flattened bitmap
- A raster image file in which all visual elements—text, graphics, and backgrounds—are merged into a single pixel grid, making individual layer editing impossible after generation.
- visual world simulator
- The paper's term for the image generation model's role: an on-demand tool that produces aesthetic references and standalone visual assets to guide the coding agent.
- Agent Design Replay
- A logging mechanism that records the agent's full decision trajectory—intent planning, asset generation, and code reflection—so users can inspect, trace, and manually intervene at any step.
- render-and-reflect
- An iterative self-correction loop in which the agent renders its generated HTML/CSS, inspects the visual output, and patches layout flaws before finalizing the design.
- imagine first, then act
- The system's core operating principle: generate an 'imagined visual' to establish aesthetic and compositional priors before writing any code or producing final assets.
- decoupled layers
- A design structure in which individual visual elements (text, images, backgrounds) exist as separate, independently editable components rather than being merged into a single flat image.
- HTML/CSS
- Standard web markup and styling languages used here as the structured, human-readable code format that encodes the design layout and enables post-generation editing.
- diffusion model
- A class of generative AI model that produces images by iteratively denoising random noise, used by systems like GPT-Image-2 and Nano-Banana to generate high-quality but non-editable raster images.
- coding agent
- An AI agent that generates structured code (e.g., HTML/CSS or SVG) to represent a visual design, offering layout control but lacking inherent aesthetic intuition.
- DOM hierarchy
- The tree-structured representation of an HTML document's elements and their parent-child relationships, used by some systems to improve structural correctness in generated web interfaces.
- aesthetic prior
- A pre-established sense of visual composition, color, and style—derived here from the image generation model—that guides the coding agent toward visually appealing layouts.
- Design2Code
- A benchmark used to evaluate how accurately a model can convert a visual design specification into functional code, measuring layout fidelity.
- SVG (Scalable Vector Graphics)
- An XML-based vector image format that encodes graphics as mathematical shapes rather than pixels, enabling resolution-independent and programmatically editable visuals.
- chain-of-thought
- A prompting or reasoning strategy in which an AI model generates intermediate reasoning steps before producing a final answer, used in agentic design systems to interleave planning with execution.