AesCode: Aesthetic Code Generation with Decoupled Cross-Modal Rewards
Xin Gao, Yifan Yang, Xin Li, Pan Wang, Yan Li, Muzhao Tian, Mingxi Cheng, Bei Liu, Kai Qiu, Qi Dai, Dong Chen, Yue Dong, Leandros Tassiulas, Rex Ying, Ji Li, Chong Luo
AesCode uses design-graph-based rewards to train code models to follow generated images as aesthetic priors.
How can we train code-generation models to produce aesthetically structured visual artifacts like infographics and dashboards by using reference images as design guidance?
Code models can generate structured, editable visual artifacts like slides and dashboards, but they lack the "visual eye" to plan spatial layouts and aesthetic styles effectively. AesCode addresses this by using a generated reference image as a conditional input and a source of aesthetic guidance, training the model to extract layout and style while strictly adhering to the prompt's content. The system represents the target as a design graph, enabling decoupled rewards: deterministic verifiers check code-level correctness, while a Visual Graph Rubric (VGR) uses a VLM to score specific visual properties. AesCode-32B matches frontier models like Claude Opus 4.8 in visual quality while significantly improving verifiable correctness, achieving an 85.89% overall score on the authors' infographic benchmark.
Paper Primer
The core mechanism hinges on the Design Graph, a structured intermediate representation that separates semantic content from visual attributes. By decomposing the canvas into nodes (elements) and edges (spatial/hierarchical relations), the model can attribute specific rewards to individual properties, preventing dense verification signals from drowning out sparse aesthetic feedback.
The training pipeline uses Group reward-Decoupled Normalization Policy Optimization (GDPO) to balance these channels. This approach normalizes each reward channel independently, ensuring that the model learns from both absolute rule-based checks and sample-specific visual rubrics without collapsing them into a single, noisy scalar.
AesCode significantly outperforms its base backbones in visual quality and verifiable correctness.
Comparison of AesCode-32B against the Qwen3-VL-32B backbone on the infographic benchmark. Overall score improved from 61.10% to 85.89%, with a 22.4 percentage point gain in visual quality.
Why is a design graph necessary instead of just using a standard image-to-code similarity metric?
Standard similarity metrics treat the reference image as absolute ground truth, which penalizes valid design variations and forces the model to replicate the generator's hallucinations. The design graph allows the model to flexibly extract layout and style while independently verifying prompt-required content.
What is the scope of this method — does it apply to all visual generation tasks?
The method is designed for structured visual artifacts like slides, posters, and dashboards where content is dense and verifiable. It relies on the ability to parse code and DOM elements into a graph, meaning it is less applicable to unstructured or purely artistic image generation.
AesCode demonstrates that image generators can function as scalable "aesthetic teachers" for structured generative models, allowing compact code models to achieve frontier-level visual design capabilities.
Introduction
Introducing AesCode to bridge text‑only code generation and visual layout needs.
Code models generate structured, editable artifacts but lack any view of the canvas, so spatial planning and aesthetic choices are often bland or mis‑aligned. Image generators produce striking layouts yet frequently misrender text and logical relations, making their raster outputs unsuitable for downstream editing. This mismatch creates a gap: text‑only code generation cannot satisfy visual layout requirements.
AesCode generates code conditioned on a reference image and leverages a design graph with decoupled reward channels to enforce layout and aesthetic constraints that pure text prompts miss.
**Figure 1.** Information-rich infographics generated as code. Prompts specify content, while image-generator references offer visual design guidance. The examples show a landscape dashboard and portrait poster rendered from generated HTML/CSS.
The core gap is that text‑only code generation lacks visual layout cues, which AesCode addresses by conditioning on reference images and using graph‑structured rewards.
The AesCode Framework
AesCode optimizes visual code generation by decomposing design requirements into graph-structured supervision and multi-objective rewards.
Visual design is inherently under-constrained: a single prompt can yield many valid layouts. AesCode resolves this by decomposing design requirements into a structured graph, allowing the model to learn from reference images without being forced to replicate their specific, potentially non-authoritative, layout choices.
Instead of treating a reference image as a rigid template, we parse it into a hierarchical graph of elements and relations. This allows the model to satisfy prompt-required content independently of the reference image's specific stylistic or structural cues.
Parse $I_{\text{ref}}$ into nodes: root, header, logo, nav, and three feature cards.
Assign edges: header contains logo/nav; root contains header and feature-group.
Collapse the three feature cards into one group node with a count attribute of 3.
The graph now enforces the "three-column" requirement via the group node, while the blue theme is treated as a non-authoritative attribute in $X_V$.
By collapsing interchangeable siblings into a group node, the model learns the structural constraint (three columns) without being penalized for minor deviations in the reference's specific spacing or color.
**Figure 2.** Overview of AesCode. Prompt requirements and reference design cues yield graph-based supervision. Conditioned on $p$ and $I_{ref}$, the policy generates HTML/CSS code $c$, rendered as $\hat{I}$.
Graph-structured supervision turns open-ended visual judgment into a checklist of explicit criteria. By binding rewards to specific nodes and relations in the design graph, we provide the model with granular feedback on where it succeeds or fails.
To optimize these rewards, AesCode employs GDPO, which normalizes reward channels to prevent dominant signals from obscuring others. This ensures that valid objectives—from text correctness to whitespace distribution—contribute proportionally to the policy update.
Benchmark Results
AesCode dramatically raises overall scores, surpassing prompt-only baselines on structural and aesthetic metrics.
Training improves AesCode overall scores by over 30 % for the 8B model and 25 % for the 32B model.
Overall rises from 51.93 % to 82.94 % (8B) and from 61.10 % to 85.89 % (32B) on the Infographic Benchmark.
A 300‑prompt suite where each prompt is paired with a generated reference image, evaluating both programmatic correctness (Rule) and visual quality (Visual).
How does the Infographic Benchmark differ from typical code‑generation benchmarks?
Typical benchmarks measure only syntactic correctness of generated code. The Infographic Benchmark adds a visual dimension: models must also produce layouts that match a reference image, and scores are split into Rule (code) and Visual (design) components.
**Table 1.** Results on our infographic benchmark of 300 instances. Models in the first two blocks receive only the prompt; those in the third, including AesCode, also receive a generated reference image. Rule averages the programmatic Text, Bound, and Chart checks; Visual averages the Content, Layout, and Style scores as defined in Section 3.4. Overall is the mean of Rule and Visual. Scores (%) are averaged over three generations per prompt; higher is better. Bold and underlined values indicate the best and second-best scores in each column.
**Figure 3.** Qualitative comparisons selected by the three-rollout mean Overall gap. Each row shows the reference and outputs from GPT-5.5 and AesCode-32B. Bars report evaluation scores; red dashed boxes mark visible GPT-5.5 errors.
AesCode outperforms prompt‑only baselines on structural and aesthetic metrics.
Component Ablations
Ablation experiments quantify how each component of AesCode contributes to visual code generation performance.
Removing the reference input only at inference reduces AesCode‑8B’s Visual score by 1.00 point, far less than the 19.55‑point drop for its backbone and the 10.04‑point drop for GPT‑5.5.
Table 3 shows Visual scores: AesCode‑8B (reference removed) vs. backbone vs. GPT‑5.5 under the same inference‑only removal condition.
When the reference input is removed both during training and inference, AesCode‑8B still attains an overall score of 79.36, demonstrating that reference‑based rewards preserve performance even without reference conditioning.
Table 3, second section, reports the absolute scores for AesCode‑8B with reference omitted during training and inference.
Across component ablations, the full AesCode‑8B model achieves the highest overall score of 82.94, confirming that each listed component positively contributes to performance.
Table 4 lists the overall scores for various component configurations, with the full model reaching the top value.
Training and Optimization Analysis
Introducing GDPO, the key trick that aligns visual design with code generation.
The paper’s premise is that conditioning on reference images and using graph‑structured rewards lets AesCode generate verifiable, aesthetically sound visual code. This section details the core GDPO mechanism that makes that possible.
Table 4 shows that reinforcement learning drives most gains, while supervised fine‑tuning provides a stronger start; under the same RL budget overall quality rises from 77.97 to 82.94. GDPO further improves visual scores by normalizing reward channels independently, which is crucial when verifiable and perceptual objectives learn at different rates.
GDPO treats each reward channel separately, scaling its contribution so fast‑learning perceptual signals don’t drown slower‑learning correctness signals.
Mean execution = 10, so normalized execution = 10 / 10 = 1.
Mean layout = 2, so normalized layout = 2 / 2 = 1.
Mean aesthetic = 8, so normalized aesthetic = 8 / 8 = 1.
Apply channel weights w = (0.5, 0.3, 0.2): final reward = 0.5·1 + 0.3·1 + 0.2·1 = 1.0.
Even though raw scores differ widely, GDPO’s per‑channel normalization equalizes their influence, preventing any single signal from dominating the policy update.
How does GDPO differ from the scalar GRPO baseline that also aggregates rewards?
Scalar GRPO applies a single global weight to the sum of raw rewards, so a large‑magnitude channel can swamp the others. GDPO normalizes each channel independently before weighting, ensuring that fast‑learning perceptual signals do not drown slower‑learning correctness signals.
GDPO adds 5.12 Visual points over scalar GRPO.
Table 4 reports a Visual score increase from 70.68 (scalar GRPO) to 75.80 (GDPO) on the benchmark.
Human evaluation confirms the automatic metrics: VGR predictions agree with human judgments on 89.3 % of predicates, and the Visual checklist aligns on 93.3 %. AesCode‑32B attains higher content satisfaction than GPT‑5.5 (86.50 % vs 83.47 %) while staying within the ±0.5 aesthetic margin.
Failure analysis reveals three recurring issues: (1) canvas overflow, where generated code cannot anticipate final pixel geometry; (2) relational‑structure loss, because sparse relation‑level rewards fail to localize missing edges; (3) semantic‑element degradation, where icons collapse to generic shapes due to insufficient fine‑grained SVG guidance.
**Figure 5.** Residual failures of AesCode-32B. Reference and output zoom-ins show (a) canvas overflow, (b) lost relational structure, and (c) semantic element degradation. Dashed boxes mark affected regions; frequencies and additional cases appear in Section F.
In summary, representing targets as design graphs and conditioning generation on reference images yields a model that matches frontier systems in human‑aligned visual quality while surpassing them in verifiable correctness. Remaining challenges include handling pixel‑level geometry, complex relational structures, and fine‑grained semantic elements, motivating future work on visually grounded iterative refinement.
Related Work
Extended related work, data construction, implementation details, and reward system for AesCode.
Recent work such as COLE, PosterVerse, DesignAsCode, and Editable Visual Design steers code generation with images at inference time, but the code model itself never sees the visual reference during training. AesCode differs by training the code model to follow a prompt‑generated reference, treating the reference as both an aesthetic prior and a training signal.
Reward design for visual generation traditionally scores rollouts against an exact target image, which fails when the reference is generated and may not match the prompt. The paper instead uses VGR to evaluate layout and style against a design graph, and GDPO to normalize heterogeneous reward channels.
Multi‑objective reinforcement learning has moved from PPO to group‑normalized methods like GRPO. Because our seven reward channels fire at different rates, we adopt MO‑GRPO to reweight by variance, and ultimately GDPO to normalize each channel separately before aggregation.
Section B details the construction of the source designs, stratified sampling, prompt inference, and benchmark splits used throughout the experiments.
B.1 explains that human‑designed graphics are clustered via CLIP features and sampled uniformly across distance intervals to ensure coverage of both central and peripheral designs. The three splits (SFT, RL, benchmark) are disjoint by construction.
B.2 describes how GPT‑5.4‑mini infers a content‑only prompt and verifiable requirements from each image, then GPT‑5.5 generates three HTML candidates per prompt, keeping the highest‑scoring one when its reward exceeds 0.85.
B.3 details that for RL and benchmark splits, GPT‑Image‑2 creates a reference image from each prompt while the original human design is never shown to the policy, preserving the deployment scenario.
B.4 summarizes Table B1, which lists item counts per split, CLIP cluster distribution, and per‑item requirement statistics (median 12, range 3–23).
B.5 notes two model calls: one to produce the prompt + requirements, and a second to generate the reference image, with strict rules preventing mismatched text requirements.
C.1 provides training configuration for both 8B and 32B backbones, including learning rates, optimizer settings, rollout counts, and token limits, and describes the sandboxed rendering pipeline.
**Figure.** (a) source design, (b) demonstration target, excerpted, (c) rendered demonstration.
C.2 reports that under GDPO, execution rewards saturate early, while text and chart channels improve rapidly; design channels rise more slowly, reflecting the difficulty of learning spatial organization.
**Figure B2** Six benchmark items, each with its GPT-Image-2 reference, the opening of its generation prompt, and its first few verifiable requirements.
**Table B1.** Composition of the three splits. Both backbones use the same RL file.
C.3 shows that a reconstruction‑only reward improves CLIP‑similarity scores but harms discrete verifiable capabilities such as chart generation, confirming the need for decoupled reward channels.
C.4 offers random AesCode‑32B outputs across dashboards, slides, and posters in multiple languages, demonstrating the model’s ability to generate diverse visual code without format‑specific tuning.
D.1 explains that each rollout records an applicability mask, a raw score, and a diagnostic payload per reward channel, allowing selective aggregation and detailed failure analysis.
D.2 defines verifiable requirements as three types—`text_exact`, `chart_data`, and `table_data`—extracted from prompts; these are used by the reward system and do not depend on candidate outputs.
Reward Channel Definitions
Defines each deterministic reward channel and how they combine into the overall reward.
Reward channels quantify how well a generated page matches its reference across multiple deterministic criteria. Each channel yields a score that is later merged into a single scalar reward used to train the policy.
Each deterministic verifier produces a scalar, and the policy is guided by a weighted sum of those scalars.
Compute the weighted contribution of Execution: $0.5 \times 0.9 = 0.45$.
Compute the weighted contribution of Text: $0.5 \times 0.6 = 0.30$.
Sum the contributions: $0.45 + 0.30 = 0.75$.
Even though Execution is near perfect, the overall reward is pulled down by the weaker Text score, illustrating why weighting matters.
How does this weighted sum differ from simply averaging the channel scores?
A simple average treats every channel as equally important, which can drown out a critical failure. By assigning explicit weights $w_{i}$, the designer can amplify high‑priority channels (e.g., execution safety) and attenuate less‑critical ones.
The channel penalizes fatal JavaScript errors exponentially, while other warnings are ignored.
The channel checks for required strings after normalizing case and punctuation, then scales the binary match by layout‑aware factors.
It measures both how much content overflows the canvas and how well the overall aspect ratio matches the reference.
It checks that required tables and charts appear with the correct structure and content, weighting type and data appropriately.
It rewards spatial alignment of content blocks while ignoring style details like colour or typeface.
It compares coarse and fine occupancy grids derived from edge‑based masks to assess overall visual density.
GPT‑5.5 generates a scene‑graph‑based rubric from the reference and then asks yes/no questions about the candidate rendering.
The three template rules ensure that VGR questions focus on structural properties rather than visual details that the reference may vary.
**Figure D1** Reward channels. Six deterministic verifiers read the rendered DOM and screenshot. *Execution*: render success, decayed by uncaught JavaScript errors. *Text*: required strings found, weighted by overlap and in-canvas visibility. *Boundary*: in-canvas fraction of overflowing elements and aspect-ratio match. *Layout*: proximity of text blocks matched to the reference. *Table/Chart*: parsed table rows and ECharts series against the required data. *Whitespace*: cosine similarity of occupancy grids with the reference. *VGR*: pass rate of sample-specific yes/no predicates answered from the candidate render. Each channel is masked by applicability and normalized separately by GDPO. Values are from two rollouts on one prompt.
Evaluation Protocol and Limitations
We detail how the benchmark isolates visual generation without using training rewards.
The final benchmark evaluates visual generation without reusing the reference‑conditioned SemanticLayout, Whitespace, or VGR rewards from Section D. All scores are derived solely from the prompt and the candidate rendering.
For each of the 300 prompts we apply a frozen checklist containing 10 727 questions (mean 35.8 per prompt), split into Content, Layout, and Style categories; the Visual score is the mean of the available dimension pass rates.
Checklist questions must be answerable with a single YES/NO from the rendered image, name the element they assess, and include any required thresholds; ambiguous or implementation‑specific queries are rejected.
The judging template instructs the evaluator to consider the canvas fixed, treat full‑bleed elements as valid, and require explicit evidence (a “where” description) for every answer; unclear answers are counted as failures.
Per‑dimension briefs define Content as element presence, Layout as spatial relations, and Style as visual quality; each dimension’s score is the pass rate of its associated questions.
External benchmarks (Design2Code, SlidesBench, GDB/lica) are evaluated under their official protocols, reporting metrics such as block matching, CLIP similarity, NIMA, PickScore, and OCR fidelity; all systems are run with a single decoding pass for comparability.
Our human study recruited 52 participants, yielding 150 completed questionnaires; each questionnaire contains reliability, Content, aesthetic, and consistency items, with responses collected from three independent judges per item.
Judge reliability is measured by comparing automatic VGR and Visual checklist answers to the majority human response on 150 sampled predicates; “Unclear” counts as a failure for the automatic judge.
Content quality results show AesCode‑32B meeting more explicit Content requirements than GPT‑5.5 while achieving comparable aesthetic ratings on 80 briefs, with statistical equivalence defined by a 95 % confidence interval within ±0.5.
Score‑preference consistency analysis reveals that higher Visual scores predict human preference; agreement is computed by rewarding selections of the higher‑scoring image, with ties receiving half credit.
The external‑benchmark human study presents anonymized pairs of system outputs and asks participants to choose the more aesthetically polished image; each pair is judged five times, and agreement is aggregated across benchmarks.
Failure analysis focuses on three residual modes—Severe boundary, Relational subgraph, and Semantic element—measured over the 300‑prompt benchmark and three rollouts per system.
Table F1 reports recurrence rates: boundary failures drop from 23.9 % (GPT‑5.5) to 4.3 % (AesCode‑32B); relational subgraph failures remain high (~75 %); semantic element failures improve from 10.9 % to 34.7 %.
Figure F1 visualizes additional instances of each failure mode, illustrating why boundary overshoot, missing global subgraph coherence, and indistinguishable icons persist despite supervision.
Relational failures persist because relation predicates are sparse (only 46 of 10 727 questions) and their credit is diluted across many criteria; semantic failures persist because icon realism relies on the backbone’s visual knowledge, which is not directly supervised.
**Figure.** A promotional infographic for the Surface Laptop for Business, showcasing its hardware features and software capabilities. The central image displays a laptop connected to two external monitors and a docking station, surrounded by callout boxes detailing specifications such as Intel Core Ultra processors, Copilot+ PC integration, AI-enhanced voice clarity, battery life, and security features. The layout uses lettered labels (H, P, B, L, C, R, A, F) to categorize different aspects of the product ecosystem.
**Figure D6.** Portrait FAQ example (instance `poster_v2_00018`). The complete Chinese generation prompt and reference match the frozen evaluation inputs. The reference is shown once, with display annotations locating the stored groups and relations.
**Figure 1.** XX Health Insurance Bureau 2024 Key Policy Interpretation.
The benchmark deliberately avoids reusing training rewards, guaranteeing that evaluation reflects true visual generation capability.