MaLiang-Harness: A Programmable Path to Image and Video Generation
Haoyu Zhao, Zihao Zhang, Xudong Wang, Jiaxi Gu, Zuxuan Wu, Yu-Gang Jiang, Shuicheng Yan
MaLiang-Harness uses persistent executable programs to bridge the gap between code-level correctness and visual intent.
How can we bridge the gap between program-level correctness and visual requirement satisfaction in code-driven image and video generation?
Large language models can generate code to create images and videos, but a program that executes without errors often fails to match the user's specific visual requirements. This discrepancy between code-level success and visual intent is the Program-to-Visual (P2V) gap. MaLiang-Harness treats visual generation as a stateful, iterative process where the model plans, executes, and revises code based on rendered visual feedback. It maintains a persistent state of the artwork, records the construction history, and verifies each revision against the user's requirements. GPT-6-Astra achieves 100% generation success on both image and video benchmarks, with 96% of image tasks and 77% of video tasks meeting all quality thresholds.
Paper Primer
The framework hinges on three mechanisms: Persistent Executable Generation (PEG) state, which keeps the program and its context stable across edits; Traceable Generation Process (TGP), which logs operations and their visual outcomes; and Revision-aware Editing and Verification (REV), which ensures every change is re-assessed against the original requirements.
General capability scores are poor predictors of visual generation performance.
Models with similar general-capability index scores show widely divergent visual quality pass rates, such as two models with identical index scores achieving 44% and 88% quality pass rates respectively. Significant; visual generation requires specialized reasoning beyond general coding or knowledge tasks.
Why is a "harness" necessary instead of just prompting a model to write code once?
A single pass often fails to satisfy complex visual requirements like spatial composition or motion timing. The harness allows the model to inspect rendered evidence, identify discrepancies, and iteratively revise the code until the output meets all criteria.
What is the primary failure mode when the framework cannot reach a solution?
The framework encounters reasoning stagnation (planning stops advancing), semantic livelock (repeated edits fail to resolve requirements), or infinite agentic loops where the model repeats tool use without improvement.
Researchers should shift from evaluating visual generation via direct pixel synthesis to measuring the model's ability to manage stateful, iterative construction processes. Programmable generation is now a viable alternative to diffusion-based synthesis for tasks requiring explicit control.
The P2V Gap in Visual Generation
We expose the Program-to-Visual gap and outline MaLiang‑Harness as a stateful solution.
The Program‑to‑Visual (P2V) gap captures the failure of a syntactically correct program to produce an image or video that satisfies the user’s visual intent. MaLiang‑Harness tackles this gap by keeping a persistent executable state, linking each edit to rendered evidence, and verifying revisions before finalization.
**Figure 1.** MaLiang-Harness aims to explore a programmable path to image and video generation, enabling diverse image synthesis, visual reasoning and editing, and creative video generation.
The core challenge is that correct code does not guarantee the intended visual outcome.
The MaLiang-Harness Architecture
MaLiang-Harness keeps a persistent program state and logs every edit to bridge the P2V gap.
The P2V gap manifests when models emit syntactically correct code that nevertheless fails to produce the intended visual output. MaLiang‑Harness tackles this by persisting the executable program across revisions and by linking each generation operation to concrete visual evidence.
PEG stores a single mutable visual program together with its assets, so every edit builds on the existing state instead of restarting from scratch.
Revision 0: $S_0 = (\varnothing,\;\varnothing,\;\varnothing,\;C_0)$.
Apply edit $a_0$: add command “draw circle radius 10 at (0,0)”. $S_1 = (P_1=\text{draw\_circle},\;A_1=\varnothing,\;Z_1=\text{centered},\;C_1)$.
Apply edit $a_1$: add command “draw leaf attached to circle”. $S_2 = (P_2=\text{draw\_circle; draw\_leaf},\;A_2=\varnothing,\;Z_2=\text{centered+leaf},\;C_2)$.
Persisting $S_k$ lets the second edit reuse the circle geometry without recomputing it, illustrating incremental construction.
TGP records every generation operation together with its inputs, outputs, and the source and target PEG revisions, providing a complete audit trail of how the visual program evolves.
Operation 1: $o_1=$ “draw\_leaf”, $x_1=$ (position relative to circle), $y_1=$ success, $k\!-\!1=1$, $k\!+\!1=2$ → creates $S_2$.
Operation 2: $o_2=$ “render”, $x_2=$ (time = 0), $y_2=$ image $R$, $k\!-\!2=2$, $k\!+\!2=2$ → no new revision.
TGP distinguishes between edits that change the program (creating a new revision) and pure renderings that merely observe the current state.
Receive prompt $p$ and output specification $\omega$.
Plan appearance, composition, and temporal dynamics; select a backend $b$.
Generate or edit the visual program $P_k$ and assets $A_k$ using the MLLM.
Commit the edit via $E$, producing a new PEG revision $S_{k+1}$.
Render the program with backend $b$ to obtain observation $R_{k+1}$.
Evaluate $R_{k+1}$ against the requirements in $C_{k+1}$.
If evaluation fails, record the operation in TGP and return to step 3; otherwise, proceed to verification.
Verification (REV) checks that all mandatory visual requirements are satisfied; when they are, export the final revision.
**Figure 3.** Overview of the MaLiang-Harness for image and video generation. Persistent Executable Generation (PEG) state maintains the program and task context across revisions. Traceable Generation Process (TGP) links generation operations to state changes and visual results. Revision-aware Editing and Verification (REV) supports revision from historical states and assesses the current output using evidence from the same revision.
**Figure 4.** **Example of code-driven image synthesis.** The “botanical portrait” is rendered from the visual program $P_k$ of a PEG state without image assets. Algorithm 1 summarizes how the program specifies geometry, appearance, and spatial composition through Canvas drawing operations.
REV ties each visual requirement to the specific PEG revision that produced the supporting evidence, and only when all requirements pass does the system export the revision as the final output.
Quantitative Performance on IBench
IBench quantifies the gap between generation success and visual quality across models.
IBench separates raw code‑generation success from the visual quality of the resulting images, letting us see how well models close the P2V gap.
This section reports the quantitative IBench results, focusing on how close each model comes to bridging the generation‑success versus visual‑quality gap.
GPT‑6‑Sol achieves perfect generation success, improving over GPT‑5.6‑Luna by 8 percentage points.
Success 100 % for GPT‑6‑Sol versus 92 % for GPT‑5.6‑Luna (Table 2).
**Figure 2.** **Model rankings on MaLiang-IBench.** Ranks are computed across 11 models; better ranks lie farther from the center, with ties averaged. GPT models achieve 92–100% generation success, versus 12–40% for DeepSeek and Kimi. Mean quality scores use successful outputs (counts in parentheses). Cost ranks favor less time and fewer tokens.
The remaining P2V gap shows that perfect generation success does not automatically yield high visual quality.
Quantitative Performance on VBench
Astra dominates on success and quality, while other models lag behind on token limits and motion.
GPT‑6‑Astra achieves perfect task completion and satisfies all four quality thresholds on ten of its thirteen successful videos.
Table 4 shows Astra’s 100 % success rate; Table 5 shows it meeting every quality criterion on ten cases.
Token‑budget exhaustion explains only part of the failures: it accounts for five of DeepSeek’s thirteen failures, one of Kimi’s ten, and three of GPT‑5.6‑Sol’s six. For Astra, motion coherence is the bottleneck, with only ten of thirteen videos satisfying the motion criterion.
**Figure 11.** Qualitative comparison on MaLiang-VBench. Groups show (a) Star Delivery, (b) The Tiny Dragon Barista, and (c) Watering-Can Rafting. Each model row contains five uniformly sampled frames in temporal order from left to right. A white $\times$ on black denotes an unsuccessful run. Full prompts are provided in Appendix 6.5.
Qualitative Visual Comparisons
Qualitative assessment on MaLiang‑IBench reveals Astra’s superior detail and higher success rate.
The qualitative grid in Figure 6 compares five models across five prompts, highlighting differences in detail, scale, and visual fidelity. Rows span diverse scenes—from a fox on a teacup to a greenhouse—while columns show each model’s output.
Astra achieves the highest mean quality score among the GPT family on MaLiang‑IBench.
Astra scores 4.54, compared to 4.33 for GPT‑5.6‑Luna and 4.25 for GPT‑6‑Luna.
**Figure 6.** Qualitative comparison on MaLiang-IBench. Rows show a fox on a teacup, an architectural concept board, a rabbit on a ginkgo leaf, a robot ensemble, and a greenhouse illustration. Columns compare five models under the same prompt. A white × on black denotes an unsuccessful run. Full prompts are provided in Appendix 6.4.
The greenhouse and robot‑ensemble rows demonstrate Astra’s capacity for intricate scene composition, while the teacup and ginkgo‑leaf rows expose remaining gaps in material realism for the other models.
Refinement and Future Directions
We examine how each component of MaLiang‑Harness impacts visual realism and traceability.
The P2V Gap captures the failure of models to produce visually compliant outputs despite syntactically correct code. MaLiang‑Harness addresses this by keeping program state persistent (PEG) and linking generation steps to visual evidence (TGP).
**Figure 5.** Revision-aware editing and verification. Edits produce new PEG revisions, and visual evidence is associated with the revision it assesses. Historical evidence supports comparison; delivery requires verification of the current revision.
More expressive rendering backends let MaLiang‑Harness plug in path‑tracing engines that model material properties, lighting, and shadows. Using a raster preview for composition, the system then refines the scene with a higher‑sample path‑traced render, revealing both new realism and new artifacts.
The improvement shows that richer physical simulation contributes to perceptual realism, but the added complexity also introduces revision‑specific defects that must be caught by PEG/TGP.
**Figure 7.** Iterative refinement with a path-tracing backend. Top: GPT-6-Astra revises a procedural breakfast scene through geometry, lighting, material, and sampling adjustments. Bottom: detail crops show “biscuit” surface artifacts introduced during refinement and reduced by geometry corrections.
Brushwork refinement mimics hyperrealist painting by issuing finer‑stroke prompts (REV) while keeping the same canvas backend. The approach preserves overall composition but often erodes existing detail, especially in foliage and stone textures.
Fine‑stroke prompts alone cannot recover the coordinated shape, shading, and texture needed for realism, highlighting the necessity of richer rendering backends.
We compare public Artificial Analysis (AA) Intelligence Index scores with MaLiang‑IBench drawing‑quality pass rates across 11 models. The Spearman rank correlation is $\rho$ = 0.65, yet models with identical index scores (e.g., GPT‑5.6‑Luna vs. GPT‑6‑Luna) show markedly different pass rates (44 % vs. 88 %).
General capability predicts visual performance only loosely; specialized visual reasoning and rendering pipelines are decisive factors.
MaLiang‑Harness records the executable program, asset provenance, and revision history, making the construction process inspectable. The trace shows how layout code, generated assets, and reviews combine to produce the final composition.
Traceability reveals why the final image may still be a draft—numerical plots pass, but photographic assets remain unsatisfactory.
Refinement can stall due to three failure modes: reasoning stagnation, semantic livelock, and infinite agentic loops. Stagnation occurs when planning no longer advances the artwork; livelock when edits keep requirements unresolved; loops when planning and tool use repeat without termination.
PEG preserves state, while TGP and REV enable inspection, but they do not guarantee termination; explicit limits on calls, time, and tokens are required.
Limitations and Failure Modes
This appendix details the construction algorithm, evaluation protocol, and highlights limits of the refinement process.
The P2V gap is the mismatch between syntactically correct code and visually satisfactory output; MaLiang‑Harness bridges this gap by persisting program state and linking generation steps to visual evidence. This appendix revisits that premise briefly before exposing the concrete construction algorithm, the detailed evaluation protocol, and the observed limits of refinement.
**Figure 8.** **Limits of prompted brushwork refinement.** Left: a Canvas illustration before and after GPT-6-Astra edits its drawing program to produce finer brushwork within REV. Right: paired detail crops show each region before (top) and after (bottom) editing. Finer marks do not yield photorealistic appearances: grass coverage and distinctive surface texture are instead reduced in several regions.
Algorithm 2 – procedural generation of the botanical portrait.
Evaluation follows a two‑stage protocol: first, tasks are judged for successful execution (decodable output and completion checks); second, successful outputs are scored on alignment, aesthetics, composition, and for videos, motion coherence. Token‑limit failures are recorded separately, reflecting cases where the model exhausts its allocated token budget before completing the task.
Comparative Analysis and Metrics
This appendix details plot conventions, color encodings, prompt specifications, and a layout example.
Radar rankings are built from six metrics taken from Tables 2 and 3: mean alignment, aesthetics, composition, generation success rate, reported time per task, and token count. Each metric is ranked across all 11 models, higher‑is‑better for quality and success, lower‑is‑better for cost, and ties receive average ranks. The rank r is mapped to radius $(12 - r)/11$, placing rank 1 at the outer edge.
General‑capability comparison uses the integer Artificial Analysis Intelligence (AA) Index scores from version 4.3.2 and the image‑quality pass rate (All Criteria ÷ 50). Across 11 models the Spearman correlation is $ρ = 0.65$; excluding the unverified DeepSeek‑V4‑Pro checkpoint reduces it to $ρ = 0.63$. The analysis therefore reflects an association rather than a causal effect of model capability.
Numerical color scales in Tables 2 and 4 encode relative changes against a baseline model. For a reported value $x$ and baseline $b$, intensity is $\min\bigl(|x/b - 1|,\,1\bigr)$. Video success‑rate intensity uses $|x - b|/100$ because DeepSeek has zero successful videos. The baseline for Video Time/Success is Kimi‑K2.6.
Quality‑table highlighting follows a similar rule: blue/orange denote higher/lower GPT results relative to GPT‑5.6‑Luna (images) or GPT‑5.6‑Sol (videos). DeepSeek and Kimi use a separate gray scale; darker gray means larger values. Bold font marks the best GPT values, excluding the descriptive “Evaluated” column.
Prompt 1 – “Fox Painter on a Teacup”: a 1536 × 864 PNG showing a flat‑style orange fox painting a blue arc on a realistic porcelain teacup, with precise placement constraints for the fox, brush, and tea.
Prompt 2 – “Mountain Teahouse Concept”: a 2048 × 1152 architectural board split into a main axonometric view and two side panels (front elevation and section), annotated with three detail callouts.
Prompt 3 – “Rabbit Violinist on a Ginkgo Leaf”: a 1536 × 1024 mixed‑media illustration of a cartoon rabbit playing a violin on a realistic ginkgo leaf, with three dewdrops and a specific size ratio.
Prompt 4 – “Garden Robot Cast”: a 2048 × 1152 landscape showing six distinct garden robots arranged in two rows, each with a unique head shape and tool, rendered in a clean children’s‑animation style.
Prompt 5 – “Greenhouse Potting Bench”: a 1536 × 1024 children’s‑book style scene featuring four anthropomorphic objects (pot, watering can, glove, seed box) arranged around a greenhouse bench, with soft pastel colors and a single frozen water droplet.
Layout‑constraint example: a 2048 × 1152, 16:9 canvas containing exactly 15 rectangles identified by IDs (H1…F1) with explicit normalized coordinates and color codes. The specification demands no overlaps, no extra elements, and precise placement of each colored block.
**Figure 9.** General capability and visual program generation on MaLiang-IBench. (a) Public AA Intelligence Index scores versus the percentage of all 50 tasks meeting every visual quality threshold. The dashed line highlights the 44-percentage-point gap between the two Luna models. (b) Models ordered by AA score (brackets). Dark teal, light teal, and gray denote all criteria met, successful generation with unmet criteria, and failure. White numbers give quality pass rates and rightmost numbers give generation success rates. †The public V4-Pro score uses checkpoint 0813.
This table lists layout components by ID, their spatial coordinates $[x, y, \text{width}, \text{height}]$, and their associated color.
Prompting and Evaluation Details
Prompt specifications for three qualitative video comparison scenarios.
Star Delivery: a pixel‑art animation where a kitten catches a star ejected from an alarm clock, rides a keyboard‑built railway, slides down book‑page rapids, and swings on a desk‑lamp pull‑cord to illuminate the desk. The scene must stay in a consistent 16:9 pixel‑art style with uniform pixel size, limited palette, stepped contours, and minimal dithering—no photorealistic assets, hand‑drawn lines, 3D voxels, or mosaic filters. Camera remains stable, with only slight lateral tracking, and all structures appear by assembling pixel blocks frame‑by‑frame.
The Tiny Dragon Barista: a 10‑second looping JavaScript Canvas 2D animation of a tiny orange dragon sleeping in a coffee cup, sneezing flame that ignites floating beans, chasing them around the cup, and finally turning latte‑foam into a dragon face before returning to sleep. The visual language mixes a semi‑realistic café still‑life with hand‑drawn fantasy overlays—inked flames, scribbled steam, glowing beans, and hand‑drawn motion trails—using a palette of espresso brown, cream, sage, and terracotta orange. The loop must end with the dragon in the exact same curled pose as the start.
Watering‑Can Rafting: a real metal watering can on a tabletop becomes the source of a doodle‑drawn white‑water ride, where tiny doodle cats add paddles, life rings, and channels that come alive as water flows, creating rapids, waterfalls, and whirlpools populated by boats and fish. The animation blends the physical can with hand‑drawn line art for water, docks, and splashes, preserving a stable camera and avoiding antialiased edges, motion blur, or realistic water effects. The scene culminates in a dramatic flood‑powered amusement ride that loops back to the initial still.
The 12‑frame quality review samples five uniformly spaced frames per model for visual comparison, distinct from the detailed frame‑by‑frame description above, ensuring evaluators see consistent action progression across all prompts.