AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design

Yaxin Luo, Haobin Jiang, Jialv Zou, Xu Huang, Wenhao Yan, Haodong Li, Zhengrong Yue, Jing Li, Xiaofu Chen, Xiaohan Zhao, Jiacheng Liu, Jiacheng Cui, Zhiqiang Shen, Xiaotong Li

AutoDesign recursively optimizes a design harness to transform multimodal sources into editable, human-aligned artifacts.

How can we autonomously optimize a "design harness"—a system of instructions and feedback loops—to improve the quality of long-horizon agentic artifact generation?

Multimodal design systems often treat human feedback as transient, failing to accumulate reusable knowledge from successful revisions or failures. AutoDesign introduces a meta-harness optimizer that treats the design system itself as an evolving program, using rollout feedback to recursively update its functional components while keeping the underlying model fixed. On the PosterBench academic poster task, this approach achieves a score of 78.32, outperforming commercial systems like Claude Design by 7.45 points.

Paper Primer

The framework separates the design process into two nested loops: an inner loop that generates and revises a single artifact, and an outer meta-harness loop that updates the system's code based on execution trajectories. The meta-harness acts as a meta-optimizer: it analyzes failure patterns across multiple runs and proposes bounded code updates to specific harness components, such as the critic or the tool-use policy.

Integrating the learned DesignHarness consistently improves performance across diverse model and agent configurations.

Average PosterBench scores increased from 54.99 to 67.39 across seven different model-agent setups.

Why optimize the harness instead of fine-tuning the underlying model?

The authors argue that persistent design capabilities—such as layout rules, provenance tracking, and tool-use policies—are better captured as reusable code in the operational scaffold than as implicit weights in a model, allowing for interpretable, localized updates.

What prevents the meta-harness from "reward-hacking" its own performance metrics?

The system employs an acceptance gate that requires performance gains on a held-out development set, ensuring that updates generalize beyond the training tasks and do not regress on unseen data.

AutoDesign demonstrates that complex, long-horizon agentic tasks can be solved by treating the agent's "operating system" as a searchable, self-improving program rather than relying solely on model-level training.

Introduction: Agentic Design Challenges

AutoDesign reframes multimodal design as harness‑centric meta‑optimization.

Current design pipelines treat the harness—the collection of prompts, validators, and rendering rules—as a fixed artifact; they cannot evolve from experience, so each new task requires hand‑crafted tweaks that quickly become brittle.

We need a way for the harness itself to be optimized, not just the underlying model, so that repeated design episodes gradually improve the whole workflow.

**Figure 1.** AutoDesign progressively improves the design harness and the quality of the artifacts. (a) Score of the poster generated by the design harness for one representative paper, tracked across meta-harness iterations. Autonomous optimization improves the initial harness before reaching a plateau, after which human guidance redirects the search and yields a further gain. (b) The optimized harness, DesignHarness, improves all Coding Agents on PosterBench by 5.0 to 19.6 points, achieving a best overall score of 81.5.

The key insight is to move from model‑centric to harness‑centric design, letting the system itself evolve its own workflow.

Defining the Design Harness

We formalize the Design Harness as a modular system that wraps a fixed model to generate artifacts.

The core difficulty is that the underlying LLM (or MLLM) is immutable, yet we need to improve the quality of the artifacts it produces. Treating the surrounding system as an independent optimization target lets us upgrade the pipeline without touching model weights.

A Design Harness is a wrapper around a fixed LLM that orchestrates inputs, tools, and feedback to turn a source document into a finished artifact.

The harness parses $c$ and creates a prompt “Generate a slide with title and bullets from the abstract”.

$\pi_{\theta}$ produces a draft slide containing a title and three raw bullet sentences.

The Evaluation component runs a Vision‑Language Model (VLM) critic that flags the title as too long.

The Orchestration module revises the prompt to “Shorten the title”, and the harness re‑invokes $\pi_{\theta}$.

The final artifact $y$ is a concise slide that satisfies the VLM feedback.

The example shows how the harness iteratively refines $y$ by looping between generation and model‑based critique, all while the underlying LLM remains unchanged.

How does a Design Harness differ from simply prompting an LLM?

Prompting an LLM is a single forward pass; a Design Harness adds persistent context, tool integration, and a feedback loop that can revise the output multiple times, turning a one‑shot generation into an iterative design process.

The meta‑harness sits above the Design Harness and treats it as the optimization target: it proposes updates to the harness components, evaluates the resulting artifacts, and accepts improvements that raise expected quality.

**Figure 3 Overview of AutoDesign.** The design harness iteratively generates and revises an artifact using critic feedback, thereby constituting the inner loop. The outer loop improves the design harness by running it on design tasks, evaluating its outputs, proposing an update to one component, and accepting or rejecting the candidate update. This process can run autonomously, with optional human guidance.

The Learning Loop Architecture

Nested loops let AutoDesign refine artifacts and then improve the harness itself.

The design harness can polish a single artifact, but without a higher‑level controller it never learns from repeated failures across tasks.

**Figure 4** The inner and outer loops of the AutoDesign Framework. The inner loop updates the artifact according to the design harness, while the outer loop updates the design harness.

The inner loop treats the design harness as a fixed program that repeatedly drafts and critiques an artifact until it stabilises.

Step 1: $y_1 = M_{\text{design}}(\varnothing, \varnothing; x,c)$ yields a coarse layout.

Step 1 critic: $f_1 = M_{\text{critic}}(y_1; x,c)$ returns “add a tree on the left”.

Step 2: $y_2 = M_{\text{design}}(y_1, f_1; x,c)$ incorporates the tree.

Step 2 critic: $f_2 = M_{\text{critic}}(y_2; x,c)$ returns “increase sky brightness”.

Step 3: $y_3 = M_{\text{design}}(y_2, f_2; x,c)$ produces the final image.

Step 3 critic: $f_3 = M_{\text{critic}}(y_3; x,c)$ returns “done”.

The trajectory $\tau$ captures a concrete revision history that the outer loop can later analyse for systematic shortcomings (e.g., missing background elements).

How does this inner loop differ from a simple generate‑then‑evaluate pipeline?

Here the designer and critic alternate repeatedly, feeding the critic’s feedback back into the designer. A one‑shot pipeline would generate once and stop, losing the chance to correct errors iteratively.

The outer loop treats the whole inner‑loop scaffold as a black box, watches many runs, and then rewrites the scaffold itself to fix recurring failures.

Task 1 yields scores $s^{(1)} = (0.8,0.6,0.7,0.9,0.5,0.7,0.6)$.

Task 2 yields scores $s^{(2)} = (0.5,0.4,0.6,0.7,0.3,0.5,0.4)$.

The optimizer $P$ notices low visual‑evidence scores (fourth dimension) across both tasks.

It proposes adding a “visual‑evidence checker” module to the harness component “feedback generator”.

The acceptance gate checks that the new harness improves the average training score from $0.63$ to $0.68$ and leaves the development score unchanged, so the update is accepted.

The outer loop turns a pattern in the raw scores (systematic low visual evidence) into a concrete code edit, and the acceptance gate guarantees that the edit does not over‑fit the training set.

Why restrict each outer‑loop iteration to modify only one harness component?

Limiting edits to a single component isolates cause and effect: any performance change can be attributed to that component, making analysis and debugging tractable.

Algorithm 1 – meta‑harness optimization loop.

**Figure 5. AutoDesign outer loop.** Each iteration proceeds through rollout, evaluation, update proposal, and acceptance. The loop runs autonomously, with optional human guidance to redirect the evaluation and optimization.

DesignHarness Implementation

DesignHarness iteratively refines artifacts using rule‑based and visual critics to ensure valid, high‑quality outputs.

The iterative design loop often stalls because a single validator either misses aesthetic flaws or over‑constrains structure. The solution is to pair a deterministic rule‑based validator with a vision‑language model critic, letting each catch what the other cannot.

Two independent critics—a rule‑based validator for structural safety and a VLM for visual quality—jointly produce a repair signal that guides the designer toward both correct and attractive artifacts.

How does Dual‑Critic Feedback differ from using only a rule‑based validator?

The rule validator catches structural violations but cannot judge aesthetics; the VLM adds a learned visual quality check. Their combined repair signal $f_k$ therefore steers the designer to satisfy both hard constraints and subjective design goals, which a single validator cannot achieve.

Attempt 1: Designer produces $y_1$ lacking the required logo image; rule validator returns a blocking diagnostic “missing asset: logo.png”.

Repair signal $f_1$ contains the missing‑asset fix; designer inserts the logo, yielding $y_2$.

Attempt 2: $y_2$ includes the logo but the VLM scores the layout 0.4 (below the 0.6 threshold), returning a non‑blocking critique “layout imbalance”.

Repair signal $f_2$ encodes a layout adjustment; designer reorders sections, producing $y_3$.

Attempt 3: $y_3$ passes all blocking checks and receives a VLM score 0.78, so the loop terminates and finalization proceeds.

The two critics address complementary failure modes: deterministic checks enforce feasibility, while the VLM guides the artifact toward human‑like visual quality.

**Figure 6** Overview of DesignHarness, optimized for human-facing artifact generation. Given source materials, the designer iteratively generates and revises an editable artifact using feedback from dual critics: a rule-based validator and a model-based visual critic. DesignHarness then finalizes the best valid candidate.

Benchmarking with PosterBench

We evaluate AutoDesign on PosterBench and report its benchmark scores.

The AutoDesign framework treats the design harness as a meta‑optimization target, letting agents iteratively improve artifacts via dual‑critic feedback. This section details how we assess the resulting posters using the PosterBench benchmark.

AutoDesign achieves the highest PosterBench Score on the 100‑paper Main Track.

Table 1 shows AutoDesign scoring 78.32, 7.45 points above Claude Design and 8.87 points above OpenDesign.

On the PosterBench‑mini subset, AutoDesign reaches 81.46, outperforming the Codex baseline (75.87) and Claude Code (74.56). When isolating the design‑harness contribution (Table 3(a)), AutoDesign scores 74.56 versus Claude Design’s 66.83 and OpenDesign’s 70.36. The coding‑harness track (Table 3(b)) highlights Kimi Code’s 82.31 top score, while the model track (Table 3(c)) records Claude 4.8 at 74.56.

**Figure 7.** PosterBench evaluation protocol. PosterBench combines rule-based algorithms for traceable spatial, OCR, numeric-grounding, and render-integrity checks with rubric-guided VLM judges for source-grounded assessment of visual evidence, layout, readability, and aesthetics.

**Figure 16** Additional matched qualitative comparisons across Design Agents with the same generation prompt; each row shows three independently generated posters for one source paper, at a common scale.

PosterBench provides a multi‑dimensional score for agentic design.

Ablation Studies

Design Harness ablations reveal consistent performance gains across model‑agent pairs.

**Table 4.** Effect of attaching DesignHarness while keeping the model and coding agent fixed. Exact PosterBench Scores for each completed configuration are ordered by final performance.

Attaching the Design Harness consistently improves PosterBench scores, with gains ranging from +5.01 to +19.56 points across seven model‑agent configurations.

Table 4 reports the exact score increases for each configuration after harness attachment.

Cost-Performance Tradeoff

Shows how AutoDesign models balance PosterBench score against designer‑API cost.

GPT‑5.5 attains the highest PosterBench score (81.46) while incurring the greatest designer‑API cost (\$10.02 per poster) among the seven AutoDesign configurations.

Figure 8 visualizes the full Pareto frontier from LongCat‑2.0 to GPT‑5.5.

**Figure 8.** Cost–performance trade-off across seven AutoDesign model configurations on PosterBench-mini, the fixed 10-paper subset; marker size denotes median runtime, and the dashed line denotes the empirical Pareto frontier.

The design‑harness ablation (variant a) shows that using the Claude Code coding harness with the AutoDesign design harness yields 74.56 score, whereas OpenDesign drops to 66.83, confirming the importance of the dual‑critic feedback loop.

Controlled Experiments

AutoDesign boosts PosterBench performance, achieving up to +19.56 points gain and a top score of 82.31.

AutoDesign improves PosterBench scores by up to +19.56 points compared to the original configuration.

Table 4 reports the per‑model gains when the DesignHarness is attached while keeping the model and coding agent fixed.

Human Evaluation

Human judges rank posters, revealing AutoDesign’s clear advantage and score‑human alignment.

AutoDesign attains a 64.0 % Bradley–Terry probability of beating a randomly sampled alternative, outpacing all baselines.

Eleven volunteers provided 936 pairwise judgments on 100 papers; the Bradley–Terry estimate for AutoDesign is 64.0 % (95 % interval 55.2–77.8 %).

**Figure 9.** System-blind human evaluation. (a) Bradley–Terry probability of beating a uniformly sampled alternative, with 95% intervals crossed by paper–reviewer bootstrap replicates. (b) AutoDesign head-to-head outcomes against each baseline. Eleven volunteer reviewers submitted 936 responses, including 933 ranking judgments and three skips.

**Figure 10.** PosterBench Score alignment with system-blind human preference. (a) Poster-level score association. (b) Human agreement increases with the PosterBench Score margin.

Related Work and Future Directions

We outline extensions to multimodal design and situate AutoDesign among related systems.

Extending AutoDesign beyond paper‑to‑poster requires a multimodal‑in, multimodal‑out agentic design system that can ingest papers, visual evidence, code, data, and human guidance, then iteratively produce the appropriate medium‑specific artifact.

Each new medium demands its own source‑output dataset, a dedicated evaluator, a rendering and validation gate, and an objective tuned to the communication setting; currently PosterBench only covers posters, leaving slides, webpages, and videos as pilot experiments.

Shared context construction, preference memories, and repair histories could serve as a substrate for cross‑media reuse, but realizing this transfer hinges on medium‑specific objectives and evaluation protocols.

At the meta‑harness level, open problems include better component selection and evaluator evolution; a selector must weigh failure attribution, uncertainty, expected improvement, and component interactions when choosing the next bounded update.

Adaptive evaluators must remain versioned and anchored by frozen reference tasks, adversarial probes, and periodic human audits to prevent reward‑hacking of a moving target.

Recent work on continual harness adaptation, self‑improving harnesses, and model‑harness co‑evolution (Karten et al., 2026; Zhang et al., 2026a; Lee et al., 2026a) points toward joint training that preserves the division while evaluating both layers against shared held‑out objectives.

**Figure 12** Future direction: a multimodal-in, multimodal-out agentic design system that integrates diverse sources and human guidance to iteratively create medium-specific outputs.

Prior poster‑generation systems (SciPostLayout, deep submodular extraction, Paper2Poster, P2P, PosterGen, PosterForest, Any2Poster) combine multimodal inputs with specialized agents and visual refinement, establishing the input, output, and representation choices for this domain.

Analogous systems generate slides, webpages, and narrated videos from papers (Fu et al., 2022; Zheng et al., 2025; Ge et al., 2025; Yang et al., 2025; Chen et al., 2025; Zhu et al., 2025), often leveraging structured representations like HTML/CSS to enable editability and visual inspection.

Critic‑driven regeneration mechanisms such as Self‑Refine, Reflexion, Voyager, and ExpeL improve multimodal outputs without altering the generation system, preserving experience across attempts.

Foundational autonomous‑agent literature (Wooldridge & Jennings 1995; Schmidhuber 1987; Schmidhuber et al., 1997) provides the conceptual backdrop for self‑referential learning and long‑term reward optimization.

Recent efforts separate model‑parameter updates from scaffold improvements (Ren et al., 2026) and introduce component‑level optimizers (TextGrad, DSPy, GEPA) as well as workflow‑search systems (STOP, GPTSwarm, ADAS, AFlow) that explore code‑ or graph‑based pipelines.

At the full‑harness level, systems such as Self‑Improving Coding Agent, MOSS, Meta‑Harness, HarnessX, Self‑Harness, and Agentic Harness Engineering study searchable harness programs, composable primitives, bounded updates, and outcome attribution; HarnessX further uses execution traces as signals for both harness evolution and future model training.

**Figure 13** Future directions for AutoDesign. The initially optimized DesignHarness already extends paper-to-poster generation to slide decks, webpages, and conference videos. Applying the same meta-harness optimization methodology to these and other multimodal outputs offers a path toward a general multimodal-in & out agentic design system.

AutoDesign embodies this direction by evolving a Design Harness that accumulates design priors while keeping model weights fixed, using an independent development acceptance gate and evaluating the final system with PosterBench and a system‑blind human study.

Supplementary Materials

Supplementary materials detail the benchmark, prompts, harness evolution, scoring, records, and extra posters.

The appendix makes the experimental interface inspectable by documenting the controlled comparisons, shared generation prompt, frozen scoring protocol, released records, and extra poster examples.

A.1 lists the benchmark inputs and the comparison matrix that defines which factor is held constant and which is varied for each track; the matrix is reproduced in Table 5.

**Table 5.** Controlled comparison matrix. Main tracks compare complete systems; controlled tracks vary only the factor named in the final column.

A.2 presents the AutoDesign generation interface, beginning with the system‑prompt excerpt that fixes the target artifact and grounding constraints, followed by the shared user prompt that all compared systems receive.

The shared user prompt (verbatim) requires a 3072 × 1536 px landscape poster, a three‑line header, three balanced columns, and strict avoidance of logos, side rails, or extra metadata.

A.3 describes the DesignHarness evolution: the optimized harness aggregates five components—grounding, specialist tools, workspace/runtime, direct control, and quality gates—into a coherent system that supports iterative refinement.

A.4 details the PosterBench evaluation interface, which scores a rendered poster on seven dimensions $q_j\in[0,10]$ with predefined weights that sum to 100.

**Table 6.** PosterBench scoring protocol. "Programmatic" denotes image-native and source-grounded checks; "VLM" denotes a dimension-specific judgment conditioned on the rendered poster and compact source context. The weights sum to 100.

A.5 releases the benchmark record schema, exposing system configuration, source case, evaluation status, aggregate score, and per‑dimension scores for independent auditing.

A.6 showcases additional poster demonstrations; Figure 17 displays LongCat‑Next and NeRF posters, while Figure 18 continues with Attention Is All You Need and DDPM.

**Figure 17.** Additional AutoDesign poster demonstrations. LongCat-Next and NeRF are rendered from their respective source papers.

**Figure 18.** Additional AutoDesign poster demonstrations (continued). Attention Is All You Need and DDPM are rendered from their respective source papers.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers