Recursive Harness Self-Improvement

Hyunin Lee, Jinglue Xu, Jeffrey Seely, Donghyun Lee, Matei Zaharia, Yujin Tang

Recursive Harness Self-Improvement (RHI) iteratively refines agent workflows to boost performance and reduce inference costs.

Can we improve model performance by iteratively optimizing the "harness"—the system of agents, prompts, and workflows surrounding a foundation model—rather than just the model weights themselves?

Foundation models rely on harnesses—the scaffolds that organize agent workflows—to generate high-quality execution traces, but manually updating these scaffolds for every new task is prohibitively expensive. Recursive Harness Self-Improvement (RHI) treats the harness as a prompt-level specification and iteratively refines it using pairwise feedback over its own revision history. This lightweight approach optimizes task-specific context management without requiring population-level search. Across 30 synthetic machine-learning research tasks, RHI consistently outperforms standard test-time scaling baselines while reducing inference costs by up to 60%.

Paper Primer

RHI optimizes the agent loop by treating the harness as a prompt-level object rather than executable code. It performs noisy local ascent by comparing the current harness output against the immediate predecessor, using the accumulated preference history to guide subsequent revisions.

RHI raises the performance ceiling of test-time scaling.

In pairwise comparisons against stronger baselines (e.g., max reasoning effort), RHI-improved agents consistently win across multiple base models. Outperforms max-reasoning baselines while reducing inference cost by up to 60%.

The performance gains are driven by improved task-specific context management rather than longer reasoning traces. RHI learns to optimize "contracts"—textual interfaces that restrict inter-agent information flow to only what is necessary for downstream decisions, effectively creating a task-specific sparsity pattern.

Why is RHI more efficient than existing population-based harness search methods?

Population-based methods require expensive black-box executions for every candidate in a large population. RHI reduces this to a single execution and one pairwise evaluation per iteration by comparing only against the immediate previous harness.

Does RHI replace the need for stronger foundation models?

No; RHI is complementary to train-time scaling. While it significantly improves the performance of a fixed model, it does not eliminate the performance gap between smaller and larger base models.

Introduction: Harness-in-the-Loop Learning

Introducing the need to treat harnesses as learnable, data‑generating components.

Model–harness co‑evolution creates a loop where better harnesses produce higher‑quality execution traces, which in turn train stronger foundation models. Provider‑built harnesses are costly to update, so we ask whether user‑constructed harnesses can be iteratively refined cheaply. This section frames the gap: harnesses are not just inference scaffolds but data‑generating components whose quality limits the overall learning flywheel.

A harness is the surrounding system of agents, prompts, and orchestration protocols that governs how a foundation model is used and, crucially, what execution traces it emits.

Test‑time scaling refers to increasing a model’s reasoning effort or size at inference to boost performance, e.g., using more reasoning hops or larger models.

**Figure 1.** Few-shot RHI raises the performance plateaus of test-time scaling. Across 30 synthetic ML research tasks, high-reasoning sonnet-4.6, opus-4.7, and opus-4.8 agents with RHI-improved harnesses achieve higher pairwise win counts than evaluated test-time scaling baselines selected from xhigh, max, and ultracode. Bars report mean wins over two LLM judges and three seeds. A crown (👑) marks mean wins above 19.5 out of 30.

**Figure 14.** Distributions of normalized cost, output tokens, and cache read/write across the 30 ML synthetic tasks. The boxplots show that the distributional trends are consistent with the mean values reported in Figures 5b, 6c, and 7e.

Treat harnesses as data‑generating components: improving them yields higher‑quality traces without expensive model scaling.

Problem Definition

We formalize the harness optimization goal and expose why existing search is too costly.

Population‑based harness search evaluates every candidate by running the full language model, which makes each iteration expensive and scales poorly to many user‑defined tasks.

The goal is to find a harness that wins pairwise comparisons against any competing harness as often as possible for a given task.

Is the Ideal Objective simply maximizing a scalar reward?

No. It maximizes the expected *pairwise* win probability, which aggregates many quality dimensions via preferences rather than collapsing them into a single scalar reward.

Sample a rival: for $H_{1}$ the rival is $H_{2}$; for $H_{2}$ the rival is $H_{1}$.

Generate outputs: $y_{1}\sim A(H_{1},x)$, $y'_{1}\sim A(H_{2},x)$; similarly $y_{2}\sim A(H_{2},x)$, $y'_{2}\sim A(H_{1},x)$.

Run Leval: suppose $\text{Leval}(y_{1},y'_{1},x_{\text{eval}})=y_{1}\succ y'_{1}$ and $\text{Leval}(y_{2},y'_{2},x_{\text{eval}})=y'_{2}\succ y_{2}$.

Compute win rates: $f_{x}(H_{1})=1$ (wins against its only rival), $f_{x}(H_{2})=0$ (loses).

Optimal harness $H^{*}_{x}$ is $H_{1}$ because it has the higher expected win rate.

This toy illustrates that the Ideal Objective directly ranks harnesses by how often they are preferred, without needing a numeric score for each repository.

Prior work sidesteps the intractable exact maximization by restricting the search to a concrete representation (prompts, workflows, tool‑use policies) and replacing the expectation over $\mu$ with a finite candidate set $S_{i}$ that can be evaluated.

Because each candidate still requires a full model execution and evaluation, such population‑based methods become prohibitive for user‑constructed harnesses; we therefore need a lightweight update rule that converges in only a few iterations while delivering measurable gains.

The RHI Algorithm

RHI turns the system harness into a cheap, iteratively refined component.

Full‑population harness search scales quadratically in both execution traces and pairwise evaluations, making it infeasible for many tasks. RHI solves this by restricting each iteration to a single new trace and a single comparison against the previous harness.

RHI treats the surrounding harness—agents, prompts, and orchestration—as a learnable module that is updated step‑by‑step using only the most recent output as a reference.

How does RHI differ from the earlier “Self‑Harness” approach?

Self‑Harness may evaluate multiple candidate edits and uses a verifier‑based pass‑count rule, whereas RHI restricts each iteration to a single pairwise comparison against the immediate predecessor and updates the harness directly with $L_{\text{harness}$}.

Iteration 1: Agent generates new outputs (b₁, b₂) using $H^{(1)}$.

Evaluator compares b₁ vs. a₁ and b₂ vs. a₂, yielding preferences P.$F^{(1)}$ = {b₁ ≻ a₁, b₂ ≺ a₂}.

History $D^{(1)}$ stores these two pairwise results.

Improvement rate s₁ = (1/2)·[1 + 0] = 0.5. If $\epsilon$ = 0.3, continue.

Optimizer updates $H^{(2)}$ = $L_{\text{harness}$}($H^{(1)}$, $D^{(1)}$), producing a revised harness that will favor the pattern “b₁ is better than a₁”.

Even with only one new execution per iteration, the accumulated preferences steer the harness toward consistently better outputs.

The workflow splits the harness into a central orchestrator that delegates work to subagents via explicit contracts, and a hop sequence that defines the order of those interactions.

Hop 1: Orchestrator sends the file snippet to S₁.

Hop 2: S₁ returns a draft implementation to the orchestrator.

Hop 3: Orchestrator forwards the function signature (contract of S₂) to S₂.

Hop 4: S₂ returns a refined signature back to the orchestrator.

Final output is assembled from S₁’s draft and S₂’s refined signature.

By limiting S₂’s contract to the signature, we avoid sending the entire draft implementation, saving attention compute while still improving the final result.

Core of Algorithm 1 – the RHI loop.

**Figure 2.** Recursive Harness Self-Improvement. ①, ②: At iteration $i$, the coding agent receives the task $x$ and the current harness $H^{(i)}$, solves the task, and produces an output `output[i]`. ③: An LLM evaluator then compares `output[i]` with the previous output `output[i-1]` and returns preference `P.F[i]`. ④: This feedback is stored in the self-comparison history. ⑤: Finally, an LLM harness optimizer uses this history to update the harness from $H^{(i)}$ to $H^{(i+1)}$.

**Figure 3 |** Decomposition of the RHI harness. We classify the harness into two primary components: agent design and agent workflow. Agent design specifies the roles and instructions of candidate agents. Agent workflow is further decomposed into contract and hop: contract defines the information exchanged between subagents and the orchestrator, while hop defines the interaction structure; overall workflow logics.

Experimental Results

RHI is benchmarked on 30 synthetic tasks across three domains, evaluated via LLM judges.

RHI is evaluated on a benchmark of 30 synthetic open‑ended ML research tasks.

The suite contains ten tasks each from quantitative finance, robotics, and pharmaceutical ML.

**Figure 4.** Structure of a prompt-represented harness $H^{(i)}$. The harness specifies candidate agents, the orchestrator–subagent workflow, and auxiliary control rules used for termination, fallback, recall, and inter-agent communication.

**Figure 5.** After two iterations, RHI raises the empirical ceiling of sonnet-4.6 test-time scaling. We compare sonnet-4.6-high+H[i] against sonnet-4.6-max, where H[i] denotes the harness after i RHI iterations. The left panel reports pairwise outcomes over 30 tasks; the right panel reports mean cost, output-token count, and cache read/write usage, normalized by sonnet-4.6-high. Results are averaged over two LLM judges and three seeds; error bars denote standard deviation.

**Figure 6.** After one iteration, RHI raises the empirical ceiling of opus-4.7 test-time scaling. The pairwise panels compare opus-4.7-high+H[0] and opus-4.7-high+H[1] against opus-4.7-high/xhigh/max over 30 tasks. The resource panel reports mean cost, output-token count, and cache read/write usage, normalized by opus-4.7-high. Results are averaged over six evaluator configurations and three seeds; error bars denote standard deviation.

**Figure 8.** RHI complements rather than replaces train-time scaling. We compare `sonnet-4.6-high`+$\mathcal{H}[i]$, $i \in \{0, 1, 2, 3, 4\}$, with stronger base-model baselines `opus-4.7-high` and `opus-4.7-xhigh`. RHI on the weaker `sonnet-4.6-high` base model does not consistently close the gap to the stronger `opus-4.7` baselines.

Experimental Settings and Metrics

RHI repeatedly outperforms stronger test‑time scaling baselines while cutting inference cost.

RHI repeatedly surpasses stronger test‑time scaling baselines, raising the performance ceiling across all three base models.

Sonnet‑4.6‑high + H[2] wins 20/30 (≈66 %) against the max baseline; Opus‑4.7‑high + H[1] beats both xhigh and max; Opus‑4.8‑high + H[2] beats xhigh, ultracode, and max.

These gains are not driven by longer generations. For Sonnet‑4.6 and Opus‑4.8 the normalized token count stays within a narrow band (≈1.7–1.9×), yet win rates increase markedly. Opus‑4.7 shows a modest token increase, but the comparison spans only two iterations, limiting inference about causality.

Optimizing a user‑constructed harness at the prompt level can outperform a provider‑built dynamic workflow. Opus‑4.8‑high + H[2] beats the ultracode variant, demonstrating that task‑specific prompt engineering can exceed built‑in multi‑agent orchestration.

**Figure.** opus-4.7-high/xhigh/max vs opus-4.7-high + $\mathcal{H}[0]$

**Figure.** opus-4.7-high/xhigh/max vs opus-4.7-high + $\mathcal{H}[1]$

**Figure 7.** After two iterations, RHI raises the empirical ceiling of opus-4.8 test-time scaling. The pairwise panels compare opus-4.8-high and opus-4.8-high+$\mathcal{H}[i]$, $i \in \{0, 1, 2\}$, against opus-4.8-xhigh/ultracode/max over 30 tasks. The resource panel reports mean cost, output-token count, and cache read/write usage, normalized by opus-4.8-high. Results are averaged over two LLM judges and three seeds; error bars denote standard deviation.

**Figure 9.** Evolution of full-harness representations across RHI iterations on sonnet-4.6-high. In figure labels, $\mathcal{H}[i]$ denotes the main-text notation $\mathcal{H}^{(i)}$, or $\mathcal{H}_x^{(i)}$ for task $x$. (c) Consecutive harness similarity. (d) Cumulative drift from $\mathcal{H}[0]$. (e) Within-domain across-task similarity.

Ablations and Component Analysis

RHI trims inference cost and reshapes harness components, revealing where gains come from.

Recursive Harness Self‑Improvement treats the surrounding harness—agents, prompts, and orchestration—as a learnable component that can be iteratively refined to boost performance and cut inference cost.

During RHI, individual harness parts (role, instruction, contract, hop) move away from their initial specifications at different speeds, a phenomenon we call component drift.

How does “component drift” differ from overall harness drift?

Overall drift treats the whole prompt as a single vector, masking which sub‑parts move. Component drift isolates each sub‑prompt (role, instruction, contract, hop) and measures its own change, revealing that contracts shift early while roles evolve more slowly.

RHI cuts inference cost by up to 60 % while still raising the performance ceiling of test‑time scaling baselines.

Measured cache read/write usage and wall‑clock time across multiple base models; the most aggressive RHI run achieved a 60 % reduction.

The first RHI iteration drops the full‑harness cosine similarity to ≈ 0.82, whereas later updates stay above 0.97, showing that the biggest semantic shift occurs at iteration 1.

Cosine similarity between embeddings of H(0) and H(1) is 0.82; similarity between H(i) and H(i+1) for i≥1 is ≥ 0.97.

Contracts’ consecutive cosine similarity rises from 0.48 to 0.72 across iterations, indicating rapid early stabilization of this component.

Average pairwise cosine similarity of contract embeddings per iteration follows 0.48 → 0.66 → 0.69 → 0.72.

Roles evolve more gradually, with consecutive similarity increasing from 0.28 to 0.33, suggesting slower drift.

Role embeddings show similarity 0.28 → 0.31 → 0.33 → 0.31 across the four RHI updates.

Performance gains plateau after 2 to 4 RHI iterations, so additional updates yield diminishing returns.

**Figure a.** Initial harness $\mathcal{H}[0]$ versus RHI-improved harnesses $\mathcal{H}[i], i \in \{1, 2, 3, 4\}$.

**Figure b.** Harness representations by RHI iteration $\mathcal{H}[i]$, $i \in \{0, 1, 2, 3, 4\}$.

**Figure 10.** Evolution of harness-component representations across RHI iterations on sonnet-4.6-high. Each subfigure shows low-dimensional projections of role, instruction, contract, and hop embeddings within a domain; figure labels use $\mathcal{H}[i]$ for iteration $i$.

**Figure.** (a) Consecutive component similarity. (b) Cumulative component drift from iteration 0. (c) Across-task component similarity.

**Figure 1.** (a) Harness updates are shaped by a controllable external factor $f_{ext}$ and a model-dependent internal factor $f_{int}$. (b) We interpret $f_{int}$ as functional specialization guidance, which suppresses redundant overlap and encourages harness components to specialize into distinct functional modules.

**Table 2.** Task mutual information $I(\text{hc}; \text{task})$ in nats for four harness components across four evaluation configurations: two embedding models and two bias treatments. Arrows indicate the direction of change for RHI iteration $i = 1$ to $i = 4$.

**Table 3.** Total correlation among the four harness components (role, instr, cont, hop) in nats across four RHI iterations and four configurations: two embedding models and two bias treatments. TC is the unconditional dependence, while TC | task is the task-conditional estimate obtained by per-task centering. Arrows indicate the direction of change from i = 1 to i = 4.

Related Work

We situate RHI among prior work on recursive self‑improvement, harness optimization, agent design, and prompt engineering.

RHI studies whether the system layer around a fixed foundation model can be improved in a multi‑agent LLM setting. It does not train a stronger model nor allocate more test‑time reasoning; instead it rewrites the prompt‑represented harness that defines agent roles, instructions, workflow hops, and communication contracts.

Recursive self‑improvement (RSI) spans from formal self‑rewriting systems like Gödel Machines to practical self‑feedback loops. Prior LLM work such as STOP, Darwin Gödel Machine, STaR, Self‑Rewarding Language Models, Voyager, SiriuS, Self‑Refine, and Reflexion all embody iterative updates of reusable components. RHI treats this iteration specifically at the harness level: the base model, evaluator, and optimizer stay fixed while the textual harness is rewritten from self‑comparison history.

A harness is the structured system surrounding a foundation model—prompts, tool‑use loops, routing, memory, verification, governance, and control flow. Works like Meta‑Harness, AutoHarness, Self‑Harness, TTHE, Continual‑Harness, Adaptive Auto‑Harness, HarnessX, Agentic Harness Engineering, and Workspace Optimization search directly over such scaffolds. RHI shares their premise that capability can improve by changing the harness rather than model weights, but differs by operating on a prompt‑represented multi‑agent harness and using pairwise self‑comparison instead of large candidate populations.

Agent‑design and workflow search treat agents and their coordination graphs as optimization variables. ADAS, GPTSwarm, AFlow, AgentSquare, EvoAgentX, SEW, EvoFlow, AlphaEvolve, ShinkaEvolve, and Ornith‑1.0 all require executable workflows or code‑level control. By contrast, RHI limits the editable object to a textual harness injected into a black‑box multi‑agent coding agent.

Prompt and pipeline optimization also relate to RHI. OPRO, TextGrad, DSPy, GEPA, and `optimize_anything` treat prompts or LM programs as tunable text parameters. RHI’s object differs: a multi‑agent harness containing roles, instructions, hops, and contracts. Ablations show that gains stem mainly from workflow and contract components rather than longer single‑agent prompts.

Multi‑agent LLM systems can boost reasoning via debate, specialization, and tool use, but such gains may simply reflect increased test‑time computation. To isolate coordination benefits, we compare RHI not only to a default harness but also to stronger same‑family test‑time scaling baselines and a built‑in multi‑agent coding harness, measuring tokens, cost, and cache activity.

Qualitative Examples

Multi‑agent RHI delivers higher Elo at lower cost while the harness converges steadily.

Across the four execution settings, the multi‑agent configuration that incorporates an explicit harness (multi(ours)) consistently attains higher Elo scores while using less normalized compute than both the built‑in multi‑agent baseline and the single‑agent union baseline.

**Figure 13.** Normalized cost versus Elo for the four execution settings under two base models. The x-axis reports mean cost over the 30 tasks, normalized per task by the corresponding `single(default)` run; the y-axis reports Elo from LLM-as-a-judge pairwise comparisons. Colors indicate evaluator configurations, markers indicate execution settings, and error bars denote standard deviation across evaluator seeds.

RHI learning is stable: cosine similarity between successive harness texts $H_i$ and $H_{i+1}$ rises toward 1, and the same trend holds for each component (role, instruction, contract, hop), indicating convergence rather than oscillation.

The built‑in multi‑agent harness (multi(default)) yields lower Elo than the single‑agent baseline (single(default)) despite higher normalized cost on both Opus‑4.7‑high and Sonnet‑4.6‑high, demonstrating that default coordination adds compute without benefit.

Providing an explicit harness $H(0)$ (multi(ours)) reverses this trend: it achieves higher Elo at lower or comparable cost than multi(default), and it also outperforms the single‑agent union baseline (single(unionOur)), confirming that coordinated multi‑agent execution contributes additional gains beyond prompt‑level role specification.

The four settings are: (1) single(default) – task prompt only; (2) multi(default) – task plus generic “create an agent team” instruction; (3) multi(ours) – task, built‑in multi‑agent capability, and the initial harness $H(0)$; (4) single(unionOur) – task with the role and instruction from $H(0)$ but executed by a single agent.

Appendix B shows the harness evolution from $H_0$, which lists a full suite of specialist agents, to $H_1$, which refines the orchestrator’s contract, adds explicit round‑robin delegation, and embeds detailed issue‑tracking fields, illustrating how RHI iteratively enriches the coordination structure.

Orchestration Protocols

Defines the contract language and recall workflow that bind all agents together.

The orchestrator issues a top‑level OrchestrationState that enumerates every required artifact, exact file path, and contract version before any code is written.

Each specialist publishes a draft handoff (e.g., DatasetRecord, ModelIOContract) which the steward routes to all consumers; a consumer may return accept or a change request that blocks progress until the issue is resolved.

When a disagreement surfaces, the orchestrator records a ConflictRecord and adopts the scientifically conservative decision, annotating the limitation in the final report and the log file.

```python { 'agent_id': 'agent_baseline_ablation', 'role': 'Invariant baseline, equivariant variants, and fair-comparison ablation specialist.', 'instruction': 'Define the controlled baseline and ablation grid. The minimum grid should include one invariant distance-only baseline and at least two equivariant rows, such as full GVP and a smaller or feature-ablated GVP variant, unless compute limits are explicitly approved by the orchestrator as a deviation. Keep comparisons fair in depth, train budget, split, optimizer, and parameter scale. Define results/ablation_results.json so downstream report and plot agents can consume it without guessing.', 'output_to_orchestrator_schema': 'Handoff plus: baseline_contract; fairness_controls; ablation_grid with model_id, variant_description, expected_files, train_budget, parameter_budget; ablation_results_json_schema; comparison_plot_spec for deliverables/plots/ablation_equivariant_vs_invariant.png; validation_checks for same split and same metric implementation; recall_triggers if variants are missing.' }, { 'agent_id': 'agent_training_reproducibility', 'role': 'Training loop, reproducibility, configuration, checkpoints, and uv workflow specialist.', 'instruction': 'Implement and validate src/train.py plus main.py integration. Use uv run commands, fixed seeds, deterministic split consumption, train/val/test separation, checkpointing, training_history JSON files, device fallback, and short smoke-run options. Coordinate with model and ablation agents to train all required model_ids. Ensure failures such as NaNs, missing data, shape mismatch, or checkpoint incompatibility produce actionable errors. Update logs.txt with exact commands and wall-clock summaries.', 'output_to_orchestrator_schema': 'Handoff plus: training_config_schema; seed_and_determinism_plan; split_consumption_plan; checkpoint_schema; training_history_schema; command_plan_full_and_smoke; failure_debug_playbook; produced_artifact_list; runtime_estimates; validation_results from uv run commands.' }, { 'agent_id': 'agent_eval_benchmarks', 'role': 'Circular metric, benchmark, per-residue analysis, and plot-data specialist.', 'instruction': 'Implement and validate src/evaluate.py and metric-producing utilities. Define circular absolute error, MAE, median, RMSE, per-residue breakdowns with counts for every evaluated model_id, and optional bootstrap uncertainty if time allows. Ensure results/metrics.json and results/ablation_results.json contain enough structured data for report, index, and plots. Cross-check that plots and report numbers are sourced from JSON rather than retyped. Coordinate with data on splits and with scicomm on claims.', 'output_to_orchestrator_schema': 'Handoff plus: metric_formulae; circular_error_algorithm; metrics_json_schema; per_residue_schema_all_models; test_index_schema; plot_data_requirements; number_source_map mapping report claims to JSON keys; statistical_notes; validation_commands; issues_for_training_or_report if values are inconsistent.' }, { 'agent_id': 'agent_profile_systems', 'role': 'Runtime, memory, environment, and profiling methodology specialist.', 'instruction': 'Implement and validate src/profile.py. Profile forward and training-step latency over batch sizes for all model_ids or the main baseline pair. Capture environment details, parameter counts, device, threads, and profiling methodology. For memory, do not rely only on tracemalloc if PyTorch native allocations are missed; include at least one robust measure such as psutil RSS delta, torch profiler profile_memory, CUDA max_memory_allocated when available, and parameter-memory estimates. If memory values are very small, explain the method limitation and provide a more meaningful estimate. Produce data for deliverables/plots/runtime_memory_profile.png and structured metrics for results/metrics.json.' } ```

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers