Harness2: Recursive Agent Harnessing for an Open World

Ruiyao Xu, Yanfei Chen, Zhongying CuiZhu, Bhavana Dalvi Mishra, Yifei Ming, Han Yu, Rujun Han, Chen-Yu Lee, Tomas Pfister

Harness2 uses recursive self-improvement to refine both an agent's execution harness and the mechanism that improves it, without ground-truth labels.

How can LLM agents automatically improve their own execution harnesses through recursive self-improvement in open-world environments?

LLM agents rely on "harnesses"—the prompts, tools, and memory structures surrounding the model—to perform tasks. In open-world settings, these harnesses often fail because they are designed for static environments and lack reliable feedback to guide improvements when tasks are novel. Harness2 addresses this by coupling two levels of recursion: task-level recursion refines the current execution harness using a contrastive "Propose-Probe-Compose" loop, while harnessing-level recursion extracts lessons from these edits to update a continual improvement harness. This framework consistently improves performance across three professional benchmarks, yielding gains of up to 11.5 points on WorkBuddy-Bench and 11.8 percentage points on legal tasks.

Paper Primer

The core mechanism is a contrastive improvement step that avoids the need for ground-truth verifiers. It proposes candidate harness edits, probes their behavior by executing them under matched conditions, and composes the final harness by merging only those changes supported by observable behavioral contrasts.

Harnessing-level recursion turns the refinement process itself into a learning signal. By retaining adopted harnesses and "experience" records—which document why specific changes worked—the system builds a library that guides future proposals, allowing the agent to get better at improving itself as it encounters more tasks.

Harness2 provides consistent performance gains across diverse models and professional domains.

Evaluations on WorkBuddy-Bench, JobBench, and the Legal Agent Benchmark (LAB) using GLM-5 and Gemini 3.5 Flash. Up to 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench, and 11.8 percentage points on LAB partial pass rates.

Task-level recursion budget scales performance monotonically.

Increasing the recursion budget ($k$) from 1 to 5 consistently improved scores on both code and office tasks, outperforming static baselines. Parallel recursion at $k=5$ added up to 3.9 points on WorkBuddy-Bench and 6.7 percentage points on LAB over the single-step baseline.

Why is this approach better than simply using a stronger model?

The paper demonstrates that a weaker model (GLM-5) equipped with an improved harness can outperform a stronger model (Gemini 3.5 Flash) using its original, base harness, suggesting that harness optimization is a distinct and powerful lever for agent capability.

What is the primary limitation of this framework?

The framework relies on the assumption that behavioral contrasts—differences in execution trajectories—provide sufficient evidence to guide harness edits in the absence of explicit ground-truth labels or validation feedback.

Recursive harnessing transforms task-specific refinement into a cumulative process, enabling agents to learn how to improve their own scaffolding as they operate in unpredictable, open-world environments.

The Challenge of Open-World Harnessing

We expose how static execution harnesses waste compute and motivate Harness2’s recursive self-improvement.

Training large models wastes compute on static execution harnesses that cannot adapt to new tasks. In open‑world settings, tasks appear unseen and provide no ground‑truth verifiers, so harness edits lack reliable feedback. Moreover, recursive self‑improvement (RSI) is highly task‑specific and does not generalize, creating a bottleneck for dynamic environments.

An execution harness is the collection of prompts, memory, tools, and orchestration that wrap a frozen LLM to turn it into an autonomous agent; it determines how the model interacts with its environment.

Static harnesses are the primary bottleneck in dynamic, open‑world environments.

The Harness2 Framework

Method introduces recursive harness updates that adapt execution components per task.

Static execution harnesses waste compute; we need a mechanism that can adapt the harness itself during test‑time.

It treats the harness—the set of editable components surrounding a frozen model—as a mutable layer that can be refined without touching model weights.

How does recursive agent harnessing differ from traditional self‑improvement of model parameters?

Traditional self‑improvement rewrites the model’s weights; recursive agent harnessing only edits the surrounding harness files, leaving the frozen model untouched. This makes the improvement applicable to any frozen model and avoids costly gradient computation.

For the current task the system proposes harness edits, probes their effect, and composes a successor harness based on observed behavioral contrasts.

Reference execution e₀ is obtained by running the agent with the initial harness H = (c₁, c₂).

Propose creates two candidates: (i) an invariant edit that adds a “planning reminder” to c₁, (ii) a domain edit that injects a short summary of the article into c₂.

Probe runs each candidate harness on the same task, producing executions $e_i$ and $e_d$ with trajectories $\tau_{i}$, $\tau_{d}$.

Compose compares $\tau_{i}$ and $\tau_{d}$ against $\tau$₀; it keeps the domain edit because it eliminates a dead‑end in $\tau_{d}$, discarding the invariant edit.

The resulting successor harness H′ contains the updated memory component only, and its execution e′ shows the fact retrieved correctly.

The step demonstrates that concrete behavioral contrasts—not a learned reward—drive the selection of harness edits.

After each task the improvement harness’s library is updated with the adopted harness and the lessons learned, so future tasks start with richer reusable knowledge.

**Figure 2.** The eight harness components in HARNESS$^2$, grouped by their role in the runtime.

**Figure 3 |** The HARNESS$^2$ pipeline. Task-level recursion proposes invariant and domain-specific harness edits, probes their effects, and composes a successor execution harness. After each task, harnessing-level recursion updates the improvement harness with reusable harnesses and lessons from the refinement process.

**Figure 4.** Two forms of task-level recursion. (a) Parallel recursion probes multiple candidates and independently composes successor harnesses. (b) Sequential recursion carries each refined harness and accumulated execution evidence into the next step.

**Figure 7 | Harnessing-level recursion.** Left: direct refinement with the improvement harness. Right: JobBench-to-WorkBuddy-Bench library transfer for one-step refinement.

Performance and Benchmarks

Experimental results showing Harness² consistently outperforms baselines across domains.

HARNESS² outperforms all baselines in every domain, achieving up to +9.4 points over the base harness.

Figure 1 left panel shows gains of +7.4 (Code), +8.4 (Web), and +9.4 (Office) over the base harness.

We evaluate three professional‑domain benchmarks—WorkBuddy‑Bench (Code, Web, Office), JobBench (end‑to‑end job tasks), and LAB (legal‑work partial pass). Baselines include the unchanged base harness, Reasoning‑Bank (fixed‑harness retrieval), Meta‑Harness (offline search on a training split), and Harness Scaling (online per‑instance revision).

Executors are Gemini 3.5 Flash and GLM‑5, run under OpenCode and Codex harnesses with Claude Opus 4.8 as the default editor. All configurations share the same improvement step; “+ Recursion” denotes parallel task‑level recursion with budget k = 5.

Meta‑Harness searches a training split for a better harness code offline and deploys the best version across all test instances.

How does Meta‑Harness differ from Harness Scaling?

Meta‑Harness performs an offline search on a training split and reuses the best harness for all test tasks, whereas Harness Scaling revises the harness online for each evaluation instance using extra inference budget, without carrying state across instances.

**Figure 1 | Overall performance.** *Left:* WorkBuddy-Bench by domain for Gemini 3.5 Flash under OpenCode. HARNESS$^2$ with parallel recursion at $k=5$ exceeds the strongest baseline in every domain, Meta-Harness, and the number above each HARNESS$^2$ bar is its gain over the base harness. *Right:* on WB-code, HARNESS$^2$ keeps improving as the budget $k$ grows.

Harness² consistently outperforms Meta‑Harness across all domains.

Ablation and Qualitative Analysis

Ablations quantify how each Harness2 component contributes to performance.

We evaluate Harness² along three axes: recursion depth, compute‑performance trade‑offs, and component ablations.

Parallel recursion reaches 80.8 on the Code benchmark at k = 5, up from 76.5 at k = 1.

Parallel recursion rises from 76.5 to 80.8 on Code as k increases from 1 to 5.

Parallel recursion reaches 83.4 on the Office benchmark at k = 5, up from 79.5 at k = 1.

Parallel recursion rises from 79.5 to 83.4 on Office as k increases from 1 to 5.

Increasing k from 1 to 5 raises the performance gain on JobBench from 4.5 to 7.5 points.

With Gemini 3.5 Flash, gains over the base grow from 4.5 to 7.5 points on JobBench as k increases.

Increasing k from 1 to 5 raises the performance gain on WorkBuddy‑Bench from 5.1 to 8.4 points.

With Gemini 3.5 Flash, gains over the base grow from 5.1 to 8.4 points on WorkBuddy‑Bench as k increases.

Ablation of three components—library, scoped editor roles, and composition—shows lower scores than the full method (77.5 / 36.0), confirming each component’s contribution (see Table 3).

**Figure 9 | Case study on JobBench.** Left: the task and the two failure modes from the base trajectory. Middle: the improved harness; tinted components were written at test time, grey ones are inherited from the base. Right: the outcome of re-running the composed harness.

**Figure 10.** Library provenance and the distribution of edits. Left: modified components compared with the library snapshot available during refinement. Containment measures the share of non-boilerplate lines found in a library entry for the same component. Lines common to more than a fifth of a benchmark’s adopted harnesses are excluded. Right: edit distribution across components.

Related Work

Related work situates our harness improvements among prior RSI and optimization studies.

Prior work on open‑world execution harnesses falls into three strands: component ablations, recursive harness refinement, and broader RSI systems. We briefly map each strand to the literature before positioning our contribution.

Ablation experiments on WorkBuddy‑Bench and JobBench reveal that removing the library drops scores by 3.1 pts (WorkBuddy) and 1.8 pts (JobBench); removing scoping costs 2.9 pts and 3.7 pts; and removing composition costs 1.6 pts and 1.9 pts. These three components jointly sustain performance.

Harnessing‑Level Recursion studies show that directly refining the base execution harness without reference executions degrades Code from 73.4 to 63.2 and Web from 69.9 to 65.0. A library built at $k\!=\!1$ restores baseline performance, while $k\!=\!5$ yields gains of +0.9, +4.4, and +5.1 pts on Code, Web, and Office respectively. Greater task‑level exploration thus amplifies downstream refinement.

Recursive Self‑Improvement (RSI) envisions agents that iteratively upgrade themselves, making progress cumulative. Early systems refined outputs via self‑feedback (e.g., self‑rewarding language models) or retained reflections for later attempts; later works such as STOP, Darwin Gödel Machine, Hyperagents, and MetaSkill‑Evolve extend RSI to program and harness evolution.

Automatic harness optimization has progressed from prompt and skill tuning to full executable harness synthesis. Offline searches over reusable tasks, task‑rewarded editor training, and diagnostic experience collection precede online test‑time adaptation. Recent methods (e.g., HCL, HSI) retain improvement state across tasks; our approach uniquely combines contrastive task‑level refinement with a single‑pass, ground‑truth‑free deployment stream.

**Figure 8.** Same model as executor and editor. Base harness versus HARNESS$^2$ with OpenCode at k=2. Annotations show score gains.

**Table 4.** Comparison of harness improvement methods. *Both* denotes offline preparation and test-time adaptation. *Label* indicates external correctness supervision; *Streaming*, one-pass task or batch updates; *Scaling*, a test-time scaling study; and *Improver Update*, updates to editing guidance or editor parameters beyond changes to the execution artifact. Appendix A.1 defines the criteria.

Failure Analysis

We outline the remaining shortcomings of Harness 2 and sketch avenues for future work.

The central claim of the paper is that static execution harnesses waste compute, and Harness 2 mitigates this by recursively improving the harness at test time. Below we examine where this promise falls short.

E.1 shows that individual harness components each contribute modest gains, but their interactions sometimes produce diminishing returns. The ablation indicates that the most expensive component—recursive self‑improvement—offers the bulk of the improvement.

E.2 evaluates reusing a learned harness across multiple tasks and finds that performance plateaus after a few reuse cycles. The improvement experience therefore depends on the diversity of tasks seen during training.

E.3 tests transfer to executor models that were not seen during harness training and reports a noticeable drop in adaptation speed. This suggests that the recursive updates rely on model‑specific inductive biases.

E.4 investigates tool‑use reliability and discovers that failures in external tool invocation propagate through the recursive loop, amplifying errors. Robustness to tool failures remains an open challenge.

E.5 compares Harness 2 against production‑grade harnesses used in deployed systems and finds that, while competitive on benchmark metrics, it lags on latency and stability under heavy load. The gap is partly due to missing engineering optimizations.

E.6 presents a systematic failure analysis, highlighting three families of breakdowns that the current recursion cannot resolve.

**Figure 14.** Failure cases in recursive harnessing. Nine results below their references, grouped by candidate selection, restrictive edits, and other execution or evaluation effects. The scores are retrospective evaluations and are not available to the pipeline.

E.7 offers qualitative case studies that illustrate both successful adaptations and the pathological scenarios shown in Figure 14. These examples underscore that the method excels when the search space contains clear gradients but struggles with flat or deceptive landscapes.

Limitations: the approach assumes access to retrospective evaluation scores, which are unavailable at test time, and it relies on a relatively simple recursion that cannot escape certain local optima. Moreover, the method has been validated only on a narrow set of executor architectures.

Future Directions: extending the recursion to incorporate uncertainty estimates, broadening the set of executor models, and integrating more sophisticated tool‑failure handling are promising paths. Scaling the method to production‑grade latency constraints will also be essential for real‑world deployment.

Extended Empirical Analysis

Ablation, reuse, transfer, and reliability analyses dissect the harness’s component contributions.

The ablation study removes prompt‑related components, skills, or scripts from the improved harness and measures the resulting performance drop.

The table presents an ablation study of the HARNESS² framework across three domains: Code, Web, and Office, with an average score provided for each configuration. The configurations compared are the "Base harness," the full "HARNESS² (full)" model, and three variants with specific components removed: "w/o prompt*," "w/o skills," and "w/o scripts."

**Figure 11 | Leave-one-out reuse.** Each task is re-run with its own entry excluded from the completed library. Annotations show gains over the original online run at $k=1$.

**Figure 12.** Cross-model transfer on WorkBuddy-Bench. The harness refined for Gemini 3.5 Flash under OpenCode is applied unchanged to GLM-5, Gemini 3.7 Flash, and Claude Opus 4.8, and compared with each executor's base harness. Annotations show the score gain.

**Table 7.** Tool-call frequency and error rates before and after one improvement step.

**Figure 13.** Comparison with production harnesses on WorkBuddy-Bench. Gemini 3.5 Flash under CodeBuddy, OpenCode, Codex, and Claude Code, each left unchanged, versus HARNESS^2 starting from OpenCode at k=5. Annotations give the gain over the strongest fixed harness per domain.

**Figure 15 | WorkBuddy-Bench, Code.** The reference emits calls as plain text without running tools. A tool-guidance edit forbidding pseudo-calls raises the score from 60.0 to 100.0.

**Figure 16 | WorkBuddy-Bench, Office.** The reference aggregates at the wrong grain and no channel row matches the key; a skill pins the grain and the derived fields, and a guardrail treats inputs as read-only and applies the stated cleaning rules before aggregating (23.6 → 100.0).

**Figure 17 | WorkBuddy-Bench, Web.** Page state is frozen at first render, so remapping and export go stale; a skill models the state the specification names and re-derives it on every change, a script checks the artifact opens offline, and a guardrail forbids network dependencies (64.0 → 100.0).

**Figure 18.** **LAB, arbitration.** Markup is left in Word's track-changes layer, so read as text the document shows only the accepted wording; a skill records deletions and insertions as in-body character markup, and a guardrail loads it for redline tasks (44.4 → 94.5).

**Figure 19.** JobBench, social-science research assistant, all eight components written. With Claude Opus 5 as executor and editor, research expands to fill the budget and files stay unwritten near the deadline; the composed harness adds, among other edits, a plugin that appends a warning to tool results after a fixed number of calls without an artifact, a subagent that returns citable tables in its own context, and durable domain memory (72.0 → 100.0).

Conclusion and Framework Summary

Conclusion recap: Harness2 refines execution harnesses via recursive self‑improvement.

Recall that training large models wastes compute on static execution harnesses; Harness2 applies recursive self‑improvement to evolve harnesses at test time.

Task‑level recursion refines behavior via contrastive improvement, while harnessing‑level recursion incorporates the resulting harnesses and their experience. Empirically, the framework yields consistent gains across three benchmarks, two executors, and two initial harnesses, and ablations confirm the importance of scoped proposals, composition, and accumulated experience. Together, these results validate recursive harnessing as a general strategy for test‑time adaptation in open‑world tasks.

Implementation Details

Implementation details and baselines for the appendix.

The baseline suite comprises the original execution harness, a static‑harness variant, and a hand‑tuned version that mirrors the final system without recursive self‑improvement. These baselines serve to isolate the contribution of the recursive harness evolution.

Executors are lightweight runtime wrappers that launch a harness in an isolated process; the initial harness is a minimal stub that loads only the core inference engine and no task‑specific logic. This separation lets the system replace the stub with a more capable harness during improvement sessions.

Improvement sessions are iterative phases where the system evaluates its current harness on a set of open‑world tasks, records performance signals, and triggers a library reconfiguration that swaps in newly generated components. The library configuration tracks which component versions are active and ensures compatibility across sessions.

Editable harness components are modular code blocks—such as task selectors, memory managers, and output formatters—that can be rewritten by the recursive self‑improvement process. Each component exposes a simple interface, allowing the system to substitute a freshly generated implementation without restarting the entire process.

Comparison of Improvement Methods

Details of the executor, harness components, and improvement session setup.

We first lay out the six comparison dimensions that distinguish harness‑improvement approaches.

Benchmarks span three domains: WorkBuddy‑Bench (code‑centric tasks), JobBench (professional deliverables), and LAB (legal reasoning), each evaluated on held‑out test streams.

Baselines include Best‑of k (multiple independent runs), ReasoningBank (memory‑augmented prompting), Harness Scaling (iterative verifier‑blind revision), and Meta‑Harness (offline optimization with a fixed library).

The executor is a frozen coding model wrapped by an off‑the‑shelf agent; the initial harness consists of four always‑on files (system prompt, guardrails, memory, tool guidance) plus on‑demand components.

Improvement sessions run under Claude Opus 4.8 as editor and selector, receive evidence as files, reject any directory containing a grader verdict, and retry once on invalid edits before falling back to the prior tree.

**Table.** Components of the system architecture, categorized by Group, Component, Activation, and Definition.

Overall, these implementation choices expose a fine‑grained component tree that the improvement harness can edit, while the executor remains immutable, enabling systematic test‑time refinement without altering model weights.

Prompting Strategies

Meta instructions for sequential harness composition, terminal selection, and experience reflection.

Prompt 4 defines a sequential adaptation loop where each round composes a new checkpoint from the full history of rollouts and candidates.

Sequential harness composition – choose a checkpoint for the next round.

Prompt 5 instructs the evaluator to pick a final answer by applying a strict hierarchy of criteria, ignoring any impression of sophistication.

Decision hierarchy for blind selection of n candidate artifacts.

Prompt 6 requires a systematic diff of each candidate harness against the base, recording the edit, observed effect, and a verdict.

Template for the Experience Reflector output.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers