Scaling Trajectories for Complex Tasks through Recursive Self-Rewrite
Zongxia Li, Yucheng Shi, Zhongzhi Li, Junyao Yang, Ruhan Wang, Chengsong Huang, Fuxiao Liu, Haitao Mi, Jordan Boyd-Graber, LeoweiLiang
Recursive Self-Rewrite (RSR) turns harness-assisted successes into reusable model capabilities by reconstructing them for a general-purpose harness.
How can we improve a model's performance on complex tasks by using diverse, specialized "harnesses" to discover successful trajectories and then rewriting them into a unified format for training?
Models often struggle to solve complex, long-horizon tasks without specialized execution support, yet training directly on trajectories from these diverse "harnesses" introduces harness-specific conventions that fail under a general-purpose interface. Recursive Self-Rewrite (RSR) solves this by using a base model to discover successful solutions across multiple harnesses, then reconstructing those experiences into clean, verified demonstrations compatible with a single general harness. This approach significantly outperforms direct supervised fine-tuning, increasing pass rates across multiple terminal-task benchmarks and improving process rewards on long-horizon problems.
Paper Primer
The RSR framework operates in three stages: discovery, reconstruction, and execution. First, the model uses diverse harnesses—each providing different levels of state management, recovery, and validation—to solve a broad pool of tasks. Second, a planner-critic-executor pipeline rewrites these successful trajectories into "runbooks" that capture the core logic while stripping away harness-specific control artifacts. Finally, the model re-executes these tasks in a fresh sandbox under a general-purpose harness, keeping only the verified, successful trajectories for fine-tuning.
RSR significantly improves model performance on complex terminal tasks compared to direct supervised fine-tuning (SFT).
On Terminal-Bench 2, RSR increased pass@3 from 53.4% (Direct SFT) to 74.2%, and on Terminal-Bench Hard, it improved from 56.0% to 63.0%. RSR consistently yields double-digit percentage point gains in pass@3 across most tested benchmarks.
Diverse harnesses provide complementary problem-solving capabilities, expanding the total number of tasks a model can solve.
The union of three harnesses solved 759 unique tasks, a 34.3% increase over the strongest individual harness in the pool. The union of harnesses solved 194 additional tasks that no single harness could solve alone.
Why not just train the model on the successful trajectories collected from the specialized harnesses?
Directly pooling trajectories from different harnesses introduces "distribution mismatch," where the model learns harness-specific prompts, workflow logic, and stopping rules that are unavailable during general-purpose inference. RSR isolates the underlying problem-solving behaviors—like planning and recovery—and rewrites them into a format compatible with the target general harness.
What is the role of the "critic" in the rewriting process?
The critic acts as a filter to prevent "answer leakage" and ensure the runbooks provide useful procedural guidance rather than just the final solution. It screens candidate runbooks to ensure they contain only valid, transferable strategies before the executor attempts to re-solve the task.
Introduction and Motivation
Models lack diverse successful trajectories, limiting their ability to solve complex tasks.
Models struggle on complex tasks because they rarely encounter diverse, high‑quality successful trajectories. Different harnesses can coax the same model into solving complementary subsets of tasks, but the resulting trajectories embed harness‑specific control logic that cannot be reused under a general harness. Consequently, the lack of reusable, diverse successes becomes a bottleneck for self‑improvement.
Models lack a rich pool of diverse, high‑quality successful trajectories, which limits their capacity to solve complex, varied tasks.
The primary bottleneck for complex tasks is the scarcity of diverse, high‑quality successful trajectories.
The Recursive Self-Rewrite Framework
Transforms diverse successful harness trajectories into unified training data via a planner‑critic‑executor loop.
RSR turns isolated successes found under specialized harnesses into reusable demonstrations for a single general‑purpose harness.
Planner builds a runbook that lists the required actions: “perform $a_1$, then $a_2$, then $a_3$” (from A) and “perform $b_1$, then $b_2$” (from B).
Critic checks each runbook for prohibited tokens (e.g., direct answer strings) and rejects any runbook that contains them.
Executor receives the approved runbook, re‑executes the task in a fresh sandbox, and produces a new trajectory $[c_1, c_2, c_3]$ that respects the general harness.
Trajectory filtering removes any hidden answer fragments that appear in $[c_1, c_2, c_3]$ but are not derivable from the task or observations.
The cleaned trajectory is stored as a training example for the next RSR iteration.
Even though the original successes came from different harnesses, the rewritten trajectories share a common format, allowing the model to learn a single, harness‑agnostic policy.
How does RSR differ from ordinary supervised fine‑tuning (SFT) on the same task?
Standard SFT trains on whatever demonstrations are provided, which may still contain harness‑specific cues or hidden answer fragments. RSR first rewrites each demonstration into a harness‑agnostic runbook, filters out any leakage, and then re‑executes the task under the general harness. The result is a clean, self‑generated trajectory that the model must solve without relying on the original harness signals.
Multi‑Harness Discovery expands the pool of solvable tasks by letting the same base model interact with several specialized harnesses, each offering a different form of control or state management.
Trajectory Rewriting converts those heterogeneous successes into a uniform learning signal: a planner drafts a runbook, a critic sanitizes it, and an executor re‑creates the solution under a single, general‑purpose harness.
Run the base model under multiple discovery harnesses and collect successful trajectories.
For each trajectory, the planner compacts it into a runbook that omits harness‑specific messages.
The critic validates the runbook, discarding any that leak the final answer or reference unsupported tools.
The executor follows the vetted runbook in a fresh sandbox under the general harness, producing a new trajectory.
Apply trajectory filtering to remove any hidden answer fragments, then add the cleaned trajectory to the training set.
Repeat the loop, allowing the model to improve its policy without ever seeing the original specialized harnesses.
**Recursive planning turns successful experience into new practice.** Diverse harnesses discover source successes. Planner–critic feedback refines runbooks within a retry budget; each screened runbook guides fresh executions under a general harness. Rewritten trajectories are verified and filtered before training. Commands and draft numbers are schematic examples.
**Figure 1** Turning multi-harness successes into learning experiences with Recursive Self-Rewrite. Different discovery harnesses enable the same model to solve more difficult tasks. Recursive Self-Rewrite rewrites these harness-assisted solutions into demonstrations under a general harness, allowing the model to learn from successful experiences that a single harness alone would not produce.
Discovery Harnesses
Three specialized harnesses generate diverse terminal trajectories for training.
To obtain a rich set of successful executions we run the same language model behind three distinct harnesses, each imposing a different interaction protocol with the terminal.
A discovery harness is a thin wrapper around a terminal that defines how the model issues commands, observes results, and decides when to stop or continue, thereby turning raw execution into a structured trajectory.
Why does adding a harness change the data if the model stays the same?
The harness determines what actions are allowed, what feedback is visible, and when the episode terminates. Changing any of those alters the sequence of states the model experiences, producing fundamentally different trajectories even with an identical policy.
Step 1: Model issues
Step 2: Model issues
Step 3: Model issues
Step 4: Model signals
This short run shows how the same model can produce a complete, successful trajectory simply by following the act‑observe‑finish protocol defined by Terminus 2.
Terminus 2 runs a minimal act‑observe loop: the model sends a command, reads the terminal output, and repeats until it emits a special finish token.
How does Terminus 2 differ from a raw command‑line interface?
Terminus 2 adds a structured finish signal and a uniform observation API, whereas a raw shell merely streams text without an explicit termination marker, making trajectory boundaries ambiguous.
StateM augments the loop with persistent workflow state and checked transitions, allowing the model to resume after interruptions and enforce intermediate conditions.
Why not store the workflow state directly in the model’s prompt?
Embedding state in the prompt forces the model to re‑process the entire history each step and makes it hard to enforce invariants; StateM externalizes state so the harness can verify and recover it efficiently.
RSRT builds on Terminus 2 by adding a verifier‑driven reflection loop: after a failed attempt the model sees only a pass/fail flag and is asked to debug its prior trajectory before trying again.
Does RSRT hand the model the correct solution after a failure?
No—the model receives only a binary pass/fail flag, so it must infer what went wrong and adjust its behavior without ever seeing the ground‑truth answer.
**Figure 2** Three harnesses represent different logics of solving problems. Terminus 2 supports a general model–terminal interaction loop. StateM adds persistent workflow state and checked transitions. Recursive Self-Reflect Terminus uses verifier outcomes to trigger reflection and continued execution, stopping on success or timeout.
Multi-Harness Discovery and Rollouts
Combining three harnesses yields dramatically more solved tasks.
The union of the three harnesses solves 759 tasks (25.9 % of the pool), a 34.3 % increase over the strongest single harness.
Table 1 reports 759 solved tasks for the union, with +194 unique tasks relative to the best individual harness.
By running several specialized harnesses in parallel, we collect complementary successful trajectories that together cover far more tasks than any single harness could.
How is Multi‑Harness Discovery different from simply increasing the number of rollouts with a single harness?
Scaling up one harness only deepens its existing exploration style; it cannot produce the exclusive solves that arise from the distinct strategies of Terminus 2, RSRT, and StateM. The union adds tasks that none of the individual harnesses can reach, even if they each receive more rollouts.
**Table 1.** The union of three harnesses lead to the most unique tasks solved. *Unique* denotes tasks solved by only that harness. For the final row, “+” denotes the additional tasks solved relative to the strongest individual harness. A task is counted as solved when at least one recorded rollout passes. Rollout counts differ across harnesses in the full pool due to sandbox errors or task errors.
**Table 2.** Discovery rollouts on the terminal task set. Passing counts successful trajectories. Tasks solved counts distinct tasks; the pooled row uses the union across harnesses. Success rate is per rollout.
**Figure 3.** Complementary task coverage across harnesses. (a) Task coverage of each harness and their union on our terminal tasks, and the pooled task set. (b) Exclusive and overlapping solve regions on the pooled set. Colored dots indicate the harnesses included in each region. Of the 2,929 tasks, 2,170 are not solved by any harness. Coverage uses all recorded rollouts; per-harness rollout counts differ.
**Figure 4.** Different harness covers differnt domains of expertise. Cell values report the percentage of tasks with at least one passing rollout, using all recorded rollouts on tasks evaluated by all three harnesses. A red marker identifies the strongest individual harness in each row. The final column reports the union, and the right-hand annotations report tasks solved exclusively by Terminus 2 (T), RSRT (R), or StateM (S).
**Table 3.** Execution styles on passing trajectories. All passing rollouts on the common task set are included. Command categories are assigned by the audit's rule-based classifier and reported as percentages of issued commands.
Harness diversity significantly increases unique task coverage.
Trajectory Rewriting and Case Studies
We detail the Trajectory Rewriting trick that turns diverse harness outputs into unified training data.
Models struggle because they see only a handful of successful trajectories. By rewriting diverse harness outputs into a common interface, we can feed many high‑quality examples to a single model.
We take a successful rollout produced under a specialized harness and translate it into a sequence of generic commands that any model can execute under the general Terminus 2 harness.
How does Trajectory Rewriting differ from ordinary data augmentation?
Standard augmentation perturbs raw inputs (e.g., synonym replacement) but leaves the execution trace unchanged. Trajectory rewriting, by contrast, reconstructs the entire command sequence under a new interface, preserving procedural knowledge while normalizing the representation.
Runbook R₁ selects the compatible parser “markdown-it” and records the command “install markdown-it”.
Execution E₁ follows R₁, skipping the initial environment‑setup turns and issuing “markdown-it parse input.md”.
E₁ finishes in 30 turns, satisfying the pass filter.
Runbook R₂ omits the parser selection and directly issues “parse input.md”.
Execution E₂ fails after 28 turns because the parser is unavailable, illustrating that missing a single guidance step can break the rewrite.
Even when the high‑level guidance is identical, the presence or absence of a single setup command can flip a rewrite from success to failure.
Recursive Self‑Rewrite produces passing rewrites for 86 % of the collected rollouts.
Across 12,893 rollouts covering 975 unique tasks, 11,094 rewrites satisfy the filtering criteria.
**Figure.** Case studies comparing RSR and StateM performance across two tasks: Markdown inline parsing and Reconstructing OpenFOAM demo outputs. Each case displays success rates, median turn reductions, a high-level RSR workflow, and sample trajectory excerpts categorized by Source, Pass, and Fail outcomes with specific command-line operations.
Model Training and Evaluation
RSR-trained models outperform Direct SFT on all evaluated benchmarks.
Models benefit from diverse, high‑quality trajectories; RSR leverages harnesses to rewrite them into unified training data.
RSR‑trained models achieve higher pass@3 and mean per‑run success than Direct SFT across all benchmarks.
Table 5 shows RSR reaching 74.2 % pass@3 on TB2 versus 53.4 % for Direct SFT, and similar gains on TB3‑TBH‑SWR100.
**Table 5** RSR achieves the best results across benchmarks. We report pass@3 and the mean per-run pass rate over three runs for TB2, TB3, TB4, TBH, and SWR100, and process reward for LHTB. In addition, Direct SFT performs worse on TB 2 after training due to circular and deadend behaviors in raw trajectories from RSR.
Direct SFT fine‑tunes the model on raw passing trajectories collected by discovery harnesses, without any rewrite or runbook reconstruction.
How does Direct SFT differ from standard supervised fine‑tuning?
Standard SFT trains on human‑written examples; Direct SFT instead uses automatically generated rollouts from the discovery harnesses, keeping the raw trajectory structure and omitting the rewrite phase.
The table presents performance metrics for different harnesses (Terminus 2, Recursive Self-Reflect Terminus, StateM, and Union of all three) across three benchmarks: TB2 (89), TBH (100), and TB3 (74). Values are reported as percentages with raw counts in parentheses. Bold text indicates the highest performance for each benchmark column.
RSR‑trained models outperform Direct SFT on all complex benchmarks, confirming the value of trajectory rewriting.
Ablation Studies
Ablation study quantifies how each execution harness affects the base model’s success on three benchmarks.
A harness is the external execution system that orchestrates the model’s actions, providing planning, verification, and recovery. We evaluate the Qwen‑3.8‑27B base model under three distinct harnesses—Terminus 2, Recursive Self‑Reflect Terminus, and StateM—on three benchmarks (TB2, TBH, TB3), running each harness once per benchmark.
StateM is best aligned with TB2, achieving the highest pass@1 on that benchmark, while the union of all three harnesses attains the overall best success rates across all benchmarks. These results confirm that different harnesses expose complementary problem‑solving strengths, even for the same base model, and motivate the RSR approach of aggregating successful trajectories from multiple harnesses for further training.