Scaling Long-Horizon Cowork Agent: Training via Recursive Environment Evolution

Hang He, Linting Wang, Guanming Liu, Bo Mao, Jinxiu Liu, Yiran Suo, Maosen Zhou, Zhiheng Xi, Guanting Dong, Zhou Shao, Rui Zheng, Jie Zhou, Chengcheng Wan, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang

WorkRover uses recursive environment evolution and turn-level credit assignment to train long-horizon cowork agents.

How can we automatically generate and scale complex, long-horizon interactive environments to train agents that perform reliable, multi-step "cowork" tasks?

Training long-horizon agents requires environments that provide reliable feedback, but synthesized tasks often contain defects—such as inconsistent specifications or broken dependencies—that make valid agent behavior appear unsuccessful. WorkRover addresses this by using a versioned harness to jointly construct and recursively repair work contexts, tasks, and verifiers. It uses Factorial Compatibility Replay to test and select minimal, dependency-aware repairs, and Verifier-to-Turn Credit Assignment to shape policy updates using workspace-grounded progress signals. Post-training Qwen3.5-35B-A3B with this framework yields significant performance gains, including a 413.25 Elo improvement on GDPval v2 and double-digit percentage point increases on JobBench and OfficeQA Pro.

Paper Primer

The core challenge in training cowork agents is maintaining alignment between the work context, the task, and the verifier. When a task is synthesized, defects often emerge that the verifier cannot correctly evaluate, leading to noisy training signals that reward incorrect behavior or penalize valid solutions.

WorkRover treats environment construction as an iterative evolution process: it uses a harness to track component dependencies and Factorial Compatibility Replay (FCR) to evaluate combinations of repair proposals. FCR is like a circuit debugger—it isolates which specific component updates resolve a failure while ensuring the fix doesn't break other parts of the task or violate the original intent.

WorkRover significantly improves agent performance across diverse professional benchmarks.

Post-training Qwen3.5-35B-A3B with WorkRover-35B-A3B compared to the backbone model. +26.27 percentage points on JobBench, +29.32 percentage points on OfficeQA Pro, and +413.25 Elo on GDPval v2.

Recursive environment evolution increases task reliability.

Comparison of FCR against full regeneration under matched repair budgets. FCR raises the Safe Resolution Rate (SRR) for defects to 70.3% (vs 60.7%) and reduces unintended changes to task intent to 1.3% (vs 14.3%).

Why is "recursive evolution" necessary instead of just generating more tasks?

Generating more tasks does not fix existing defects in the construction pipeline. Recursive evolution allows the framework to identify and repair mismatches in the work context, task, and verifier that are only exposed during agent rollouts, ensuring the training data remains high-quality and aligned.

How does the credit assignment method differ from standard reinforcement learning?

Standard reinforcement learning typically uses terminal rewards, which provide little guidance on which specific actions led to success. Verifier-to-Turn Credit Assignment (VTCA) uses deterministic workspace checks to identify progress at each turn and distributes a bounded share of that credit to prerequisite actions based on recorded tool and artifact dependencies.

WorkRover demonstrates that environment reliability is as critical as model architecture for long-horizon agents; by automating the repair of training environments, researchers can scale agent training without manual intervention.

Introduction and Motivation

We expose the need for scalable, verifiable environments to train long‑horizon cowork agents.

A Cowork Agent is an LLM‑driven system that performs professional tasks across files, tools, and applications, coordinating actions over long sequences.

Training such agents demands interactive environments that can be generated at scale, tested, and iteratively refined while keeping work contexts, tasks, and verifiers aligned. Existing pipelines offer only limited support for this recursive repair process, creating a gap between environment construction and reliable long‑horizon RL training.

WorkRover is a framework that lets construction agents jointly synthesize and validate work contexts, executable tasks, and evidence‑grounded verifiers, adapting their actions to workspace feedback rather than following a fixed recipe.

We evaluate WorkRover by post‑training Qwen3.5‑35B‑A3B, producing WorkRover‑35B‑A3B, which outperforms the baseline on five cowork‑agent benchmarks, achieving large gains on JobBench, OfficeQA Pro, and GDPval v2.

**Figure 1.** Cross-benchmark performance. Results on five cowork-agent benchmarks.

**Figure 2.** Cowork agents: challenges and approaches.

The shift from simple chat to long‑horizon cowork agents requires scalable, verifiable environments.

Training Efficiency: VTCA

Core mechanisms that enable long‑horizon training via environment evolution and turn‑level credit.

The method builds on three phases: constructing a reusable work context, synthesizing a long‑horizon task, and generating a verifier that can assess task completion.

Phase I builds a reusable Work Context $C_i$ from files, apps, and skill libraries.

Phase II creates a Task $T_i$ by matching the desired edit scope, horizon, and skill complexity to the resources in $C_i$.

Phase III synthesizes a Verifier $V_i$ from an Evaluation Contract derived from $T_i$, then freezes it for a one‑time admission test.

Instead of rewarding an episode only at the end, VTCA distributes a bounded share of positive credit to each turn that contributed to progress, using recorded tool outputs and artifact‑version dependencies.

Turn 1 receives $2/10 = 0.2$ of the terminal reward as credit.

Turn 2 receives $3/10 = 0.3$ credit; it also incurs a trace cost of 0.05 for an extra tool call.

Turn 3 receives $1/10 = 0.1$ credit; no trace cost.

Turn 4 (the final turn) receives the remaining $4/10 = 0.4$ credit, completing the episode.

VTCA ensures that each turn’s contribution is proportional to its observed progress, while penalizing superfluous actions via trace costs.

E‑GRPO updates the policy using only the final advantage of an episode, but the advantage is computed from turn‑balanced credits provided by VTCA.

Why does E‑GRPO not need per‑turn gradients if VTCA already distributes credit?

VTCA’s turn‑balanced advantage already aggregates the effect of each turn into a single scalar. E‑GRPO treats that scalar as the episode’s advantage, so the policy update can be performed in one step, preserving the benefits of turn‑level credit without the overhead of per‑turn gradient computation.

**Figure 4.** Skill-library coverage. Six largest categories; others aggregated. Tile sizes show shares of skills (a, c) or category memberships (b).

**Figure 5. Overview of VTCA.** Same-task rollouts yield terminal rewards and checked progress. Bounded prerequisite allocations and local trace costs shape turn-balanced GRPO advantages.

Recursive Environment Evolution

Repair loop that fixes environment‑task mismatches while preserving the original task intent.

When the Work Context, Task, Harness, or Verifier are inconsistent, rollouts either abort or produce spurious rewards, stalling long‑horizon learning.

The system repeatedly rebuilds the environment‑task package until all components agree, treating each rebuild as a small “mutation” that is validated before being kept.

How does this differ from simply patching the failing component after a rollout?

Simple patching would ignore transitive dependencies: changing the Context might require a new Task definition or a different Verifier rubric. Recursive Evolution explicitly closes the dependency graph, ensuring every affected module is updated together, which prevents silent regressions.

FCR enumerates every feasible combination of component edits, validates them jointly, and picks the smallest change that restores compatibility.

$\chi$ = (1,0,1) → candidate set includes $\Delta$C and $\Delta$V.

Dependency closure adds a required update $\Delta$H because the new Verifier expects a different Harness interface, yielding $\Gamma$($\chi$) = {$\Delta$C, $\Delta$V, $\Delta$H}.

Rebuild the package with these three updates and replay the rollout using the same policy and seed.

Outcome: the rollout succeeds, and the verification score improves, confirming compatibility.

Even though only two components were directly edited, the closure step automatically pulls in a third dependent component, illustrating why joint repair testing is essential.

Why does FCR test joint repairs instead of only the individually successful edits?

Some incompatibilities only disappear when two components change together (e.g., a new Context may require a matching Verifier rubric). Testing joint repairs reveals these hidden dependencies, whereas isolated tests would falsely label the edits as failures.

Diagnose the failure and generate a set of candidate edits $\Delta$X for each affected component.

Form all binary vectors $\chi$∈{0,1}^d representing subsets of edits; prune vectors that create obvious dependency conflicts.

For each surviving $\chi$, compute the dependency‑closed update set $\Gamma$($\chi$) = Close($\chi$).

Rebuild the package with $\Gamma$($\chi$) and replay the rollout under identical conditions.

Select $\chi^*$ that yields the smallest number of version updates in $\Gamma$($\chi^*$); break ties by fewer rebuilt dependencies.

Confirm the selected repair with fresh rollouts and, if successful, commit the updates to the training collection.

FCR repair selection pseudocode.

**Figure 3.** Environment construction and recursive evolution in WORKROVER. The Harness coordinates Work Context construction, Task synthesis, and Verifier synthesis and validation. FCR repairs compatibility mismatches under frozen safety gates; verified trajectories support policy learning.

**Figure 7. FCR mechanism, stability, and evolution.** (a) Pairwise joint necessity (C: Context; T: Task; H: Harness; V: Verifier). (b) Natural-trigger outcomes. (c) Repair reliability across three generations, measured by SRR and sealed-audit pass rate. (d) Task success across five repair iterations.

Benchmark Performance

Benchmark results show WORKROVER‑35B‑A3B performs competitively across all five tasks.

Table 1 summarizes performance of our WORKROVER‑35B‑A3B model and baselines on five benchmarks.

**Table 1.** Benchmark results. Bold and underlined mark the best and second-best scores per model group. Subscripts show changes from Qwen3.5-35B-A3B: ↑ increases and ↓ decreases (Elo or percentage points). Higher scores are better. Details in Appendix D.1.

WORKROVER‑35B‑A3B with VTCA consistently outperforms baselines on all five benchmarks.

Ablation Studies

Ablation studies quantify how each component contributes to higher credit assignment, environment repair, and construction success.

Recall that the paper’s core idea is to give turn‑level feedback (VTCA) and to automatically repair environments (Recursive Environment Evolution) so long‑horizon RL can learn efficiently.

VTCA outperforms episode‑only GRPO on all five benchmarks, with the largest gain on Automation‑Bench.

Automation‑Bench improves by +5.7 % over E‑GRPO when using VTCA.

VTCA also leads on OfficeQA Pro, confirming the benefit of turn‑level credit.

OfficeQA Pro improves by +5.4 % compared to E‑GRPO.

On GDPval v2 the advantage translates into a large Elo increase.

VTCA gains 43 Elo over E‑GRPO.

These results support the hypothesis that providing feedback for individual turns yields stronger learning than rewarding only the final trajectory.

FCR raises safe defect resolution while keeping task‑regression and token cost low, demonstrating that selecting compatible edits is more effective than applying all proposals.

Removing relation‑aware expansion or validation‑guided rewriting causes the steepest drops, confirming that both dependency‑aware expansion and rewrite steps are essential for high‑quality task construction.

Related Work

Survey of environment synthesis, verification, and credit‑assignment techniques for long‑horizon agents.

Executable environments and task synthesis. Professional agent training requires executable tasks, recoverable state, and observable outcomes across related artifacts. Systems such as AppWorld, EnvScaler, ClawGym, Agent‑World, ToolForge, MetaTask, and CLI‑Universe each provide mechanisms for constructing such environments, while WorkRover integrates task construction with file relations and professional skills, assembling Tasks and Verifiers within versioned Harnesses and shared Work Contexts.

Grounded acceptance and verifier reliability. Acceptance checks must agree with the instruction and the evidence available in the workspace. ANCHOR builds instructions, environments, solutions, and verifiers from a shared workflow specification; FACET grounds artifacts in shared state and repairs inconsistencies while preserving source intent; Hacker‑Fixer audits tests and environments with adversarial cases while checking legitimate solutions. WorkRover freezes instruction‑grounded requirements and criterion‑evidence bindings, then evaluates implementations using violating mutants and independently valid solutions, with reference artifacts supplying recomputable anchors without adding requirements.

Environment evolution and repair selection. Envs‑FORGE studies joint revision of instructions, fixtures, solutions, tests, and environments, while EchoVerse studies executable environment evolution. Program‑repair research such as GenProg reduces test‑passing patches through structural differencing and delta debugging. FCR (Factorial Compatibility Replay) freezes component‑level proposals over Context, Task, Harness, and Verifier before replaying their combinations; materialization includes dependency‑induced changes and promotion checks that avoid regressions under fixed requirements. The selected repair is the smallest admissible closed edit set among the tested combinations, though bounded enumeration does not guarantee a global optimum.

Local progress in long‑horizon learning. Methods differ in the evidence used to assign local feedback: GiGPO uses state comparisons, HiPER uses a planning hierarchy, TRACE forms temporal differences in frozen‑model reference‑answer likelihoods, and TRCA combines deterministic transition‑rubric scores with cumulative condition coverage increments. VTCA (Verifier-to-Turn Credit Assignment) instead evaluates successive observable workspace states with frozen checkers. Signed progress can register both completion and setbacks in persistent, mutable artifacts, while unsupported evidence supplies no local signal and terminal evaluation remains applicable to supported task outcomes.

Evidence attribution and shaping limits. STAMP maps verified citations to first‑exposure actions; DRACO uses judge‑attributed steps with sign‑preserving GRPO advantage redistribution. VTCA follows recorded direct use of exact tool outputs or artifact versions, assigning at most half of positive progress to eligible prerequisite turns and leaving negative increments local. Incorrect provenance can misallocate credit, and missed links leave prerequisites unrepresented. Potential‑based reward shaping preserves optimal policies under stated assumptions, but VTCA’s workspace‑grounded advantage shaping does not guarantee policy invariance nor an unbiased policy‑gradient estimate.

Data Construction Details

Describes how work contexts are built, resources assembled, and case studies illustrate the process.

A Work Context is a typed graph that captures files, versions, and their semantic, structural, temporal, provenance, and workflow relationships for a given source workspace.

The inventory comprises 21,444 app skills, 5,918 file skills, and 36,064 job skills spanning 317 roles. App and job skills each receive a single category label across 15 and 10 categories, while file skills belong to 22 categories with 11,273 memberships. The six largest categories per library are highlighted in the paper; the rest are grouped as “Other.”

Workspace construction draws from a single research record’s exported file collection. Files are catalogued by path, format, size, and content hash; bounded previews expose headings or sheet names for selective reading. Only files, notes, tables, or reports that pass relevance and leakage checks may enter the Starter State; raw logs, caches, and unsupported web caches are excluded.

A job skill’s requirements drive eligible‑file retrieval within one source collection. Branch plans must differ in professional decisions, operations, or principal products and cannot import unselected files or mix source collections. Feasible plans become workspaces and Task Blueprints that specify file roles, dependencies, deliverables, and modification constraints, with observable criteria for each required outcome.

Fresh execution uses the instruction and selected files to define a frozen Starter State. A Demonstrator then creates a Reference State; historical sessions cannot substitute. The Contract admits only valid Reference States that respect the instruction, visible invariants, and independently recomputable anchors, while rejecting mutated Starter States.

Task synthesis packages an instruction, a resettable snapshot, an environment specification, metadata, and a fresh Demonstrator realization. Core files must have defensible relations; input ablation removes unnecessary bindings, and distractors cannot alter the dependency closure. Gates verify instruction clarity, readable inputs, sufficient dependencies, reproducible restoration, and outcome anchors.

**Figure 8** From Deep Research files to verified task workspaces. Instructions and visible invariants define requirements; the dashed green arrow supplies recomputable Reference State anchors. Case 1781 uses two IPSS workbooks; shaded cells mark output values to populate.

Table 4 defines three task‑construction profiles (P1–P3) that scale edit scope, demonstrator horizon, skill composition, noise ratio, cross‑environment dependencies, and distractor tiers from low to high complexity.

The Blueprint binds each outcome to extractable or independently recomputable artifact evidence. Review checks include opening/parsing, formulas, merged cells, annotations, version history, layout, and embedded objects; visual outcomes also require rendered inspection.

Table 5 lists representative workflows (artifact revision, data reconciliation, etc.) together with their required inputs, dependencies, and observable outcome evidence such as consistent versions or recomputed aggregates.

**Figure 9.** Recorded example 1875: prompt-engineering course design. This researcher branch writes `teaching_design`.md, inspects four segments, and adds the BIG-Bench Hard paper citation in three places. The proposed task reuses captured CoT, SoT, and DSPy paper-source archives to produce a method comparison workbook and three lab briefs with baselines and grading rubrics. Claims cite local source sections; the original archives remain unchanged.

Figures 9 and 10 pair recorded research workflows with their exported file collections; shaded cells indicate output fields to populate. New tasks still require fresh Demonstrator execution and Verifier checks, while raw traces and internal provenance remain outside Solver inputs.

The quarterly executive handoff case study demonstrates how a high‑level summary is turned into a concrete task workspace, following the same construction pipeline and verification steps described earlier.

Training and Verification Details

This section details how the Verifier checks states, how FCR repairs mismatches, and how credit is assigned per turn.

The Verifier enforces per‑criterion contracts using weights $w_k$, thresholds $\gamma_k$, and critical flags $b_k$. It evaluates each turn, produces a binary pass $y_{i,k}$, and aggregates scores to decide whether a trajectory is admissible.

**Figure 10** Recorded example 1781: policy research on older adults living alone. The proposed task selects `hprj2024_gaiyo_kekka1`.xlsx and `hprj2024_gaiyo_kekka2`.xlsx, retaining category definitions, units, and source-sheet references. The revised `policy_memo`.md is observed through tool readbacks. Source-page references at m39 cite IPSS and MHLW.

**Figure 11. Recorded quarterly executive handoff.** (a) File, capability, and app context. (b) Selected rollout stages. (c) Revenue and export-path evidence supporting consistent updates. (d) Deliverables and evaluation, archived evaluation, retained follow-ups, and validation scope.

C.2 FCR Mechanics and Promotion describe how a single proposal is frozen per context, then replayed across dependency‑closed update sets $\Gamma(\chi)$. Conflicts render cells unavailable; otherwise the repair is materialised, preserving contract weights, thresholds, and flags.

C.3 FCR Evaluation Protocol samples 300 natural mismatches from a 10 K pool and injects 260 concealed defects of varying factor cardinalities. Each configuration is tested with five matched seeds, and verification outcomes are compared using sealed label agreement.

C.4 Verifier‑to‑Turn Credit Assignment (VTCA) uses the canonical traces to deduplicate producer turns, applies bias terms $b_{i,r,s,t,k}$, and computes the critical delta $\delta^{\text{crit}}_{i,r,t}$ using normalized weights $\bar{w}_{i,k}$. Only complete observations telescope; malformed calls incur a trace cost $c_t\in[0,1]$.

Evaluation and Setup Details

Summarizes material boundaries, task construction stages, and case‑study details.

Table 6 enumerates the source materials used to build tasks, the role each material plays during construction, and whether the Solver can see the material. The rows distinguish original research queries, logs with internal analysis, and internal manifests, each with distinct visibility rules.

Table 7 details the sequential stages of task construction and the corresponding admission checks that verify each stage. Stages range from indexing the source to the final acceptance check, ensuring that no completed‑answer leakage occurs and that all dependencies are provably supported.

Table 8 presents three case studies—course design, household projections, and a quarterly executive handoff—showing the inputs provided, the work products requested, and the scope of evidence used for verification. Each case highlights which files are frozen, which outputs are generated, and the verification outcomes (e.g., a 0.946 score for the executive handoff).

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers