Scaling Agentic Data for Long-Horizon Terminal Intelligence
Shun Zou, Ziyu Ma, Yi Zou, Yong Wang, Lin Chen, Zehui Chen, Guanhua Chen, Xiangxiang Chu, Feng Zhao
TerminalHorizon synthesizes long-horizon training data to teach agents sustained problem-solving, feedback-driven revision, and verification.
How can we synthesize long-horizon terminal data to train agents that move beyond simple command execution to sustained, multi-step professional problem solving?
Terminal agents often struggle with long-horizon tasks because they lack experience in sustained problem-solving, frequently stopping after finding an initial solution or failing to connect intermediate results to final deliverables. TerminalHorizon is a data synthesis engine that constructs complex, multi-stage terminal tasks by identifying and restoring work that environment construction would otherwise simplify or omit. It uses a two-stage process: challenge-guided reconstruction to ensure the agent must perform the intended analysis, and progressive expansion to link new requirements to prior feedback and results. Training on this data improves performance across 12 benchmarks, including a 14.2% gain on Terminal-Bench 2.0 and significant improvements in real-world productivity tasks, while enabling agents to continue experimenting and refining methods over 12-hour research budgets.
Paper Primer
The core challenge is that synthetic environments often "solve" parts of a task in advance, allowing agents to skip critical exploration, implementation, or verification steps. TerminalHorizon addresses this by using an independent challenger to identify these gaps, then reconstructing the environment to force the agent to perform the missing work.
The method hinges on two loops: an inner loop that uses independent execution to diagnose and refine task requirements, and an outer loop that progressively expands tasks under the same goal. This forces the agent to build on earlier judgments and feedback, effectively chaining exploration, diagnosis, and verification into a single, coherent problem-solving process.
TerminalHorizon-27B significantly outperforms the base model on terminal-task benchmarks.
Performance on Terminal-Bench 2.1 improved by 13.85% (46.82% to 60.67%) compared to the Qwen3.5-27B base model. The model achieved these gains using fewer than 3,000 trajectories, demonstrating high data efficiency compared to existing methods.
The model demonstrates superior sustained autonomous research capabilities in long-horizon scenarios.
In 12-hour recursive self-improvement (RSI) case studies, the model continued experimenting beyond initial solutions and retained effective improvements in final deliverables, outperforming the base model by 27.03% in hidden evaluation scores on multi-view point cloud fusion. The model consistently outperformed the base model across all 12 benchmarks, including coding, real-world productivity, and general reasoning.
Why is this approach better than simply training on more, shorter terminal trajectories?
Shorter trajectories fail to capture the "agentic behavioral patterns"—such as revising plans based on feedback or building on intermediate results—that are essential for long-horizon tasks. TerminalHorizon specifically targets these connections across stages, which cannot be learned from isolated, short-turn interactions.
What is the primary assumption behind the "challenge-guided" reconstruction?
The paper assumes that if an agent can solve a task without performing the intended analysis or implementation, the environment has leaked information or simplified the work too much. By using an independent challenger, the engine identifies these "shortcuts" and forces the environment to require the agent to perform the actual work.
This paper shifts the focus of terminal agent training from individual command execution to the organization of sustained, multi-stage problem-solving processes. Researchers should prioritize the quality of "agentic patterns" in training data over raw trajectory volume.
Introduction: The Long-Horizon Gap
We expose why short command sequences fall short for professional, long‑horizon tasks.
Short, isolated command sequences let agents demonstrate individual operations, but they never reveal how an agent accumulates evidence, revises strategies, or turns intermediate results into final deliverables.
A Long‑Horizon Task requires an agent to pursue a single professional goal over many interdependent steps, continuously adapting to feedback and building on earlier results.
When environments are synthesized automatically, the construction process can inadvertently solve the core analysis, implementation, or verification work, stripping the trajectory of the very agentic behaviors we aim to teach.
To close this gap we introduce TerminalHorizon, an automated engine that first identifies missing work in a real‑world problem and then reconstructs the environment so the agent must perform that work itself.
**Figure 1.** Motivation for TerminalHorizon. Short trajectories can teach agents to perform terminal operations, but not to autonomously complete complex, long-horizon tasks. Our long-horizon trajectories connect agentic behavioral patterns, such as exploration, strategy revision, and verification, into a complete autonomous problem-solving process.
The shift from local command execution to sustained professional task completion is the core advance of TerminalHorizon.
The TerminalHorizon Synthesis Framework
Method details the TerminalHorizon pipeline that builds and expands tasks to elicit agentic behaviors.
Training on short, isolated command sequences leaves agents unable to sustain exploration and iterative refinement required for professional long‑horizon tasks. The TerminalHorizon pipeline addresses this gap by synthesizing multi‑step, challenge‑guided trajectories.
It builds executable environments that expose the full chain of work an agent must perform, preserving the core problem‑solving steps for later analysis.
Step 1: Agent receives $I$ and observes $E$ (two parts, screwdriver).
Step 2: Agent picks up the screwdriver (action $a_1$).
Step 3: Agent aligns part A with part B (action $a_2$).
Step 4: Agent screws the parts together (action $a_3$).
Step 5: Verifier $V$ confirms the joint, ending the episode.
The agent must discover the hidden joining step; the public view never tells it that a screwdriver is required, forcing genuine problem solving.
It repeatedly runs independent challengers on partially constructed tasks, discovers what work is still missing, and refines the task so the agent must actually perform that work.
Planner reads the existing task’s instruction and identifies two missing capabilities: (i) handling error codes, (ii) logging intermediate results.
It creates two briefs: $q_{1,1}$ = “Add error‑code handling to the widget assembly task”; $q_{1,2}$ = “Require logging of each assembly step”.
Each brief is handed to the Deep Research Agent, which gathers documentation and defines concrete seeds $z_{1,1}$ and $z_{1,2}$ for environment construction.
The briefs expose concrete gaps that the independent challenger will later test, ensuring the final task forces the agent to perform the newly required work.
Independent execution of a candidate task reveals which work is already solved.
Independent challenger attempts the task using only the public view, producing a trajectory.
Diagnosis compares the trajectory to the original seed, identifying missing work.
Construction agent refines the task to restore the omitted work while preserving the public specification.
Progressive horizon expansion adds new requirements linked to the refined task.
Batch planner aggregates accepted tasks and feedback to generate the next set of research briefs.
Independent challenger loop – executes a policy on the public view only.
**Figure 2.** Overview of TerminalHorizon. Challenge-guided synthesis refines task environments through independent execution to preserve core problem-solving work. Progressive horizon expansion deepens tasks along the same goal to support richer agentic behavioral patterns.
Analysis of Synthesized Data
We detail the dataset composition, interaction statistics, and behavioral pattern distribution of TerminalHorizon.
This section quantifies the scale and richness of the TerminalHorizon dataset used for training. The statistics below illustrate why the data support sustained, multi‑step problem solving.
TerminalHorizon trajectories exhibit a mean interaction length of 52.8 turns, far surpassing prior terminal‑task synthesis methods.
Analysis of 2,852 filtered trajectories shows the average number of turns per episode.
**Figure 3.** Comprehensive analysis of task environments and trajectories in TerminalHorizon.
**Figure 4.** Interaction turns per trajectory: mean, P50, and P90.
Distribution of task environments and trajectory characteristics in TerminalHorizon provides a rich, long‑horizon training signal.
Benchmark Performance
TerminalHorizon‑27B sets new benchmarks across coding, agentic, and reasoning tasks.
TerminalHorizon‑27B sets new benchmarks across coding, agentic, and reasoning tasks, eclipsing its Qwen3.5‑27B predecessor.
Terminal‑Bench is a suite of evaluation tasks that probe a model’s ability to execute multi‑step, goal‑directed commands in a terminal‑like environment.
TerminalHorizon‑27B improves over the Qwen3.5‑27B base by up to 14.2 % on Terminal‑Bench 2.0.
Table 1 shows TerminalHorizon‑27B achieving 55.8 % versus the base model’s 41.6 % on TB 2.0.
**Table 1.** Main results on Terminal-Bench 1.0 and 2.0. Bold marks the best score in each benchmark. - denotes unavailable scores, and † denotes results from official model cards.
**Table 2.** Performance across coding, agentic, and general reasoning benchmarks.
**Table 3.** Interaction statistics and task success with two teacher models.
TerminalHorizon‑27B delivers top‑tier performance across all benchmarks while dramatically reducing the amount of long‑horizon training data required.
Long-Horizon Task Applications
We showcase TerminalHorizon’s long‑horizon capabilities and situate them among prior work.
To assess system‑level long‑horizon ability we compare TerminalHorizon‑27B with Qwen3.5‑27B on several RSI‑Exam cases under an identical 12‑hour autonomous‑research budget.
RSI‑Exam is a multi‑view point‑cloud fusion benchmark that measures how well an agent can iteratively improve a 3‑D reconstruction over extended research time.
**Figure 7.** Long-horizon optimization on RSI-Exam. More cases in Appendix C.
Long‑Horizon Agents must retain information, track progress, and coordinate actions across stages. Prior work supplies context, memory, and state‑management mechanisms to preserve continuity during extended interactions.
Terminal Environment Synthesis scales interaction experience by programmatically constructing and evolving environments. Existing pipelines focus on recursive reference‑solution expansion, whereas we emphasize cross‑stage connections: findings inform decisions, feedback guides strategy revision, and intermediate results support final delivery.
Training on terminal‑task trajectories transfers effectively to sustained autonomous research, enabling agents to keep improving beyond an initial solution.
Extended Long-Horizon Case Studies
TerminalHorizon-27B consistently beats the baseline across diverse long‑horizon tasks.
TerminalHorizon‑27B outperforms Qwen3.5‑27B on all 15 categories of Terminal‑Bench 2.1, achieving up to +66.7 pp higher mean pass@1.
Figure 8 shows the category‑wise mean pass@1 for both models.
Table 5 reports RSI‑Exam case studies where TerminalHorizon‑27B gains 7–27 pp over the base model while using a fixed 12‑hour research budget.
**Figure 8.** Category-wise mean pass@1 on Terminal-Bench 2.1.
**Figure 9.** Data efficiency on Terminal-Bench 2.0. TerminalHorizon-27B achieves 55.8% with approximately 3k training trajectories.
Trajectory Filtering Criteria
Ablation study defines process‑oriented criteria for selecting teacher trajectories.
We evaluate teacher trajectories with three process‑oriented filters before using them for supervised fine‑tuning (SFT). The filters assess validity, environment grounding, and multi‑turn training value, ensuring that only informative interactions are kept.
Trajectory validity requires coherent context and traceable action‑observation links. Trajectories that suffer severe truncation, protocol corruption, or rely on answer leakage are excluded, while recoverable parsing errors are tolerated.
Environment‑grounded progress demands that actions be driven by evidence such as file contents, logs, or command outputs. Demonstrations that merely guess or repeat ineffective actions without incorporating feedback are rejected.
Multi‑turn training value measures whether a trajectory exhibits connected problem‑solving steps—evidence gathering, implementation, verification, hypothesis revision, and error recovery—rather than arbitrary length or mechanical loops.
Overall, we retain trajectories that satisfy the three criteria even if they end unresolved; we discard only those that depend on prohibited shortcuts, maintain incorrect assumptions throughout, or provide no productive learning signal.
**Figure 13.** Additional point-cloud reconstructions for five scenes beyond Figure 7. Each row uses the same camera, scale, and point size, without post-hoc alignment, smoothing, or point removal. Colors indicate point-to-reference distance, saturated at 0.05 scene units. Scale bars represent 0.5 scene units, and F-scores are multiplied by 100.
Decontamination Analysis
We verify that training tasks do not overlap evaluation benchmarks via multi‑stage decontamination.
To ensure that performance gains are not due to memorized evaluation data, we exhaustively compare every training task against each benchmark instance.
G.1 COMPARISON CORPORA – The comparison set contains 3,251 evaluation instances drawn from twelve benchmark groups, including Terminal‑Bench 1.0‑2.1, AutomationBench, VitaBench, $\tau$²‑bench (Airline and Retail), MATH‑500, GSM8K, AIME 2024‑2025, and KOR‑Bench Cipher. Each training task is represented by its full instruction.md, while each evaluation instance is represented by its problem statement or user request.
G.2 TEXT OVERLAP AND NEAR‑DUPLICATE RETRIEVAL – After Unicode NFKC normalization and lowercasing, we tokenize into contiguous sequences of letters, digits, and underscores. Exact‑span overlap is captured by $E_n(x,y)=\mathbf{I}[G_n(x)\cap G_n(y)\neq\emptyset]$ for $n\in\{13,14\}$, where $G_n(\cdot)$ denotes the set of $n$‑grams. We also compute Jaccard similarity $J_5(x,y)=\frac{|G_5(x)\cap G_5(y)|}{|G_5(x)\cup G_5(y)|}$ and TF–IDF cosine similarity $\text{STF}(x,y)$ on unigram/bigram representations. Math‑aware 8‑gram overlaps are retrieved to catch short formula reuse, and base‑2 unigram Jensen–Shannon divergence $\text{JSD}$ characterizes distributional differences.
G.3 SEMANTIC ASSESSMENT OF CANDIDATE PAIRS – For each benchmark we collect all full‑text matches, exact 13/14‑gram matches, the top 10 pairs by $\text{STF}$, the top 5 by $J_5$, and the top 5 by math‑aware 8‑gram overlap. After deduplication, the resulting candidate set is reviewed by gpt‑5.6‑luna, which compares core objectives, initial conditions, key objects, constraints, required transformations, and completion criteria to decide whether a pair is contaminated.
G.4 DECONTAMINATION RESULTS – Table 9 shows that no benchmark contains an exact match or a high‑similarity span; the maximum $J_5$ is 0.006024 and the maximum $\text{STF}$ is 0.083445. Math‑aware 8‑gram retrieval yields 405 overlapping pairs, which together with lexical neighbors produce 190 candidates for semantic review. None of these candidates is classified as contaminated or uncertain, so all 1,535 training tasks are retained.
The figure illustrates a five-step process flow (numbered 1 through 5) alongside a set of activities categorized by functional areas: Explore, Diagnose, Build, Verify, and Coordinate. Each step in the top flow describes a specific technical event (e.g., "Repair retry", "Repair identity", "Trace response", "Read request", "Recheck chain"), while the bottom section maps these events to ongoing processes using horizontal bars with endpoints.
The figure illustrates a five-step process flow (numbered 1 through 5) mapped against five functional categories: Explore, Diagnose, Build, Verify, and Coordinate. Each step includes a brief description of tasks, such as "Build B1-B4" and "Verify delivery." Below the process flow, horizontal bars represent the duration and scope of activities within each functional category, showing how they align with the numbered steps.
**Table 8.** Benchmark text used for decontamination. Paths are relative to each benchmark distribution.
**Table 9.** Decontamination against 1,535 training tasks. $N$: benchmark instances, $E_n/M_8$: exact/math-aware overlap pairs, Contam./Reviewed: contaminated/reviewed pairs.
Numerical Calibration and Service Recovery
Reconstruction restores work and meets performance targets across three benchmark domains.
This appendix evaluates three reconstruction scenarios—radio‑interferometric calibration, RabbitMQ service recovery, and family benefits reconciliation—to measure how much work is restored when the environment is altered.
Reconstruction succeeds in all three domains, meeting or exceeding the required performance thresholds.
Radio‑interferometric RMS values are 0.896, 1.784, and 1.944 (all < 2.8); RabbitMQ recovery confirms 71 orders and 216 outbox rows; payroll reconciliation applies a \$312 adjustment while preserving consistency.
Across the three cases the pattern is identical: the initial environment supplies most of the work, reconstruction removes that work and forces the agent to perform it, and only when the agent’s restored capabilities are exercised does the workload reappear.
Qualitative Task Evolution
Qualitative analysis of task evolution across photo and hotel lineages.
We qualitatively examine how tasks evolve across three lineages: two photo‑library reconciliation branches and one hotel event‑planning lineage.
**Table 10.** Two photo-library evolution branches from the same parent task. Numbers indicate successive generations within each branch.
Branch A’s A1 generation introduces a second synchronization that must respect independent edits on both sides, A2 adds retirement/restoration checks, and A3 handles dynamic file grouping and primary‑rendition selection.
Branch B’s B1 generation replaces classified relations with evidence from media metadata, B2 accommodates independent destination edits and target‑only material, and B3 resolves transformed representations and identity evidence before merging.
**Table 11.** Three generations of hotel event-planning evolution, retaining the same ten activities.
The evolution illustrates how new constraints—such as venue disruptions or metadata‑driven identity—force agents to augment prior solutions with additional checks and operations.
Implementation Details
Implementation details, training settings, and additional analyses.
We fine‑tune the Qwen3.5 family (27B, 35B‑A3B, 122B‑A10B1) using LLaMA‑Factory’s full‑parameter supervised fine‑tuning (SFT) on the multi‑turn trajectories generated by TerminalHorizon‑3K.
All three model sizes share the same recipe, summarized in Table 4, and we keep the checkpoint after three epochs of training.
The table lists various hyperparameters and their corresponding values used for model training, including optimizer settings, learning rate, scheduler, weight decay, epoch count, sequence length, batch size, precision, optimization strategy, attention backend, and random seed.
Evaluation runs on eight H20 GPUs via vLLM with tensor parallelism 8 and a 262,144‑token context window; decoding settings (temperature 1.0, top‑p 0.95, top‑k 20, max output 65,536 tokens) are applied uniformly to Terminal‑Bench, AutomationBench, and general reasoning benchmarks.
Terminal‑Bench 1.0, 2.0, 2.1 use Harbor2 and Terminus‑2 sandboxes; each task allows three hours of agent time, and we report mean pass@1 over three independent runs, retrying up to two times for infrastructure failures.
AutomationBench v1.0.6 evaluates 600 rollouts with a 50‑step limit per trajectory, reporting the mean Partial score and Pass Rate after three independent runs, retrying incomplete samples up to three times.
VitaBench cross‑domain uses 100 tasks across three service domains, each run four times with 300 interaction steps, temperature 0, and a 65,536‑token output cap; success is measured as Avg@4 over the 400 runs.
$\tau$2‑bench evaluates Airline (50 tasks) and Retail (114 tasks) with four runs per task, 200 interaction steps, and temperature 0 for all agents; we report the mean task success rate, ignoring retries that do not affect the final count.
General reasoning benchmarks (MATH‑500, GSM8K, AIME 2024/2025, KOR‑Bench Cipher) are run with their native evaluators, generating a single answer per problem and reporting mean accuracy over three runs.
Across the 16 categories of Terminal‑Bench 2.1, TerminalHorizon‑27B matches or exceeds the base Qwen3.5‑27B, delivering especially large gains in debugging (+46.7 pp), data science (+25.0 pp), and machine learning (+22.2 pp).
When comparing trajectory efficiency, TerminalHorizon‑27B reaches 55.8 % pass@1 on Terminal‑Bench 2.0 using only ~3 k trajectories, outperforming Terminal‑Universe‑27B by 3.0 pp while consuming less than one‑tenth the data (≈3 k vs 32 k trajectories).
Professional Domain Coverage
Defines agentic behavioral patterns and their taxonomy across professional domains.
Our benchmark spans 1,535 tasks distributed over 25 professional domains. Figure 14 visualizes this breadth, showing domains from software to agriculture, healthcare, and scientific research.
**Figure 14.** Overview of TerminalHorizon’s 25 professional domains.
An agentic behavioral pattern is the logical glue that links what the agent does, what it observes, and what it decides next while pursuing a concrete goal.
Tables 6 and 7 list the concrete patterns belonging to each category, providing concise definitions for all 27 patterns.
The figure illustrates a five-step workflow process (numbered 1 through 5) and maps these steps to five functional categories: Explore, Diagnose, Build, Verify, and Coordinate. Each step includes a brief description of the task (e.g., "Reduce pilots," "Sample warming," "Test cooling," "Fit two models," "Commit and close"). Below the workflow, horizontal bars with endpoints indicate the duration or scope of specific activities within each functional category, such as "Problem-driven exploration," "Hypothesis-driven diagnosis," and "Sequential decisions in dynamic environments."
In the TLS gateway repair case, relational tracing, diagnosis, and modification‑recheck iteration combine to incrementally fix the handshake logic.
The neutron‑diffraction study demonstrates problem‑driven exploration, hypothesis‑driven diagnosis, and dependency‑driven progression as the agent refines temperature measurements and model selection.
In the synthetic care‑staffing scenario, sequential decision‑making, error‑directed correction, and verification loops coordinate resident transfers across six planning windows.
Resource Optimization Details
Procedures for verifying resource schedules, handling venue changes, and updating recovery plans.
Resource quantities are refined into specific installed equipment, mobile units, and crew, each with compatibility constraints and explicit teardown, transfer, and installation times.
Travel time between each group’s consecutive activities is checked; venues or times are adjusted when needed to ensure every group can complete its full schedule, not merely attend isolated activities.
Resource compatibility is verified and ready times are calculated after prior use, teardown, transfer, and installation; three directly affected activities trigger venue changes that are committed in a single atomic revision without altering any activity times.
In the third generation, two candidate plans are rejected because shared equipment cannot serve the proposed sequence; for example, a mobile display released at 16:30 needs 15 min teardown, 20 min transfer, and 20 min installation, making its earliest ready time 17:25, which conflicts with Workshop A’s original 17:15 start.
The accepted plan moves the activity to Oak, combines its installed whiteboard with the mobile display, and shifts the start to 17:30 on the public 15‑minute planning grid.
These cases illustrate how evolution changes both what the agent must determine and how decisions depend on earlier work, linking identity recovery to metadata merging, venue changes to attendee flows, and resource availability to schedule revision.
J.1 Trajectory Filtering Prompt implements three filtering criteria from Appendix F, reviewing complete trajectories and returning an evidence‑backed KEEP/DROP decision for each record.