AutoSFT: An Agentic System for Automated Data Research in Supervised Fine-Tuning
Bingxiang He, Pengqi Sun, Shihao Peng, Shaoxuan Li, Huiyu Xiao, Hongjian Shi, Zixuan Fu, Ziqing Qiao, Yuxin Zuo, Cheng Qian, Xinqiang Yu, Jiayi Li, Xiusi Chen, Yizhen Song, Xiangyang Wang, Iofai Lei, Yudong Wang, Ning Ding, Zhiyuan Liu, Xiaozhi Wang, Chaojun Xiao
AutoSFT decomposes data research into specialized agent roles to cover the full range of data-engineering actions.
How can we automate the iterative refinement of SFT data recipes using a multi-agent system instead of a single agent?
Automated data research for supervised fine-tuning (SFT) often collapses into a single, repetitive action because agents struggle to manage both execution and analysis within a limited context window. AutoSFT solves this by isolating execution from analysis and delegating tasks to specialized sub-agents: an executor handles training and evaluation, four analysis agents examine data from different perspectives, and a synthesis agent generates new examples for undersupplied capabilities. This structure enables the system to perform a wider range of data-engineering actions, resulting in significant gains on instruction-following and code-generation benchmarks compared to single-agent baselines.
Paper Primer
The core move is the decomposition of the research loop into a meta-agent that orchestrates three distinct sub-agent roles. By separating the heavy lifting of training and evaluation into an isolated executor, the meta-agent preserves its context for high-level data strategy, while parallel analysis agents provide diverse, evidence-based proposals for data improvement.
Multi-agent decomposition significantly improves instruction-following performance.
Adding analysis sub-agents and a synthesis sub-agent to the research loop. +6.5 on IFEval (analysis) and +3.4 IFEval / +2.9 IFBench (synthesis).
The system's performance gains transfer to the code-generation domain.
Applying the full AutoSFT loop to MBPP+ code generation. +3.7% improvement over a single-agent baseline that failed to improve.
Why does a single agent fail to explore the full space of data-engineering actions?
The single agent's context window becomes saturated with raw execution logs and job status updates, forcing it to discard research history and limiting its reasoning to simple mixture-weight adjustments rather than complex interventions like row repair or synthesis.
What is the primary advantage of the synthesis sub-agent over simply selecting from an existing pool?
The synthesis sub-agent generates new examples specifically for capabilities that are undersupplied or missing in the initial pool, allowing the system to address model weaknesses that cannot be fixed by reweighting or pruning existing data.
Introduction
We expose the bottleneck of manual data curation and propose a multi‑agent loop to automate SFT data research.
Current pipelines rely on human engineers to curate supervised fine‑tuning (SFT) data, a step that becomes the dominant bottleneck as model size grows. Prior agents are given only one kind of control—either selecting rows from a fixed pool or invoking a generator—so a single‑agent loop quickly converges on a single data‑action family and wastes most of its context on raw command output.
The loop repeatedly proposes, executes, and evaluates modifications to the SFT training set, treating the data itself as the search object.
# Scope Start from the pinned input snapshot named by the task context. Examine the staged pool and, when available, the incumbent round data and its exposure. Assess current content rather than trusting filenames, old labels, or prior quality annotations. Look for useful material lost by selection, duplicated or conflicting supervision, insufficient coverage, and rows whose substantive task can be recovered by repairing the instruction, response, or metadata. # Proposal <- identical in all four analysis skills Deliver an executable full-scale plan, not a small training dataset. Name the source populations, selection or transformation steps, quality checks and unknowns, expected coverage, attainable token volume, budget tradeoffs, and how the meta agent can assemble the formal set. Distinguish measured findings from estimates and expected effects. A useful negative result or unresolved uncertainty is valid. Do not invent a gain or apply an arbitrary quality gate. Analysis subagents cannot edit the shared pool, the round data, the recipe, or the memory. Persist the proposal through the task report and cite the source snapshots and artifacts it rests on.
**Figure 15** | Two capabilities over the run with synthesis. The line is the capability's whole-question strict pass rate each round, and the bars are the rows synthesized for it that round.
The key shift is moving from manual, brittle curation toward an agentic system that can explore the full space of data‑engineering actions.
The Research Loop
Iteratively refines data recipes while keeping the base model fixed, using a policy‑driven loop.
The loop isolates the data recipe as the only mutable component; the model checkpoint stays frozen, so any performance change must come from the recipe.
Think of a chef who repeatedly tastes a sauce (the feedback) and tweaks the ingredient list (the recipe) while keeping the cooking equipment (the base model) unchanged.
How does this multi‑agent loop differ from a single‑agent data‑curation pipeline?
A single‑agent pipeline can only edit rows directly; the multi‑agent policy can invoke separate agents to generate new rows, rewrite responses, or restructure the whole recipe, giving a richer action space than the single‑agent baseline.
Round 0: Trainer samples rows 1 and 3, trains on them, and produces checkpoint $C_0$.
Evaluation reports that row 3 fails the target metric, while row 1 succeeds.
Feedback $F_0$ therefore contains a failure flag for row 3.
Policy $\pi$ maps $F_0$ to action $a_0$ that increases the weight of row 3 to 2 and adds a new row 5 with weight 1.
New recipe $R_1$ has weights $[1,0,2,0,1]$; the pool $D_1$ now contains 5 rows.
The loop can both amplify promising examples and inject fresh data, illustrating why fixing $M_0$ forces the policy to improve performance solely through recipe manipulation.
Data‑research actions operate on rows—each row bundles an instruction, a response, and its metadata. An action specifies which rows to train, how often they appear, their order, and whether any row is edited or newly written.
Figure 2 visualizes a single loop round: the recipe $R_t$ feeds the trainer, the trainer’s output is evaluated, feedback $F_t$ is produced, and the policy $\pi$ consumes $F_t$ (plus history $H_{<t}$) to emit the next action $a_t$, which updates the recipe.
**Figure 2.** One round of the research loop. Only $\pi$ varies across the configurations we compare.
Limitations of Single-Agent Research
Single agents underperform due to data and context bottlenecks.
A single Codex agent fails to hit the IFEval 80 or IFBench 50 targets after 33 rounds (11.5 h) of execution.
Submitted 33 rounds, evaluated 32; none reached the benchmark thresholds.
A single agent repeatedly reuses the same context and data actions, so it cannot scale its reasoning over long trajectories.
How is the single‑agent bottleneck different from simply giving the agent more wall‑clock time?
Extra time only lets the agent repeat the same limited set of actions; without new data moves or retained context, additional epochs cannot produce higher benchmark scores. The bottleneck is structural, not just a matter of duration.
PostTrainBench measures the quality of a model after a full fine‑tuning cycle, focusing on instruction‑following ability.
**Figure 5** | Training data per round under a wall-clock budget.
**Figure 6.** The single agent's context. Dashed lines mark compactions labelled with the round.
Single agents fail to scale reasoning over long trajectories because they repeatedly prune context and lack diverse data actions, causing diminishing returns despite ample compute.
AutoSFT System Architecture
AutoSFT orchestrates multiple agents to iteratively refine training data recipes.
Single‑agent loops drown in verbose training logs, leaving the useful signal buried. AutoSFT solves this by pulling the heavy‑weight training and evaluation work into a dedicated executor, while lightweight agents focus on data‑level insight.
The system splits the data‑research loop into specialized agents, each handling a narrow piece of the pipeline, so the central meta agent can concentrate on high‑level coordination.
How is the meta agent different from a simple controller that just triggers subagents?
The meta agent does more than schedule; it reads evidence files, weighs competing proposals, and explicitly updates the data recipe $R_{t+1}$ based on a chosen action $a_t$. A plain controller would issue commands without this evidence‑driven selection step.
The meta agent receives the current data pool $D_t$ and recipe $R_t$.
It hands $D_t$ and $R_t$ to the executor subagent, which trains model $C_t$, evaluates on $K$ benchmarks, and writes feedback $F_t$ plus artifact paths.
Four analysis subagents each read $D_t$, $R_t$, $F_t$, and the history $H_t$, then output a single executable data proposal.
The synthesis subagent inspects $F_t$ for undersupplied capability targets and, when needed, synthesizes new instruction‑response pairs that satisfy programmatic verifiers.
The meta agent aggregates all proposals, selects an action $a_t$, and produces the next recipe $R_{t+1}=a_t(D_{t+1},R_t)$.
The loop repeats with the updated data pool $D_{t+1}$.
**Figure 1.** Overview of AutoSFT. The meta agent commits a recipe. An executor trains and evaluates it separately and returns artifact paths. Analysis subagents read these artifacts from complementary angles, and a synthesis subagent writes data for undersupplied capabilities. The meta agent integrates their proposals into the next recipe. All roles share a candidate data pool and a skill library.
Each family is a primitive operation that reshapes rows of a training recipe, either by adding, removing, or reordering instruction‑response pairs.
Mixture: combine rows 1 and 3 to form a new row I₁+I₃ → R₁+R₃, increasing the recipe to five rows.
Pruning: drop row 2 because its response R₂ is flagged low‑quality, reducing back to four rows.
Repair: edit row 4’s response from R₄ to R₄′ (fixing a formatting error).
Resupervision: replace row 1’s response with a higher‑quality version R₁′ while keeping I₁ unchanged.
Synthesis: generate a new row I₅→R₅ that satisfies the undersupplied capability “no commas in the instruction”.
Ordering: reorder rows to the sequence 3, 1, 5, 4, 2 for better curriculum progression.
Each family targets a distinct dimension of the recipe (size, quality, content, or order), and chaining them lets the system sculpt the data pool precisely toward the evaluation targets.
How does the “Mixture” action differ from ordinary data augmentation?
Mixture actually merges two existing rows into a single composite row, changing both instruction and response simultaneously. Standard augmentation typically perturbs a single example (e.g., synonym replacement) without creating a new combined entry.
**Figure 3.** The six families of data action, drawn as their effect on the rows of a recipe. Shading marks what the action changes.
Experimental Results
AutoSFT’s multi‑agent loop boosts performance across all evaluated tasks.
AutoSFT automates SFT data research by running a multi‑agent loop that iteratively refines training‑data recipes, sidestepping the manual‑curation bottleneck.
Synthesis subagent raises IFEval performance to 81.58, a +3.43‑point gain over the base configuration.
Best IFEval 81.58 (synthesis) vs 78.15 (base) after three epochs under a 100 M‑token cap.
This table outlines the experimental settings categorized into Agent, Training, and Evaluation components.
**Figure 7.** Context used by the meta agent over each run and the composition of its recorded events.
**Figure 8.** Left: IFEval per round, with dotted guides at the rounds named in the text. Right: distribution of data actions over Figure 3 in agent traces for w/ and w/o Analysis settings.
**Figure 9** | Scores per round. Left two panels: synthesis ablation on instruction following (Base vs Base + Synthesis, Section 5.3). Right panel: three configurations on code generation (Section 5.4). Bands give the standard deviation over evaluation repeats.
Related Work
We situate AutoSFT among prior data selection, synthesis, and benchmark efforts.
Early work established that careful data curation can outweigh sheer scale, with small, well‑chosen sets matching or surpassing much larger corpora. Selection approaches prune or reweight examples based on information gain or diversity, while synthesis techniques bootstrap instructions from seed sets, evolve complexity, or extract alignment data. Some methods also target preference data, curating feedback or preference training pairs, yet they all produce a fixed dataset without closing the loop on downstream training performance.
AutoSFT is an agentic system that automates data research for supervised fine‑tuning by iteratively refining training data recipes. Prior systems address either selection or synthesis: PostTrainBench audits 84 agent runs converging on SFT; TREX searches recipes via MCTS; AutoSelection formalizes operator‑program search; DataChef trains a policy with a proxy reward; Luo et al. iterate on data alone with gains plateauing after round 8–15; Autodata revises generation from solver feedback; ANDES packages synthesis as a callable skill; BigBang scales self‑evolving synthesis with a critic proxy; Arbor, AutoSOTA, and the Anthropic automated researcher organize research with hypothesis trees, multi‑agent decomposition, and diverse initial directions respectively.
Benchmarks now evaluate agents on research tasks ranging from single Kaggle problems to full R&D pipelines, as well as AI infrastructure and self‑improvement. Closest to our setting, DataEnvGym frames data generation as an RL loop, Curation‑Bench gives agents iterative control over a fixed pipeline, and RSIBench‑Data and DataPrep‑Bench benchmark data‑centric recipe search. We differ by requiring from‑base retraining each round, repeated evaluation to bound noise, and a verification gate on synthesis.
Single-Agent Run Details
Single‑agent ablations reveal how component removal impacts performance.
This appendix isolates the single‑agent run that the paper uses as a baseline for the multi‑agent system. All 32 rounds share micro‑batch size 1 and a minimum learning rate of $10^{-7}$, and the actions are either “reweight” (W) or “replay” (P).
**Table.** The table displays 32 rounds of an experimental process, organized into two side-by-side blocks of 16 rows each. Each row includes the Round number, Action (W or P), number of Rows, IFEval score, and IFBench score. The highest values for IFEval (0.6776) and IFBench (0.3996) are bolded in Round 9.
Peak IFEval of $0.6776$ is reached at round 9 and never exceeded in the subsequent 23 rounds.
Table 1 shows the IFEval column peaking at $0.6776$ in round 9, while all later entries are lower.
Replay round 24 attains IFEval $0.6360$, a drop of $0.0416$ relative to the peak.
Table 1 lists $0.6360$ for round 24 (action P) compared with the peak $0.6776$ at round 9.
The table lists various sources of data, their corresponding byte counts, and their percentage share of the total.
Two rounds merit special attention. Round 15 repeats the exact recipe of round 9, showing that accidental convergence can occur. Rounds 19–24 exhibit a mismatch between the agent’s textual description and the actual recipe changes, which is why actions are labeled by the difference between consecutive recipes.
Experiment Details
Ablation results show analysis cues boost accuracy and synthesis improves many capabilities.
We first evaluate the impact of the analysis subagents by altering the cue wording and cue order while keeping the rest of the run identical.
The synthesis subagent, which proposes full‑scale plans rather than tiny datasets, consistently improves capabilities that depend on diverse content such as repetition, sentence count, word count, and comma avoidance, each backed by dozens of evaluation questions.
**Figure 16.** Per-capability comparison of the two runs of Section 5.3.
Limitations and Scaling
This section details the scaling limits of AutoSFT as revealed by the additional tables.
The reported results rely on a single 1.7 B‑parameter base model, so the observed gains may not transfer to larger scales. Moreover, each configuration was evaluated with only one search run, which bounds checkpoint noise but not the variability across independent searches.
Scaling AutoSFT to larger models or broader domains remains an open challenge; the current implementation’s gains are tied to a single 1.7 B base and limited search budget.
Implementation Details
Implementation details: how the meta‑agent organizes capabilities, stores round history, defines skills, and enforces synthesis constraints.
The meta‑agent constructs a capability taxonomy to choose data per round. It begins with the benchmark’s published constraint types for instruction‑following, then expands the taxonomy when observed failures reveal missing groups. When a benchmark offers no grouping, the agent derives leaves directly from failure patterns, each leaf encoding a concrete requirement (e.g., “answer without a comma”).
Round history $H_t$ stores more than recipes and scores: the hypothesis, its verdict, the lesson learned, and unaccepted proposals. Registering the hypothesis before training forces a causal ordering, preventing post‑hoc fitting of predictions to outcomes. Verdicts are split into a directional score and a finer‑grained observation, allowing the system to detect score changes without the intended effect and vice‑versa.
A skill is a written procedure owned by a single role; skills never invoke one another, and execution follows the owning agent’s sequence. Table 3 enumerates the skill library by role, showing that the four analysis skills provide a common proposal template while differing only in their initial evidence. This shared template lets the sub‑agents disagree productively without exposing the full action space.
**Table 3.** The skill library, by the role that owns each skill.
Generation constraints require the sub‑agent to phrase each capability requirement without copying evaluation wording, and rows must be produced via model API calls rather than scripted assembly. Before delivery, rows undergo an $n$‑gram overlap check against the evaluation set to ensure they teach the capability rather than merely echo the benchmark. The sub‑agent also builds an internal verifier from the translated acceptance criteria, iteratively generating and revising candidates until the verifier is satisfied.
Resources and Infrastructure
Implementation details, resource estimates, and component breakdown for the AutoSFT experiments.
All experiments share a single implementation for agent execution, supervised fine‑tuning, and evaluation. A research agent coordinates each round and decides which data to keep, while a separate batch‑inference backend generates and reviews synthesized examples.
Each candidate recipe is fine‑tuned from the same base checkpoint; only the research history and evaluation feedback persist across rounds. Table 5 lists the settings that are identical for every run.
The comparisons differ in only a few settings, shown in Table 6. The analysis and synthesis ablations modify these settings to isolate the effect of each component.
The pipeline consists of four stages. In the Research stage the agent reads the target’s constraints, extracts quoted failures, and inspects pool rows that already satisfy adjacent capabilities. In Generation the sub‑agent designs a system prompt, produces many candidate rows, and discards any that fail validation.
Verification translates the target requirement into acceptance criteria and runs a programmatic verifier; semantic fit is checked by a model reviewer, and read‑only subagents spot‑check quality, diversity, and fit. Decontamination adds an n‑gram overlap check against the evaluation sets to ensure the batch teaches the capability rather than memorizing the benchmark.
Resource usage is estimated from the synthesis experiment, which has the largest per‑round budget of 100 M tokens. Training consumes about 0.8 GPU‑hours per round on eight H800 GPUs; evaluation adds 0.15 GPU‑hours on a single GPU, yielding roughly 130 GPU‑hours for a 20‑round run. The combined agent and generation backend processes about 2.6 × 10⁸ tokens per round, costing ≈ USD 95 per round or ≈ USD 1,900 for the full run.
Across ten distinct runs the total hardware consumption is on the order of 10³ GPU‑hours and USD 104, with variation due to hardware utilization, evaluation protocol, and agent activity. The meta‑agent retains 88.46 % of R10’s tokens and replaces the rest with 2,752 varied examples, raising strict repetition from 39.02 % to 73.78 % (+34.76). The “repeat first” directive in R18 improves repetition without boosting the best score, and extending it to all 974 examples raises strict repetition only modestly (23.78 %→25.00 %).
**Table 5.** Settings shared by every experiment.
**Table 6.** Settings that differ across the three comparisons. Within each comparison the run with and without the component share these settings, so a difference is attributable to the component alone.