DART-SD: Diamond-Topology Aware Retrieval and Tuning for Self-Distillation of Multi-Turn Tool-Calling Agents

Hangrui Xu, Jiarui Wang, Yang Yang, Chuanbo Zhu, Fangda Chen, Ziqi Wu, Jingming Cai, Yan Song

DART-SD improves agent tool-calling by replacing global trajectory imitation with topology-guided localized recovery.

How can we improve multi-turn tool-calling agents by distilling only the critical decision points in a trajectory rather than the entire sequence?

Current agent training forces models to imitate full, linear trajectories, which causes "topological collapse" when tasks involve order-independent sub-goals that naturally form a diamond-shaped graph of valid paths. DART-SD models these tasks as an Interaction-State Transition Graph (ISTG) and identifies a "Critical Topological Breakpoint" where a student model deviates from success-reachable states. It then performs localized supervision only on the recovery steps, protecting the valid reasoning prefix from destructive gradient updates. This approach consistently outperforms standard supervised and reinforcement learning baselines across five benchmarks, often allowing smaller models to surpass their teachers.

Paper Primer

The core problem is that multi-turn tool-calling tasks are often order-independent, meaning multiple valid sequences can reach the same goal. Standard training treats these as rigid, linear paths; when a model explores a valid but non-imitative path, the global loss function penalizes it as an error, effectively training the model to memorize specific sequences rather than learning the underlying task logic.

DART-SD is a topology-aware distillation framework: it maps teacher rollouts into a graph of cumulative information states and uses this graph to surgically correct student errors. The method is like a GPS re-routing a driver: instead of forcing the driver back to the exact original path, it identifies the current location, finds the nearest valid intersection on the map, and provides instructions only for the remaining leg of the journey.

DART-SD achieves superior performance across diverse tool-use benchmarks compared to standard distillation and reinforcement learning paradigms.

Evaluations on FTRL, BFCL, ToolHop, $\tau$-bench, and RoTBench show consistent gains in trajectory precision and task completeness for both 4B and 8B model scales. The method enables smaller student models to outperform their teachers on benchmarks like FTRL, ToolHop, and $\tau$-bench.

Why does this approach use a graph-based representation instead of standard reinforcement learning?

Standard reinforcement learning often suffers from credit misassignment, where rewards are spread uniformly across all steps, inadvertently penalizing valid exploratory steps within failed trajectories. The graph-based ISTG allows the model to distinguish between informative state transitions and useless operations, enabling precise, localized supervision.

What is the role of the "Critical Topological Breakpoint" (CTB)?

The CTB marks the exact moment a student model's interaction state drifts outside the "success-reachable region" defined by teacher rollouts. By identifying this point, the framework can isolate the specific segment of the trajectory that requires correction, leaving the preceding valid reasoning prefix untouched by the training loss.

By shifting from global imitation to topology-guided localized correction, DART-SD enables models to learn the core logical backbone of tool-use tasks rather than just memorizing linear sequences, significantly improving generalization and efficiency.

The Problem of Trajectory Bloat

We expose why full‑trajectory imitation hampers multi‑turn tool agents and introduce DART‑SD.

Training multi‑turn tool‑calling agents by imitating entire trajectories is inefficient: it forces the student to copy redundant or erroneous steps, collapsing the rich decision space into a single brittle path.

When a task contains many order‑independent sub‑goals, the optimal solution forms a combinatorial diamond lattice; forcing a single linear trajectory discards the many valid alternatives, inflating the training signal with redundant steps.

The standard supervised fine‑tuning approach applies a global loss to every step, overwriting any useful exploration the student might perform.

Group Relative Policy Optimization distributes a single reward across the whole sequence, which misassigns credit and penalizes correct intermediate actions.

**Figure 1.** Comparison of training paradigms. (a) SFT Teacher Boosting applies an indiscriminate global loss, which overwrites valid exploration. (b) Standard-RL (GRPO) misassigns credit through a uniformly distributed reward spread. (c) DART-SD dynamically identifies the Critical Topological Breakpoints (CTB) and applies localized correction while preserving valid exploration.

Full‑trajectory imitation collapses the diamond‑shaped solution space, stifling exploration and hurting performance.

The DART-SD Mechanism

DART‑SD builds a graph of information states, finds the first irrecoverable step, and uses it to focus self‑distillation.

DART‑SD tackles trajectory bloat by refusing to train on every redundant step of a multi‑turn tool‑calling episode. Instead it isolates the exact point where a student rollout first leaves the region of teacher states that can still reach success, and concentrates learning on the recovery after that point.

Construct the Interaction‑State Transition Graph (ISTG) from all teacher rollouts.

Project each student state onto the budget‑filtered success‑reachable region to locate the first non‑projectable step (the CTB).

Condition an augmented generator on the retained student prefix and privileged teacher references to produce a recovery continuation.

Apply masked causal language modeling only to tokens generated after the CTB.

Repeat the process across self‑distillation rounds, progressively pushing the CTB later in the trajectory.

The ISTG records what factual information the agent has gathered (main nodes) and which useless tool calls it has performed (auxiliary nodes), turning a rollout into a compact graph of information acquisition.

Step 1 (teacher): $\alpha_x(e_1)=\{A\}$ → $I_1=\{A\}$, main node $X_1$.

Step 2 (teacher): $\alpha_x(e_2)=\{B\}$ → $I_2=\{A,B\}$, main node $X_2$.

Step 1 (student): $\alpha_x(e_1)=\{A\}$ → $I^s_1=\{A\}$, main node.

Step 2 (student): non‑informative $e_u$ → $\alpha_x(e_u)=\emptyset$, $U^s_2=\{e_u\}$, auxiliary node.

Step 3 (student): fails to acquire $B$, so $I^s_3=\{A\}$, auxiliary node.

The graph shows that the student diverges after the auxiliary node, creating a branch that never reaches the teacher’s $B$ atom.

How does the ISTG differ from a plain execution trace?

An execution trace records every raw tool call in order. The ISTG first collapses semantically equivalent calls into a single atom and then separates calls that add new information (main nodes) from those that do not (auxiliary nodes). This yields a compact graph that captures *what* the agent knows rather than *how* it arrived there.

We keep only those teacher states that can still reach a successful outcome within a task‑specific budget, and we align each student state to the deepest reachable teacher state that already contains the student’s information.

Student step 0: $I^s_0=\emptyset$, $\rho_0=1$ (matches $v_0$).

Student step 1: acquires $A$, $I^s_1=\{A\}$ → $A^{\text{main}}_1$ contains $v_1$, so $\rho_1=1$.

Student step 2: makes a useless call, $U^s_2=\{e_u\}$, no teacher auxiliary node with parent $v_1$ and $|U|=1$, so $A^{\text{aux}}_2=\emptyset$, $\rho_2=0$.

The first non‑projectable step occurs at $t=2$, which will be identified as the CTB.

Is the projection simply a shortest‑path test?

No. It also requires the student’s information set to be a subset of the teacher’s and matches node types (main vs auxiliary). Thus a student auxiliary node cannot be projected onto a teacher main node even if the distance is short.

CTB is the earliest step where the student can no longer be aligned with any reachable teacher state of the same type, marking the point where recovery must be injected.

Step 0: $\rho_0=1$ (matches root).

Step 1: $\rho_1=1$ (matches teacher node with $I=\{A\}$).

Step 2: auxiliary node, no matching teacher auxiliary node → $\rho_2=0$.

Thus $t_C=2$, and $a_C$ is the teacher node with $I=\{A\}$.

The CTB pinpoints the exact transition where the student first loses a same‑type anchor, providing a natural cut‑point for recovery generation.

How is a CTB different from a generic failure point?

A generic failure point merely indicates that the rollout ended unsuccessfully. A CTB is defined relative to the success‑reachable projection: it is the *first* step where the student can no longer be aligned with any reachable teacher state of the same type, making it a precise, curriculum‑driving signal.

**Figure 2.** Overview of DART-SD. (1) DART-SD constructs an ISTG from teacher rollouts, where main and auxiliary nodes model information acquisition and useless exploration. (2–3) Failed student rollouts are collected and replayed in the same interaction-state space, then projected onto the budget-filtered success-reachable region $\mathcal{R}_x^+$, where the first projectable-to-non-projectable transition defines the CTB. (4–5) Conditioned on the retained student prefix and privileged teacher references, DART-SD generates a recovery continuation after the CTB and applies localized supervision only to the generated assistant tokens. (6) This CTB-guided SFT loop is progressively repeated across self-distillation rounds.

Performance and Efficiency Gains

DART‑SD beats teachers on key benchmarks and trims tool‑call traces.

DART‑SD outperforms teacher models on three benchmarks while delivering higher overall Solve‑F1.

Table 1 shows DART‑SD achieving 45.66% Solve‑F1 versus the teacher’s 40.6% on FTRL, and surpassing the teacher on ToolHop and $\tau$‑bench.

**Table 1.** Performance comparison of different training methods on five tool-use benchmarks using Qwen3-4B and Qwen3-8B backbones. Training-based baselines are organized into distillation-based ♠ and reinforcement learning ◊ paradigms. All trainable methods are trained on FTRL and evaluated on both the in-domain FTRL test set and four out-of-domain benchmarks. The best and second-best results within each backbone are highlighted.

**Figure 3.** Performance comparison of Qwen3-8B, DART-SD, and the teacher across five tool-use benchmarks. DART-SD improves upon Qwen3-8B on all benchmarks and **surpasses** the teacher on FTRL, ToolHop, and $\tau$-bench.

**Table 2.** Average tool-call length of successful trajectories across progressive SFT iterations on the FTRL test set. DART-SD progressively improves Solve-F1 while shortening its tool traces, eventually producing more efficient traces than the golden solutions given during data construction.

Component Analysis and Robustness

We revisit the core idea: DART‑SD trims full‑trajectory imitation to the essential decision points, letting a smaller student surpass its teacher.

Recall that full‑trajectory imitation forces a model to copy redundant or erroneous steps; DART‑SD instead isolates the critical decision points where the student deviates from success.

**Table 4.** Performance comparison under the thinking setting. Results are reported using FTRL Solve-F1, the average BFCL Multi-Turn score, and ToolHop AC.

**Table 5.** General capability evaluation on representative benchmarks. The best results are highlighted.

**Table 6.** Component ablation of DART-SD on the FTRL test set using Solve-P, Solve-R, and Solve-F1.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers