EVISKILL: Grounding Skill Evolution in Replayable Evidence
Yan Zhou, Yili Wang, Yiwei Dai, Qinggang Zhang, Xin Wang
EVISKILL improves LLM agent skills by grounding revisions in replayable execution evidence rather than raw trajectory feedback.
How can LLM agents reliably accumulate and refine procedural skills from interaction experience without parameter updates?
Existing skill-evolution methods often fail because they treat execution trajectories as black-box feedback, leading to edits that overfit incidental behaviors or lack necessary execution conditions. EVISKILL addresses this by organizing evolution into Replayable Evidence Cards, which link proposed skill edits to specific, re-executable trajectory segments. The system verifies these edits through targeted replay and preserves supported changes across epochs, even when a global revision is rejected. This evidence-grounded approach consistently outperforms standard experience-driven baselines, achieving an average gain of 17.93 percentage points over agents without skills across three interactive benchmarks.
Paper Primer
EVISKILL functions as a verification-first framework for procedural knowledge. It treats skill evolution as a three-stage pipeline: extracting evidence cards from trajectories, re-executing those specific segments to validate proposed edits, and maintaining a ledger of provisional edits that can be refined over multiple epochs.
The core mechanism is the Replayable Evidence Card: a persistent record that anchors a proposed correction to the exact task context and trajectory interval that justified it. This is like a debugger for agent behavior — instead of guessing why a skill update failed, the system re-runs the specific interaction to see if the edit actually produces the intended behavioral change.
EVISKILL consistently improves task success rates over existing experience-driven skill evolution methods.
Across 18 model-dataset settings, EVISKILL achieved the best accuracy in 14 and ranked in the top two in 16. Average gain of 17.93 percentage points over NoSkill baselines.
Global rejection of a skill revision does not imply that all constituent edits are ineffective.
Analysis of provisionally retained edits showed that 72.9% of locally supported edits from rejected revisions were eventually incorporated into globally accepted skills in later epochs. Nearly 3/4 of "failed" edits were actually useful once given further refinement or validation.
Why is "replay" necessary if the edits are already derived from successful trajectories?
Trajectory-derived edits often capture incidental behaviors or omit necessary preconditions. Replay allows the system to filter out unsupported proposals and refine those that fail to produce the intended behavioral change in the target environment.
What happens to edits that don't pass the initial validation check?
Instead of being discarded, these edits are stored in a Provisional Edit Ledger. They remain available for cross-epoch refinement, allowing subsequent experience to potentially validate or correct them as more data accumulates.
For researchers building agentic systems, this paper demonstrates that skill evolution should be treated as a persistent, evidence-grounded process rather than a one-shot distillation of trajectory data. The ability to selectively retain and re-verify constituent edits is key to stable performance gains.
Introduction and Motivation
We expose why current experience‑driven skill evolution is unreliable and motivate an evidence‑grounded alternative.
Current skill‑evolution methods treat each interaction as a fleeting lesson: they extract edits from trajectories but discard the surrounding evidence, and they rely on a single global validation verdict that can reject useful local corrections or accept spurious ones. This mismatch between local, context‑dependent feedback and the need for durable, generalizable skill updates makes the evolution process brittle and prone to overfitting.
When edits are derived from isolated experiences without preserving the execution context that justified them, the resulting skill updates can either overfit a single failure or miss essential conditions, leading to unstable and non‑generalizable behavior.
**Figure 1.** Experience-driven Skill Evolution vs Evidence-grounded Skill Evolution.
The shift from blind experience accumulation to evidence‑grounded verification is the key to making skill evolution reliable.
Reliability of Skill Revision
We expose reliability gaps in existing skill‑revision pipelines.
Our preliminary analysis uncovers two failure modes: (1) edits derived from execution traces often fail to produce their intended behavioral changes, and (2) a revision rejected by global validation can still contain edits that are individually beneficial but are masked by other harmful edits.
Section 2.1 evaluates the behavioral impact of edits generated by Trace2Skill on three interactive benchmarks. Re‑executing the edited skills reveals a spectrum of outcomes—some edits boost task success, others leave performance unchanged, and a few degrade it.
**Figure 2.** Re-execution outcomes of Experience-Derived Edits.
Section 2.2 probes the information hidden in revisions that fail overall validation. By selectively preserving edits from such rejected revisions, we observe consistent performance gains on the same benchmarks, demonstrating that useful edits can survive a failed revision.
**Figure 3.** Performance of selectively retained edits from globally rejected revisions.
An Action‑Agent baseline pairs a fixed policy (the “action”) with a learned observation model (the “agent”) to evaluate how well a skill can be executed without any revision.
The EVISKILL Framework
The EVISKILL pipeline turns execution evidence into verified skill edits.
The previous section highlighted that skill updates often lack reliable grounding. EVISKILL addresses this by recording execution evidence in reusable cards and verifying edits through replay before they become permanent.
A card is a compact record that ties a proposed skill correction to the exact slice of past execution that motivated it, enabling targeted replay for verification.
Step 1: Store Card 1 = $\langle 1,\;G_1,\;\text{add safety check}\rangle$.
Step 2: Store Card 2 = $\langle 2,\;G_2,\;\text{retry on failure}\rangle$.
Step 3: Both cards are added to the evidence pool $C_e$ for the current epoch.
Cards capture just enough context to replay the exact failure scenario without keeping the whole trajectory, which is crucial for scalable verification.
How does a Replayable Evidence Card differ from a standard replay buffer?
Standard buffers store raw transition tuples for off‑policy learning, while a Replayable Evidence Card stores a high‑level correction together with the minimal trajectory segment that justifies it, enabling focused verification rather than generic sampling.
The editor groups compatible cards into windows and asks a language model to turn each window into a concrete edit that can be applied to the skill.
Why not just fine‑tune the skill with behavior cloning instead of synthesizing edits?
Behavior cloning learns a new policy from scratch, discarding the structure of the existing skill. Evidence‑Grounded Edit Synthesis makes localized modifications to the current skill, preserving what already works while targeting the precise failure identified by the cards.
Each candidate edit is replayed in the exact context captured by its cards; the evaluator decides whether the edit fixes the problem, needs refinement, or should be rejected.
Why is replay needed instead of relying solely on validation performance?
Validation measures overall skill quality and can hide rare but critical failures. Replay checks the edit against the precise context that motivated it, guaranteeing that the correction actually resolves the identified issue.
Collect one trajectory per training task using the current Working Skill $S_W^{e}$.
Extract Replayable Evidence Cards $C_i$ from the trajectories with the evidence extractor $f_{\text{ext}}$.
Group active cards into Evidence Windows $B_k$ and synthesize candidate edits $U_e$ with the editor $f_{\text{edit}}$.
Replay each edit $u\in U_e$ in its supporting contexts $G(u)$ and evaluate with $f_{\text{eval}}$, obtaining decisions $r_u$.
Form the verified edit set $U^{+}$ by accepting successful edits and refining reflected ones.
Apply $U^{+}$ to the Working Skill to obtain the candidate revision $S_e^{\text{cand}}$.
Validate $S_e^{\text{cand}}$ on the validation set; if it improves performance, update the Validated Skill $S_V^{e+1}$, otherwise perform post‑rejection replay and refine edits.
Update the evidence pool, carry forward active cards, and proceed to the next epoch.
**Figure 4.** Overview of EVISKILL. Current trajectories and cross-epoch comparisons provide evidence for edit synthesis. Replay verifies and refines edits, while global validation and post-rejection replay govern skill updates, provisional edit retention, and evidence reuse across epochs.
Algorithm 1 – Overall procedure of EVISKILL
By tightly coupling evidence extraction, edit synthesis, and replay verification, EVISKILL ensures that every skill change is both grounded in concrete experience and rigorously checked before becoming permanent.
Experimental Results
EVISKILL refines skills by replaying edits against recorded execution evidence.
EVISKILL improves skill evolution by grounding proposed edits in Replayable Evidence Cards, allowing the agent to verify and refine edits through replay before committing them.
EVISKILL achieves the highest task‑success accuracy in 14 of 18 model‑dataset settings, delivering an average gain of 17.93 percentage points over the NoSkill baseline.
Table 4 (main results) shows EVISKILL leading in 14/18 settings and the reported average gain.
Removing the Replay‑Guided Edit Verification component drops average accuracy by 2.49, 4.56, and 2.85 pp on ALFWorld, AppWorld, and ScienceWorld respectively, confirming that replay is essential for turning proposed edits into effective behavioral corrections.
Disabling Cross‑Epoch Evidence Propagation also harms performance, with the largest drop on AppWorld (from 89.68 % to 85.52 %). This shows that preserving supported edits across epochs enables later refinements that would otherwise be lost.
Replay filtering reveals that 20.7 %–42.5 % of candidate edits are initially unsupported, yet many of these are recoverable: 13 of 16 reflected edits on AppWorld and 49 of 54 on ScienceWorld are refined into accepted edits after replay feedback.
**Table 1.** Task-success accuracy (%) across three interactive benchmarks. $\Delta_{NS}$ denotes the percentage-point change relative to No skill for the same model and dataset. Bold and underlined values indicate the best and second-highest distinct accuracies within each model–dataset setting.
**Figure 5.** Component ablations averaged over the three GPT backbones.
**Figure 6.** Replay-guided edit verification. (a) Initial decisions over all candidate edits. (b) Refinement outcomes among edits initially assigned to reflection, showing counts and within-group percentages. Acceptance indicates local replay support, not global revision acceptance.
**Figure 7.** Promotion of the 96 provisionally retained edits.
**Figure 8.** Number of distinct epochs in which each Evidence Card is selected for an Evidence Window.
**Table 4.** Task-success accuracy (%) across three interactive benchmarks. $\Delta_{NS}$ denotes the percentage-point change relative to No skill for the same model and dataset. Bold and underlined values indicate the best and second-highest distinct accuracies within each model–dataset setting.
**Figure 9.** Family-level component ablations across all six Action-Agent backbones. Each panel reports task-success accuracy averaged over the three backbones in the corresponding model family. w/o Replay removes Replay-Guided Edit Verification, while w/o Cross-Epoch disables Cross-Epoch Evidence Propagation. (a) Average results across the three GPT backbones. (b) Average results across the three Qwen backbones.
EVISKILL consistently outperforms baselines across all benchmarks.
Related Work
We situate EVISKILL among prior work on agent skills, experience memory, and skill evolution.
Agent skills provide reusable procedural guidance that LLM agents can load without modifying model parameters, covering tool use, action sequences, constraints, and recovery steps (Wang et al., 2023; Liang et al., 2026; Zhou et al., 2026a;b). Prior work embeds scripts or resources to support embodied interaction, web navigation, and coding workflows (Chen et al., 2026b; Gao et al., 2026b). Empirical studies show that a skill’s benefit depends on its ability to help the agent execute tasks effectively (Li et al., 2026a; Zhang et al., 2026d; Han et al., 2026; Zhong et al., 2026; Liu et al., 2026b).
Experience memory methods let agents reuse past interactions through verbal reflections, extracted lessons, and stored workflows (Shinn et al., 2023; Zhao et al., 2024; Wang et al., 2025; Park et al., 2023; Zhou et al., 2026a; Zhong et al., 2024; Zheng et al., 2024). Procedural and reasoning memory approaches further organize experience into retrievable guidance that can be updated during later tasks (Fang et al., 2026; Cao et al., 2026; Ouyang et al., 2026; Zhang et al., 2026b; Mi et al., 2026). These works illustrate how execution histories can inform future decisions across an interaction lifecycle.
Skill‑evolution research leverages execution experience to revise skills, e.g., extracting trajectory‑local patches (Ni et al., 2026), using failures to guide revisions (Yu et al., 2026; Liu et al., 2026c;a), and converting diagnoses into textual update signals (Wang et al., 2026). Methods such as EvoSkill, SkillOpt, and SkillGrad iteratively evaluate candidates, retain rejected edits as feedback, and accumulate recurring patterns (Alzubi et al., 2026; Yang et al., 2026a). Collectively, these approaches motivate grounding skill edits in concrete evidence before committing them.
Preliminary Study Analysis
Ablation analyses of edit outcomes and a full benchmark reveal EVISKILL’s accuracy gains.
We first examine edits generated by Trace2Skill on training trajectories from ALFWorld, AppWorld, and ScienceWorld.
Each edit $u$ is evaluated in isolation by applying it to the preceding skill version and rerunning the associated task set $T(u)$; an edit is labeled **accept** if task‑success accuracy rises, **reject** if it falls, and **reflect** if it stays unchanged.
Next we probe revisions that were rejected at the validation stage.
For each rejected revision we let GPT‑5.5 pick up to three promising edits; applying those edits together to the prior validated skill yields a measurable training‑accuracy gain despite the whole revision’s failure on the validation set.
EVISKILL improves task‑success over the No Skill baseline by +5.97 percentage points on ALFWorld.
Table 4 shows EVISKILL at 96.27 % versus No Skill at 90.30 %.
Extended Results
EVISKILL’s gains extend across backbones, ablations, and efficiency metrics.
Full EVISKILL reaches 88.06% accuracy on ALFWorld, the highest among all variants.
Table 5 shows Full at 88.06% versus 86.32% (Cards Only), 86.82% (Edits Only), and 84.58% (w/o Cross‑Epoch).
Across the six backbones, EVISKILL consistently tops the leaderboard on ALFWorld and ScienceWorld, while on AppWorld it ties or leads in three out of four settings, confirming robustness to backbone choice.
**Figure 10.** Replay-supported correction of origin edit lineages.
Ablation (Table 5) reveals that removing Replay hurts GPT backbones more (≈4.5 pp loss) whereas removing Cross‑Epoch harms Qwen backbones more (≈3.5 pp loss), indicating complementary contributions.
**Table 5.** Cross-epoch ablations.
**Table.** Statistics regarding the dataset, support type, number of cards, edit-linked cards, replay-retained cards, final provisional cards, validated cards, and the ratio of provisional to validated cards.
**Table 7.** Supporting trigger-range statistics.
**Figure 11.** Cross-epoch lifecycle of Evidence Cards over four skill-evolution epochs. (a) Lifecycle-state shares among Cards processed from each epoch's input Evidence Pool, pooled across 18 runs; $n$ gives the input pool size. (b) Distribution of unique Evidence Cards by the number of distinct epochs in which they are assigned to an Evidence Window. Pre-resolved denotes Cards resolved without any Evidence-Window assignment. Percentages are computed within each benchmark, and All is the count-weighted aggregate.
**Figure 12.** Post-rejection replay and subsequent edit incorporation.
Per‑dataset gains are 3.77 pp (ALFWorld), 1.99 pp (AppWorld), and 3.01 pp (ScienceWorld), each surpassing the baselines.
Case Studies and Efficiency
Four case studies show how replay‑guided verification evaluates candidate edits.
We evaluate four representative outcomes—accept, reject, reflect → accept, and reflect → reject—by replaying candidate edits in their original execution contexts.
**Figure 13.** **Acceptance.** The Evidence-supported search edit changes behavior on the bound replay range, discovers the previously missed target, and completes the task, leading to accept decision.
Applying the search‑order edit eliminates repeated empty‑container checks and yields successful fork placement.
Figure 13 demonstrates the agent’s corrected behavior and the score increase.
**Figure 14.** Rejection due to behavioral noncompliance. Replay skips the comparison required by the edit, resulting in `reject`.
The complete‑comparison edit does not change the agent’s behavior; the score remains 0.0.
Figure 14 shows the unchanged trajectory and persistent failure.
**Figure 15.** Reflection followed by acceptance. Initial replay exposes a missing exact-instance binding requirement in an otherwise useful placement rule. Reflection repairs this omission, and the revised edit subsequently completes the placement successfully, resulting in an accept decision.
Adding an exact‑instance requirement to the placement edit fixes the bee‑placement failure.
Figure 15 records the revised replay achieving full success.
**Figure 16.** Reflection followed by rejection. Reflection removes the conflicting life-stage heuristic and repairs the written lifespan rule, but the corrected instruction remains ineffective on one replay range. The revised edit is therefore not retained.
Even after removing the conflicting heuristic, range B fails, causing the edit to be rejected.
Figure 16 highlights the persistent error in range B.
Across four distinct tasks, a single Evidence Card links replayable trigger ranges that share the rule “maintain the exact acquired instance through transformation and final placement.”
Table 10 lists the four tasks and the shared rule, demonstrating how one card consolidates reusable behavioral guidance instead of four independent task‑specific rules.
In a cross‑epoch scenario, a tracked edit that only checks download status repeatedly removes liked albums. Replay‑guided reflection adds an “or” condition, producing a revised edit that passes verification and is later incorporated into the validated skill set.
Experimental Setup
Details of datasets, task splits, and baseline methods used in experiments.
We evaluate on three interactive benchmarks—AppWorld, ScienceWorld, and ALFWorld—each targeting a distinct domain of procedural interaction.
**Table 2.** Summary of task splits for AppWorld, ScienceWorld, and ALFWorld datasets.
AppWorld offers a controllable API environment where success requires planning multi‑step calls and maintaining intermediate states.
ScienceWorld focuses on scientific experimentation, demanding manipulation of objects, interpretation of feedback, and recovery from intermediate failures.
ALFWorld is a text‑based embodied setting for household tasks, evaluating agents on navigation and object interaction through textual actions.
Baselines are split into non‑evolving approaches (NoSkill, LLM Skill) that do not update skills, and skill‑evolution methods that refine skills from execution experience.
**Table 3.** Overview of the evaluated methods and their skill-evolution strategies.
NoSkill runs without any external skill document, whereas LLM Skill supplies a fixed initial skill S₀ that never changes.
Trace2Skill, EvoSkill, SkillGrad, SkillOpt, and EVISKILL each update skills using execution feedback, with EVISKILL uniquely attaching proposed edits to Replayable Evidence Cards for verification before incorporation.
Methodological Details
Details the EVISKILL algorithm, evidence cards, replay process, and implementation specifics.
The algorithm maintains a validated skill $S\!V_e$ and a set of tracked edits $Q_e$ across epochs. At each epoch it builds a working skill $S\!W_e$, extracts evidence cards, and synthesizes candidate edits.
EVISKILL evolution loop – high‑level pseudocode.
Replayable Evidence Cards $C_i=\langle z_i, G_i, u_i\rangle$ store a persistent identifier $z_i$, a set of trigger ranges $G_i$, and the proposed correction $u_i$. Cards are filtered by a grounding verifier before entering the Evidence Pool $\mathcal{C}_e$.
Trigger‑Range Replay loads the recorded trajectory prefix up to step $s$, reconstructs the environment state, and then executes the candidate edit $u$ from step $s$ onward. The replay evaluator returns one of three feedback labels: accept, reflect, or reject.
Edits that survive replay are consolidated; accepted edits become part of the validated skill, reflected edits are rewritten and re‑evaluated, and rejected edits are discarded unless later rescued by post‑rejection replay. The lifecycle tracks cards as active, protected, archived, or stale.
Implementation uses four skill‑evolution epochs, HDBSCAN clustering with $\text{min\_cluster\_size}=4$ and $\text{min\_samples}=2$, and a batch size of 32 for the Qwen3‑Embedding encoder. Each Evidence Window contains 4–20 cards, and at most 128 edits are considered per epoch.
Prompt Templates
Prompt templates for the five EVISKILL phases are provided in their original Markdown format.
This appendix lists the five Markdown‑formatted prompt templates that drive the three EVISKILL phases: extracting Replayable Evidence Cards, verifying card grounding, synthesizing evidence‑grounded edits, evaluating edits via replay, and extracting contrastive evidence.
**Figure 17.** Cross-epoch evidence reveals a persistent edit deficiency, guiding correction through reflection and replay.
H.1 REPLAYABLE EVIDENCE CARD EXTRACTION parses trajectories to produce cards ⟨i, $G_i$, $d_i$⟩ that link a proposed correction $d_i$ to its trigger ranges $G_i$.
Template for extracting a Replayable Evidence Card (phase 1).
H.2 EVIDENCE CARD GROUNDING VERIFICATION checks whether a card’s cited trajectory spans substantiate its proposed edit.
Template for grounding verification (phase 2).
H.3 EVIDENCE‑GROUNDED EDIT SYNTHESIS merges compatible cards into concrete markdown edits, preserving card identifiers.
Template for edit synthesis (phase 3).
H.4 REPLAY‑GUIDED EDIT VERIFICATION evaluates a candidate edit by replaying the skill with the edit applied.
Template for edit verification (phase 4).
H.5 CONTRASTIVE EVIDENCE CARD EXTRACTION compares pre‑ and post‑edit trajectories to surface persistent deficiencies.
Template for contrastive evidence extraction (phase 5).