Aspire: Can Models Self‑Evolve from Vague Goals?

Yuhao Wu, Jingyuan Zhang, Jiajun Shi, Yuxuan Zhang, Xinping Lei, Junting Zhou, Zexuan Wang, Yuchen Wu, Huan Zhou, Duo Wang, Yinzhu Piao, Yongchang Peng, Yunfeng Shi, Jin Chen, Zuo Wang, Jinkai Liu, Jiaheng Liu, Wenxuan Zhang, Shen Yan, Wenhao Huang, Ge Zhang

Aspire formalizes self-evolution from vague goals, revealing that agents can complete training loops but rarely achieve retained capability gains.

Can LLMs effectively improve their own capabilities when provided with only vague, high-level goals rather than explicit task instructions?

Existing self-evolution benchmarks provide agents with explicit tasks and rewards, effectively reducing "learning" to simple optimization. In practice, however, humans must first translate broad, vague goals into concrete objectives and validation criteria before any training can begin. Aspire introduces a benchmark where agents receive only a natural-language capability goal, forcing them to operationalize the goal, construct their own training data, and manage validation without seeing the hidden evaluation items. Experiments show that while agents routinely complete the engineering loop of data selection and training, they struggle to produce retained improvements, often specializing to narrow proxies or regressing on base capabilities.

Paper Primer

Aspire treats goal operationalization as the primary bottleneck in autonomous post-training. The system acts as a controller that hides the evaluation set from the agent, forcing the agent to build its own validation signals and training plans from a broad capability direction.

Vague-goal self-evolution rarely results in retained capability gains over the base model.

In 24 final-only runs across six goals, only one model-goal pair exceeded its base score; in adaptive-feedback settings, most trajectories that show local progress remain below the initial base performance. Only 3 of 24 final-only checkpoints outperformed their base model, with most agents failing to preserve base performance after training.

The authors find that vague goals redirect search effort toward goal interpretation, increasing the density of tool calls related to task materials and proxy evaluation scripts compared to explicit-task settings. However, this increased "operationalization" work does not translate into higher aggregate performance on hidden evaluation sets.

Why does this paper distinguish between "closing the training loop" and "closing the capability loop"?

The authors observe that agents can successfully execute the mechanical steps of training—selecting data, running updates, and generating checkpoints—without actually improving the model's performance on the intended capability, often because they optimize for narrow, flawed proxies.

How does the agent's harness evolution perform compared to human-engineered baselines?

Even the strongest successor harness generated by an agent remains numerically below the reference Qwen-Agent, indicating that current agents struggle to improve the runtime logic, tool use, and validation procedures that mediate their own performance.

Introduction and Motivation

We expose the gap between explicit task‑tuning and vague‑goal‑driven self‑evolution.

Human learners routinely start from a broad capability direction—e.g., “become a better physicist”—and must decide what to improve, how to improve it, and how to verify progress. By contrast, most LLM self‑evolution work assumes an explicit task, metric, and evaluation script, reducing the problem to optimizing a fixed objective. This leaves a critical gap: agents must interpret vague goals, diagnose capability gaps, construct learning signals, and assess genuine improvement without any externally provided task definition.

It is the process by which an agent turns a high‑level, natural‑language capability direction into concrete learning objectives, data selections, and self‑evaluation signals.

**Figure 1.** From explicit-task optimization to vague-goal-driven self-evolution. (a) With an explicit task definition, humans specify the task format and composition, difficulty anchor, metric, and success criterion, leaving the agent to search primarily over how to improve a fixed objective. (b) With only a broad capability direction, the agent must diagnose capability gaps, decompose sub-goals, construct learning and validation signals, and thereby decide what to optimize as well as how. Official evaluation items remain sealed; external evaluation tests whether gains on an agent-built proxy transfer to the intended capability.

The shift from explicit task‑tuning to goal‑driven self‑evolution forces agents to solve the missing operationalization step.

The ASPIRE Benchmark

The ASPIRE benchmark defines a hidden evaluation and minimal toolset for studying self‑evolution.

Measuring self‑evolution requires an external signal that the agent cannot directly manipulate. ASPIRE keeps the evaluator on the controller side and hides the benchmark definition, items, and reward from the agent, so progress is observable without exposing a training task.

ASPIRE provides a sealed evaluator that returns only a bounded aggregate score, preventing the agent from seeing the underlying tasks while still offering a measurable signal of capability improvement.

Goal G₁ is issued; the agent proposes a new dataset and fine‑tunes a model checkpoint.

The controller evaluates the updated model on the three hidden items for G₁ and returns an aggregate score of 0.62.

Goal G₂ is issued; the agent edits its inference harness and submits the edited version.

The controller evaluates the harness on its three hidden items for G₂ and returns an aggregate score of 0.48.

The agent never sees the individual item scores or rubrics; it must decide whether the observed aggregate improvements justify further updates.

How does ASPIRE differ from a conventional benchmark?

In a conventional benchmark the agent can inspect task definitions, item examples, and per‑item scores. ASPIRE hides all task‑level details and only reveals a single aggregate number, forcing the agent to learn from sparse outcome feedback rather than direct supervision.

PostTrainBench is the sealed, task‑specific evaluator used in RQ1; it preserves the original benchmark while replacing the explicit task identifier with a vague goal.

Why keep the original benchmark hidden when using PostTrainBench?

Hiding the benchmark prevents the agent from tailoring its updates to the known test items, ensuring that measured progress reflects genuine capability growth rather than overfitting to a disclosed task.

ASPIRE separates what is being evolved (model weights $M$ or harness $H$) from the decision model $D_r$ that directs the search, allowing experiments to focus on either component while keeping the controller fixed.

Candidate $M_1$ is trained on a new dataset; the controller evaluates it and returns an aggregate score of 0.55.

Candidate $M_2$ is trained on a different dataset; the controller evaluates it and returns an aggregate score of 0.48.

The controller selects $M_1$ (higher score) as the retained checkpoint for the next round.

Because the harness stays unchanged, any score difference can be attributed solely to the weight updates.

What would happen if the decision model $D_r$ changed mid‑round?

Changing $D_r$ mid‑round would break the fixed‑controller assumption, making the evolution surface ill‑defined; the protocol requires $D_r$ to remain constant throughout a round to ensure that observed score changes are comparable.

**Table 1.** Evolution surfaces in ASPIRE. The component under study changes while $D_r$, the controller, and the evaluator remain fixed within the round.

**Figure 2.** Construction and information boundary of the hidden evaluation set. **Top:** expert-authored candidates pass independent review, blind difficulty screening, overlap and duplication audits, scorer binding, and immutable versioning. **Bottom:** 520 evaluation items cover six goals. Items and judging assets remain controller-only; the protocol exposes no item-level results and only the aggregate scores it permits.

**Figure 3.** The minimal interactive environment and outcome protocols. Weight candidates follow a data–train–verify path, whereas harness candidates follow an edit–freeze path. A unified agent tool exposes composable data, training, status, and validation actions on the agent’s own data; the controller maps them to managed backends and enforces budget and lineage constraints. Evaluation returns only the aggregate outcome permitted by the protocol. Across rounds, the controller may reuse the current decision model or explicitly promote a verified trained descendant; any handoff begins only in the subsequent round.

**Table 3.** The six vague goals and their corresponding evaluation items. All 520 top-level items are newly authored by domain experts.

RQ1: Vague Goals vs. Explicit Tasks

Vague goals reshape outcomes and search trajectories versus explicit‑task baselines.

RQ1 examines how vague goals reshape post‑training performance and the underlying search process compared with explicit‑task baselines.

Vague‑goal final scores are lower overall, dropping roughly 6 points compared with explicit‑task references.

Claude Opus 4.8 vague‑goal 27.07 vs 32.90 reference; GPT‑5.6 vague‑goal 29.58 vs 36.23 reference.

**Figure 4.** RQ1 final-score and trajectory comparison. (a) Official PostTrainBench references for Claude Opus 4.8 Max and GPT-5.6, alongside vague-goal scores across seven benchmarks. (b) Within matched Claude Opus 4.8 pairs, vague-goal trajectories show more decision and GPU-idle time and less active GPU work. (c) Claude Opus 4.8 under vague-goal prompting obtains less evaluation feedback per training start than official GPT-5.6, while official Claude Opus 4.8 Max has nearly the same feedback density. Panel (c) is a descriptive cross-system reference, not a matched prompt effect. Exact values and accounting appear in Appendix B.

RQ2: Iterative Self-Evolution

Iterative Self‑Evolution yields occasional above‑base gains but most checkpoints regress.

Both RQ2 settings start from instruction‑tuned checkpoints, called base models, whose initial evaluation defines a base score. An update must add goal‑relevant behavior without erasing the useful capabilities already encoded.

The model repeatedly trains a descendant, evaluates it on a vague goal, and feeds the aggregate score back to decide the next update.

How does Iterative Self‑Evolution differ from ordinary fine‑tuning on a fixed dataset?

Fine‑tuning optimizes a static loss on a static dataset, whereas Iterative Self‑Evolution closes the loop: after each training burst the model’s own aggregate score determines whether the next burst proceeds, making the training trajectory itself a decision variable.

Iteration 1 trains a descendant; its aggregate score rises to 41.2 → accepted.

Iteration 2 starts from the accepted descendant, trains another 5 h; score drops to 39.5 → rejected, keep previous 41.2.

Iteration 3 repeats from the 41.2 checkpoint; score climbs to 42.0 → accepted as final model.

The loop can recover from regressions (iteration 2) while still making net progress, illustrating why sparse feedback alone does not guarantee monotonic improvement.

The final‑only protocol runs two independent terminal trainings per model–goal pair and reports the arithmetic mean of the two final checkpoints (Avg@2). No intermediate checkpoints are evaluated, so the method tests open‑loop execution.

The adaptive‑feedback protocol keeps the decision model fixed while training multiple descendants, using sparse aggregate scores to decide when to evaluate or branch.

Final‑only runs that continue for 16–20 training jobs consume 26.57–32.29 GPU‑hours yet yield negligible scores (e.g., 0.159 on 4B mathematics). Without intermediate evaluation, the trajectory cannot reveal regressions that later checkpoints hide.

We also observe a numeric‑label SFT failure mode: Self trajectories concentrate on a single answer format, collapsing diversity and limiting transferability.

Answer to RQ2: isolated above‑base goal scores appear, but reliable improvement is absent. Agents reliably close the training loop, yet most checkpoints fail to preserve the base performance.

**Figure 6.** Scores on the hidden evaluation set for all 30 adaptive-feedback configuration–goal cells: two Self configurations and three external decision-model settings. Large values are the scores of the best evaluated checkpoints; small values give the base score. The two cells without an evaluated checkpoint show the fallback base score for accounting but are hatched and labeled as having no checkpoint. For evaluated cells, color encodes the best evaluated checkpoint score divided by the base score. Printed values preserve the original score scale.

**Figure 7.** Adaptive-feedback outcome and trajectory decomposition. Of 30 cells, 28 produce an evaluated checkpoint, 21 produce an eligible checkpoint, and one yields a retained improvement (a). Of 62 consecutive within-round checkpoint transitions, 28 increase (b), but even a monotonic self-optimization trajectory can remain far below the base score (c).

**Figure 8.** Adaptive-feedback score-compute trajectories for four configurations using the same Qwen3.5-4B base model. Scores are normalized to each goal's base score, and curves connect evaluated checkpoints by settled training GPU-hours.

RQ3: Harness Evolution and Conclusion

One‑step harness evolution yields distinct successors but none surpass the fixed reference.

The central premise is that agents should interpret vague, high‑level goals and drive their own improvement without explicit task definitions.

A harness is the editable “wrapper” that translates a vague goal into concrete prompts, tool calls, and post‑processing; evolution means generating a new wrapper $H_1$ from an existing one $H_0$ while keeping the underlying model frozen.

How does harness evolution differ from standard fine‑tuning of the model?

Fine‑tuning updates the model weights $\theta$, altering its internal representations. Harness evolution leaves $\theta$ untouched; only the surrounding wrapper $H$ changes, so any gain or loss is attributable to the wrapper’s logic rather than learned parameters.

One‑step harness evolution does not exceed the fixed reference harness on the hidden academic‑writing evaluation.

Table 2 shows the best successor (GPT‑5.6 Sol) scoring 27.22 / 25.97 versus the reference 28.64 / 27.65.

**Figure 5.** Final-only mean change from the base score before rollback. Each point is the arithmetic mean of two separately executed terminal runs for one model-goal pair. Scientific and academic reasoning with Qwen3.5-9B is the only two-run mean above its base score. Constituent run scores appear in Appendix C.3.

Harness evolution can yield distinct behavior without weight updates, but current one‑step methods still lag behind a fixed reference.

Appendix A: Benchmark Construction

Details how the ASPIRE benchmark’s goals, data hygiene, and prompt contracts are constructed.

Section A.1 lists six vague goals—scientific reasoning, humanities knowledge, health reasoning, mathematical reasoning, a composite logic‑reliability‑instruction goal, and academic writing—each mapped to a mutually exclusive group of evaluation items. The table shows the item counts (75, 110, 100, 126, 89, 20) summing to 520 unique top‑level items across six non‑overlapping groups.

A.2 describes the contamination‑control pipeline: domain experts author every item from scratch, then the system removes exact and semantic duplicates within and across reporting groups and audits overlap against public benchmarks. A second overlap gate during dataset registration rejects any match before the hidden set enters SFT, GRPO, or continual‑pretraining, after which the set is frozen under a SHA‑256 manifest.

A.3 specifies prompt contracts. Prompts are generated from a versioned setting profile; a vague‑goal prompt includes the high‑level capability, model identifier, action contract, budgets, and a rule that only aggregate scores are returned. An explicit‑task contract additionally reveals task format, difficulty anchor, metric, and success criterion. Every campaign records the setting ID, prompt‑profile version, rendered prompt SHA‑256, and experiment ID, and these bindings are verified so a vague‑goal run cannot silently resume under a different contract.

Appendix B: Score and Trajectory Accounting

Table 4 reports RQ1 scores for official and vague‑goal models across benchmarks.

Table 4 gives the exact percentages visualized in Figure 4(a); the official columns are system‑level PostTrainBench references, while the vague‑goal columns are the results of our approach.

**Table.** Matched process accounting.

Appendix C: Extended Results

Resource accounting and detailed results for adaptive‑feedback and creator trajectories.

The appendix compiles the quantitative accounting of GPU‑hours, runs, and plans for the adaptive‑feedback and final‑only protocols, and presents the detailed performance deltas ($\Delta_{term}$) observed across the six vague‑goal categories for the Qwen3.5 models.

**Table 6.** Resource accounting for RQ2–RQ3. Training GPU-hours exclude inference, judging, deployment, and idle controller time.

The table presents performance metrics for Qwen3.5-4B and Qwen3.5-9B models across six categories of "Vague goal," comparing a "Base" score against two experimental runs ("Run A" and "Run B"), their "Mean," and the difference from the base score ($\Delta_{term}$).

Across the 30 configuration‑goal cells in the adaptive‑feedback protocol, 28 produced evaluated checkpoints, yet only two surpassed their base scores: Qwen3.5‑4B Self science (44.00 → 45.33) and Terra‑optimized Qwen3.5‑4B mathematics (17.86 → 20.10). The latter is the sole retained improvement under the complete‑run selection rule.

Search profiles reveal that eight of the 22 multi‑checkpoint trajectories are non‑decreasing. Luna’s SFT‑driven runs settle after 89.57 GPU‑hours, Terra after 91.73 GPU‑hours, and Sol after 76.56 GPU‑hours, yet only Terra’s mathematics trajectory yields a checkpoint exceeding the base score.

Self‑directed proxy choices differ: Qwen3.5‑4B imports GSM8K or Hendrycks mathematics data in 30 of 32 dataset‑import events, while Qwen3.5‑9B imports mathematics in 11 of 15 events and relies more on goal‑specific data. Update mechanisms also diverge, with Qwen3.5‑4B using GRPO in 4 of 12 plans and Qwen3.5‑9B employing SFT in 9 of 11 plans.

The trajectory evidence boundary (C.5) notes that the portable adaptive‑feedback artifact records extensive event logs but omits hidden evaluation items and per‑item judge reasons, limiting causal analysis to aggregate checkpoint progression.

Compute accounting (C.6) reiterates that training GPU‑hours exclude inference, judging, deployment, and idle controller time, emphasizing that the final‑only protocol alone consumes 59.269 GPU‑hours for the 4B model and 89.967 GPU‑hours for the 9B model.

Creator‑trajectory case analysis (RQ3) examines three GPT‑5.6 creator runs. Luna’s early‑stopping harness improves structural completeness but reframes technical queries; Terra’s reviewer reversion removes a final‑answer invariant, causing sharp score drops; Sol’s broader testing adds a completeness check that prevents empty final responses without improving scores.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers