The Trinity: Capability-Driven Recursive Self-Improvement for Multimodal Agents

Weidong Tang, Ruiheng Wang, Jingxuan Zhang, Hexin Wang, Haoyuan Chen, Zhengfeiyang Zhang, Yixing Ma, Zichen Jiang, Ziyu Guo, Haodong Li, Zhengbo Jiao, Zhiqing Cui, Xinxiang Yin, Shuo Liang, Zhenglin Wan, Hongzhan Lin, Wangbo Zhao Kaipeng Zhang, Pengfei Zhou, Odin Zhang, Yang You

The Trinity organizes multimodal agent evolution around shared capability requirements rather than task categories.

How can multimodal agents recursively improve their own reasoning and control capabilities by co-evolving task environments and agent "Harnesses" (modular skill libraries)?

Multimodal agents struggle to generalize because manual Harness iteration cannot keep pace with the diversity of tasks, leading to overfitting on specific benchmarks. The Trinity shifts the basis of improvement from task-specific optimization to a hierarchical capability tree, where shared requirements across heterogeneous tasks guide the evolution of reusable module configurations. This approach improves mean performance by 34 points across training benchmarks and 26 points on unseen tasks, including real-world embodied interaction.

Paper Primer

The framework operates through three roles: Creator, Explorer, and Evolver. Evolver identifies shared capability requirements from verified execution trajectories to build a hierarchical tree, while Creator generates simplified, verifiable simulated worlds for each node to focus learning on core decision processes. Explorer then evolves reusable module configurations along this tree, using selective inheritance to accumulate validated updates.

Capability-driven organization outperforms task-specific specialization.

Comparing flat task-domain grouping against the hierarchical capability tree shows that the tree structure supports more effective selective inheritance and module refinement. The capability tree achieves the highest performance on both training and unseen benchmarks while requiring fewer training environments.

Capability-aware routing effectively matches tasks to specialized Harness configurations.

An agent acting as a Router uses public task information and verified execution experience to select the best-fit branch of the Harness tree. Full self-routing scores 71.27 on training and 60.96 on unseen benchmarks, approaching the Oracle upper bound of 73.16 and 63.61.

Why shift from task-driven to capability-driven self-improvement?

Task-specific optimization leads to Harness overfitting and poor generalization across heterogeneous tasks. By grouping tasks by shared requirements—such as perception or state tracking—the framework makes the underlying mechanisms the common learning objective, enabling generalization across different modalities and surface forms.

What is the role of the "Real2Sim2Real" process?

Creator abstracts common requirements from heterogeneous tasks into simplified, verifiable simulated worlds. This allows the agent to learn and refine specific capabilities in a controlled environment, while evaluation on the original, complex tasks measures the actual transfer of those capabilities.

Introduction and Motivation

The paper frames environment creation and agent scaling bottlenecks as motivation for a capability‑driven RSI framework.

Scaling multimodal intelligence hits two hard limits: building large, heterogeneous, verifiable environments is expensive, and the Harness that defines an agent’s capabilities must be manually updated for each new task. As tasks proliferate in modality and complexity, these bottlenecks prevent continuous improvement.

A recursive self‑improvement loop that jointly evolves multimodal agents and their worlds by organizing tasks into a capability tree, enabling agents to refine modular Harnesses.

Environment creation and agent scaling are the primary bottlenecks; addressing them via capability‑driven recursive self‑improvement is essential.

Multimodal Agentic Tasks

Defines task challenges and outlines the Harness’s six control‑flow protocols.

Long‑horizon multimodal agentic tasks combine perception, reasoning, and generation with memory, planning, and adaptation across observation–action sequences. Agents must retain evidence, revise hypotheses, and adjust plans as new observations arrive. Errors in early steps can cascade, making coordinated Harness organization essential.

A Harness configuration $\\phi$ coordinates a frozen model $f_{\\theta}$ through six control‑flow protocols: perception, memory, inference, planning, orchestration, and loop. The Base Harness $H_{0}$ serves as the initial configuration, and the equations formalize how each protocol processes observations, history, and resources. These roles are independent of task categories and the capability‑tree structure.

The Trinity Framework

How the Trinity builds evolving Harnesses via a Capability Tree and RSI.

Multimodal agents often hit a wall when faced with tasks that differ from their training distribution. Their internal modules are static, so they cannot incorporate new capabilities without costly retraining. The Trinity addresses this bottleneck by letting agents iteratively improve themselves while the environment co‑evolves.

A Harness is a self‑contained module that can be plugged into any agent to add a new capability, much like swapping in a new tool without redesigning the whole machine.

Start with $H_0parameters\theta_0$.

Collect a few trajectories where the agent bumps into obstacles.

Fine‑tune $\theta_0$ on these trajectories, producing $\theta_1$ for $H_1$.

Store $\theta_1$ as the child node; the parent parameters $\theta_0$ remain unchanged.

When a new task requires obstacle avoidance, the router selects $H_1$ and uses $\theta_1$ directly.

Only the parameters relevant to the new capability are updated, so earlier navigation behavior is preserved without extra training.

How does a Harness differ from simply fine‑tuning the whole model on a new task?

Fine‑tuning rewrites all parameters, risking loss of previously learned abilities. A Harness updates only a subtree, keeping the rest of the model exactly as before, which guarantees that earlier capabilities remain intact.

The Capability Tree is a trie that groups training tasks by their shared requirements, so common capabilities are represented once and inherited by all descendant tasks.

Sort each set: $p_{\tau_1}= [a,b]$, $p_{\tau_2}= [a,c]$, $p_{\tau_3}= [a,b,d]$.

Insert $p_{\tau_1}$: create root→a→b nodes.

Insert $p_{\tau_2}$: reuse root→a, then add new child c under a.

Insert $p_{\tau_3}$: reuse root→a→b, then add child d under b.

The resulting trie has a shared prefix a→b for tasks 1 and 3, and a separate branch a→c for task 2.

The trie automatically captures shared capabilities (node $a$) and isolates task‑specific ones (nodes $b$, c, d), enabling efficient reuse.

Why not build a flat list of requirements instead of a trie?

A flat list would duplicate shared requirements for every task, inflating the amount of learning and storage. The trie merges common prefixes, so the same capability is learned once and reused across many tasks.

RSI is the loop where the current Harnesses generate data, the Evolver extracts new requirements, and the Creator builds better worlds, feeding the improved Harnesses back into the loop.

How does RSI avoid the “runaway” problem where the agent keeps inventing new capabilities forever?

RSI stops when the Evolver cannot extract any new requirement from the latest trajectories. Because the Capability Tree is built from a fixed training set, the space of possible requirements is bounded, guaranteeing termination.

These three agents form a pipeline: Explorer gathers experience, Creator builds worlds that highlight missing skills, and Evolver turns that experience into new capability categories.

Normalize public multimodal benchmarks into a unified specification of goals, observations, actions, feedback, and budgets.

Group benchmarks into five requirement domains (rule discovery, constrained problem solving, perception & control, multi‑agent interaction, long‑horizon environments).

Fix the train/unseen split before any capability induction.

Explorer collects trajectories from training instances using the base Harness $H_0$ under frozen settings.

Store trajectories, hash‑chain them for integrity, and discard failed runs.

Evolver derives requirement sets $K_\tau$ from task specifications and trajectory checks.

Independent agents propose capability categories $K$; a judge consolidates them.

Sort each $K_\tau$ by the deterministic order $\prec$ to obtain $p_\tau$.

Insert each $p_\tau$ into a Trie, merging common prefixes to form the Capability Tree $T$.

Assign each training task $\tau \in B_{tr}$ to the leaf node representing its full requirement set.

Unseen tasks $\tau \in B_{un}$ are later evaluated against the fixed tree without modifying it.

The router reads a task’s public specification, walks the Capability Tree to find the deepest node whose requirements are satisfied, and selects the corresponding Harness branch.

**Figure 1 | Overview of The Trinity.** Creator, Explorer, and Evolver connect multimodal task-space expansion (left) with capability-driven Harness evolution (right) through verified feedback. The lower panels summarize tree construction, world exploration and creation, and capability-aware routing; the charts report mean and benchmark-level Train/Unseen performance across RSI iterations.

**Figure 2.** Detailed pipeline of The Trinity. Three stages build task, capability, and Harness trees. Worlds are verified and screened; Harnesses evolve through selective inheritance and local refinement. Routing selects one compatible Harness and stores verified feedback for later tasks.

Performance and Evolution Results

Iterative Harness upgrades boost performance across RSI iterations.

The Trinity framework links each node of the capability tree to concrete training benchmarks, then lets Explorer and Creator iteratively refine Harnesses along that structure.

The Base Harness is the initial collection of modules that the system starts with before any recursive self‑improvement iteration.

How does the Base Harness differ from the later Harnesses produced by RSI?

The Base Harness contains only the original modules; each subsequent Harness (e.g., H⁽¹⁾, H⁽²⁾) incorporates new or refined modules that were validated by the host‑executed verifier, so performance differences directly reflect the added capabilities.

**Table 8.** Full benchmark results. Scores on a 0–100 scale for the Base Harness $H_0$, the Harnesses H^(1)–H^(4) after each RSI iteration, and the Codex and DeepSeek Harnesses on the unseen benchmarks; – marks runs that were not performed.

Across RSI iterations the Harnesses achieve sizable performance gains, validating the recursive self‑improvement loop.

Related Work

We position our approach among prior self‑improving harness and capability‑driven methods.

Prior work on self‑improving agents spans Gödel machines, LLMs that rewrite their own code, automated agent design, prompt optimization, and learning from self‑feedback. Recent systems synthesize harnesses from environment feedback, repair them after failed trajectories, and treat harnesses as optimizable code, often evaluated against benchmarks such as Codex and DeepSeek. Our approach differs by inducing a capability tree from training trajectories, sharing prefixes across heterogeneous benchmarks, and evolving harnesses in creator‑generated worlds that exclude target‑task layouts.

Codex and DeepSeek are large language model baselines whose performance on unseen benchmarks serves as a reference point for our evaluations.

Capability Tree Details

Details the capability taxonomy and defect classification used for ablation analysis.

The appendix expands on the Capability Tree introduced earlier, showing how training trajectories are mapped to a structured taxonomy of capabilities and associated defect types.

Defect types are concrete failure patterns that expose gaps in a given capability, providing a fine‑grained signal for ablation.

How does this defect taxonomy differ from a generic error log?

An error log merely records failures; the defect taxonomy classifies each failure by the specific capability it reveals, turning raw errors into structured signals that guide targeted ablations.

**Figure 11 | Capability tree.** Each node is one combination of capabilities, read along the branch from the root, and each benchmark sits at the node of its combination. Bricks keep the colours of Figure 9; unseen benchmarks are in italics; * marks rebuilt environments.

Table 7 enumerates the defect taxonomy: columns “Training” and “Unseen” count how many training and unseen benchmarks exhibit each defect. For example, defect F18 (non‑convergence detection) appears nine times in both columns, while several defects (e.g., F04, F06) are absent from unseen benchmarks, indicating they were fully resolved during training.

Harness Tree Evolution

Ablation of per-node versus shared Harnesses reveals the impact of modular retention on performance.

Explorer and Creator start from $H_0$ but differ in how they retain learned modules across the capability tree.

Initialize each node's Harness from $H_0$ with no learned modules.

Inherit selected parent modules, preserving their versioned functions.

Add or revise local modules based on Evolver's analysis of successful and failed trajectories.

Validate every update via the host’s evidence, interface, and version‑lineage checks.

Retain an updated parent module only if it yields strictly more verified task completions than the previous version.

Use the retained Harness at the node for routing during benchmark execution.

**Figure 12.** Explorer Harness tree. Every node Harness starts from $H_0$ and combines the modules it selectively inherits with its own local modules. Brick colours mark capability families, and labels give module IDs.

Begin each iteration with a Base Harness $H_0$ extended by pre‑RSI modules.

Traverse nodes top‑down; each node inherits the parent’s updated modules for the current iteration.

Within an iteration, verifier‑detected failures trigger local module evolution at that node.

At iteration end, consolidate all validated module proposals into a single shared library.

Replace previous versions with validated revisions; merge overlapping capabilities and discard near‑duplicates.

The resulting shared library becomes the Base Harness for the next iteration, exposing a single module per call.

Ablating the per‑node Harnesses in favor of a shared library reduces the system’s ability to specialize modules, leading to lower task success rates.

Visual Evidence Record

Records how H0 and H(4) differ across benchmarks, nodes, and embodied tasks.

The appendix evaluates 44 benchmarks comparing the original Harness H0 and the refined Harness H(4).

Appendix E.1 pairs H0 and H(4) on the same configuration of each of the 44 benchmarks, revealing differing reasoning and outcomes.

**Figure.** A comparison of two planning trajectories for a household query. The top row shows a failed trajectory ($H_0$) where repeated use of banned items leads to failure. The bottom row shows a successful trajectory ($H_4$) where the agent iteratively bans flagged items and adjusts its plan, ultimately achieving success.

**Figure 10 | Flat capability tree.** Bricks keep the colours of Figure 9; unseen benchmarks are in italics; * marks rebuilt environments.

Conclusion

Conclusion, discussion of limits and future work, and author information for the Trinity framework.

We propose Capability‑Driven Recursive Self‑Improvement via the Trinity framework, which couples world generation with Harness evolution through Creator, Explorer, and Evolver. Organizing heterogeneous tasks around shared capability requirements yields simplified, verifiable worlds and guides the evolution of reusable modules. These modules, linked to capability–module correlations and controlled interventions, provide empirical support for using capability priors in module design.

Limitations: the framework relies on web search for benchmark discovery, which can suffer from variable coverage and result quality; current experiments focus on self‑improvement at the Harness level. Outlook: capability‑driven learning offers a paradigm where shared capability requirements guide optimization, and we plan to expand the capability tree dynamically and extend the approach to model‑parameter training, closing the loop with capability‑aware routing and RSI.

Core contributors include Weidong Tang, Ruiheng Wang, Haoyuan Chen, Zhengfeiyang Zhang, Yixing Ma, Zichen Jiang, Ziyu Guo, Haodong Li, Zhengbo Jiao, Zhiqing Cui, Xinxiang Yin, Shuo Liang, Zhenglin Wan, Hongzhan Lin, Wangbo Zhao, Kaipeng Zhang, and others across Inf Rec, Xidian University, Tongji University, NUS, UC Berkeley, CUHK, and additional institutions; corresponding authors are Pengfei Zhou, Odin Zhang, and Yang You.

Task Tree Inventory

A detailed overview of the benchmark task tree and its composition.

This appendix enumerates the full benchmark task tree that underpins the evaluation suite.

A hierarchical organization of benchmarks that groups them by domain and subclass, forming the capability structure used to guide agent development.

**Figure 9.** Task tree. The 44 benchmarks are grouped into five domains and 15 subclasses. Training benchmarks take the colour of their domain and unseen benchmarks a lighter shade of it; SOON and CaP-X show their simulation and real-robot versions; * marks environments rebuilt from published rules.

Evaluation Methodology

Evaluation setup and benchmark results for all Harness variants.

The evaluation uses a frozen Claude Opus 5 backbone with adaptive thinking at low effort, and each benchmark’s native driver validates actions, returns observations, and assigns outcomes.

Base Harness $H_0$ runs with no learned module and no skills; for $H(i)$ each instance selects the best‑scoring compatible node Harness under the Oracle setting and augments it with evaluation‑time skills summarized from five preparation trials.

Levels, action budgets, and scoring are fixed before the first run and shared by every Harness. Benchmarks report results on a $0$–$100$ scale (task‑completion, win, or solve rates), any published bounded score, and for ALE‑Bench the rank percentile among human entrants.

The Codex and DeepSeek Harnesses were evaluated on the $16$ unseen benchmarks (excluding real‑robot versions) using the same backbone and settings via their native tool interfaces. Table 8 lists every score together with each benchmark’s measure, number of distinct instances, and number of episodes; “*” marks environments rebuilt from source rules through agent web search.

Training benchmarks use fixed instance sets (Section 4.1), and ARC‑AGI‑3 covers the $25$ public games. Six unseen benchmarks run on fixed subsets: Pencil Puzzle Bench (Golden $30$ set), PHYRE ($50$ test tasks of fold 0), Video‑MME‑v2 (stratified $400$ of $800$ videos, $1{,}600$ questions), DiG‑bench ($13$ of $21$ public games, tiers 1–3 plus first games of tiers 4–7), ALE‑Bench ($26$ of $40$ problems) and SciGym ($126$ of $137$ small systems that passed the safety filter). NetHack is scored after a budget of $500$ steps.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers