Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design

Qing Zong, Jiayu Liu, Junhao Shen, Zecong Tang, Linsi Wu, Yuxuan Liu, Rui Wang, Zhaowei Wang, Weiqi Wang, Cheng Qian, Xiusi Chen, Yangqiu Song

A survey of co-evolutionary agentic systems, where agents and environments adaptively reshape one another.

How can agentic systems evolve beyond static human-designed tasks by using co-evolutionary pressure between agents and their environments?

Agentic systems often plateau because they learn against static tasks or fixed feedback, limiting their ability to improve after deployment. This survey proposes a three-stage taxonomy for co-evolution, where agents, environments, and even the evolution mechanisms themselves adapt in a coupled loop to remove human-engineered constraints. The authors argue that this shift from isolated self-improvement to mutual adaptation is the primary pathway toward open-ended systems that can generate continuous novelty.

Paper Primer

The paper organizes the literature into a progressive taxonomy of evolutionary freedom. It moves from Stage 1 (Agent–Agent), where peers adapt to each other; to Stage 2 (Agent–Environment), where agents reshape the tasks, feedback, and worlds they inhabit; to Stage 3 (Meta Co-Evolution), where the system revises its own evolution mechanism.

Co-evolutionary systems enable open-ended improvement by escaping the "Red Queen" effect of static learning.

The taxonomy maps how systems transition from fixed-environment learning to self-directed evolution, where the evolution mechanism itself becomes an adaptive component. The framework provides a unified foundation for scaling agentic systems beyond human-designed paths, targeting unbounded adaptive capacity.

Why is co-evolution necessary if an agent can already "self-evolve" by updating its own weights or memory?

Single-entity self-evolution remains bounded by static external conditions. Co-evolution forces mutual adaptation, where the counterpart (peer or environment) also changes, preventing the system from converging to a fixed endpoint.

What is the "Meta Co-Evolution" stage, and how does it differ from standard meta-learning?

Meta co-evolution occurs when the evolution mechanism itself (the rules for what, when, and how to evolve) is revised by the system. Unlike standard meta-learning, this revision is induced by the joint trajectory of a lower-level co-evolving ecosystem and subsequently alters that ecosystem's future conditions.

The Case for Co-Evolution

Co‑evolution transforms static learning loops into dynamic, self‑directed improvement cycles.

Agentic systems quickly hit performance plateaus when they evolve within fixed tasks and static feedback, because only the agent changes while the surrounding context stays unchanged. Co‑evolution breaks this limitation by letting agents and their environments adapt together, forming a dynamic, self‑directed evolutionary loop.

**Figure 1.** Comparison between single-entity self-evolution and multi-component co-evolution.

Co‑Evolutionary Pressure is the mutual adaptive force that arises when multiple agents and their environment continuously influence each other's changes, preventing stagnation.

We organize the literature with a three‑stage taxonomy that progressively expands the scope of what may evolve: first agents alone, then agents plus their environments, and finally the evolution mechanism itself. Each stage sheds more human‑engineered constraints, moving toward open‑ended improvement.

Shifting from static to dynamic evolutionary loops unlocks continual improvement beyond fixed human‑designed paths.

Defining the Agentic Landscape

We map co‑evolutionary research into three expanding stages and expose the missing meta‑level mechanisms.

Across the surveyed literature, researchers agree that allowing agents to evolve together yields richer dynamics, yet most work stops at agent‑environment interaction, leaving meta‑level evolution largely unexplored.

An Agentic System $S$ is a collective of interacting agents $A$ operating inside an environment $E$, where each agent bundles a model backbone $m_i$ and an optional harness $h_i$ (memory, tools, prompts, etc.).

Co‑evolution refers to a coupled self‑evolution where at least two units within a system simultaneously adapt and continuously reshape each other’s evolutionary pressures.

**Figure 2.** The progressive taxonomy of co-evolution, which reflects an expanding boundary of evolutionary freedom.

Agents evolve together while the environment stays fixed. Each agent’s progress alters the challenges faced by its peers, and the collective structure $\Pi$ may also reorganize, creating a tightly coupled evolutionary loop within the agent collective.

Both agents and the environment evolve jointly. The environment is no longer a fixed backdrop; agents reshape it, and the altered environment feeds back new pressures onto the agents, forming a bidirectional adaptive loop.

Beyond evolving agents and environments, the system also revises its own evolution mechanism $\Omega$ via a higher‑level process $\Gamma$. This meta‑level adaptation allows the rules of change themselves to become subject to evolution, opening a path toward open‑ended novelty.

**Table 1.** Comparison with surveys in adjacent areas. We retain each survey's original taxonomy and report where co-evolution appears in its organization.

Adversarial Agent Co-Evolution

Adversarial co‑evolution drives agents to out‑perform each other, yielding robustness and safety gains across vision, control, and language.

Across adversarial studies, the dominant pattern is an arms‑race dynamic that repeatedly raises task difficulty, while the conspicuous gap is limited exploration of multi‑agent diversity beyond pairwise setups.

Agents improve by constantly confronting opponents whose success makes the next challenge harder, creating a self‑reinforcing escalation loop.

How does adversarial co‑evolution differ from conventional adversarial training?

Conventional adversarial training uses a fixed perturbation model (e.g., FGSM) that does not adapt, whereas adversarial co‑evolution treats the adversary as a learning agent whose policy evolves alongside the defender, continuously reshaping the difficulty landscape.

This camp treats a generator and a discriminator as opposing agents. The generator learns to synthesize realistic images, while the discriminator learns to distinguish real from fake, each improving through the other's failures.

Here a control policy (the protagonist) learns against an adversary that injects destabilizing forces into the environment. The adversary’s goal is to maximize the protagonist’s cost, forcing the controller to develop resilience to unseen perturbations.

Agents engage in zero‑sum games (e.g., fighting, hide‑and‑seek) where each episode pits two learning agents against each other. Successively stronger strategies emerge as each side adapts to the opponent’s latest tactics.

Two language agents play a word‑guessing game: one tries to force the other to utter a secret word, while the counterpart attempts to infer it. Their opposing goals generate a linguistic arms race that improves both prompting and comprehension abilities.

In this camp, an attacker LLM generates jailbreak prompts while a defender LLM learns to refuse them. The loop iterates: successful attacks are added to the training set, prompting the defender to adapt, which in turn forces the attacker to discover new jailbreak styles.

**Figure 3.** Paper landscape of co-evolution in agentic systems.

Collaborative and Organizational Co-Evolution

We map collaborative co‑evolution approaches, highlighting shared patterns and key gaps.

Multiple agents improve together under shared goals, so each agent’s progress forces its partners to adapt.

How is collaborative co‑evolution different from adversarial co‑evolution?

Collaborative co‑evolution aligns agents around a shared objective and reinforces each other’s successes, whereas adversarial co‑evolution pits agents against each other, driving improvement through competition.

Parallel and role‑differentiated collaboration both rely on shared rewards, but only the latter tackles credit‑assignment for distinct roles; the field still lacks systematic methods for scaling complex role hierarchies.

This camp treats agents as peers that share a single task reward and co‑adapt as each partner changes. Early MARL work (Lowe 2017; Foerster 2018; Yang 2018) established the paradigm, and recent LLM‑agent studies extend it with group‑relative advantages and shared search trees. Methods such as MARS2 spread test rewards across a shared tree, while CORAL diffuses learned skills through a communal memory.

This camp assigns fixed, distinct roles to agents (e.g., producer‑reviser, verifier‑generator). Improvements in one role directly shape the learning signal for the other, often via a produce‑and‑revise loop. Systems such as CORY swap roles periodically, while RL Tango trains a verifier to give process feedback. Credit‑assignment mechanisms are explicitly modeled to attribute shared rewards to each role.

This camp goes beyond fixed roles, allowing the organization itself—role assignments, interaction topology, and workflow—to co‑evolve with agent policies. R3DM discovers emergent roles from behavior, SkillMAS restructures team composition when execution traces reveal mismatches, and MetaAgent‑X jointly trains workflow designers and executors. The focus is on dynamic restructuring of the team as a first‑class evolutionary variable.

Task-Space Co-Evolution

Task-space co-evolution reshapes the problems agents face, driving continual skill growth.

Across task‑space studies, the field agrees that fixing a static task pool stalls agents, while dynamically adapting tasks—either by re‑sampling existing problems or by inventing new ones—consistently yields higher competence and better generalization.

Think of the environment as a teacher who continuously hands each learner a problem that sits just beyond what they can already solve, forcing the learner to stretch its capabilities step by step.

How does task‑space co‑evolution differ from classic curriculum learning?

Curriculum learning re‑orders a *fixed* set of tasks based on difficulty, but it never creates new problems. Task‑space co‑evolution either resamples from a static pool *or* synthesizes novel tasks, so the task distribution itself changes as the agent improves.

This camp keeps the task pool static and focuses on *which* tasks the agent sees next. Early work used learning‑progress signals to bias sampling (curriculum learning). Later methods steer sampling toward the agent’s current competence (continuous‑parameter curricula) or replay past levels that still offer learning potential. Regret‑based selectors prioritize configurations where the agent lags behind a stronger reference, and multi‑agent variants adjust exposure to teammates of varying ability. Recent LLM‑agent systems resample tasks that match the model’s own failure modes, while SEAD samples simulated user profiles that become harder as the service agent improves.

This camp synthesizes new tasks rather than reshuffling an existing pool, placing them near the agent’s competence frontier. Goal‑generation methods let embodied agents discover reachable yet challenging objectives. In LLM‑agent settings, generators calibrate difficulty to the solver’s success rate, convert failed rollouts into fresh tasks, or pair each task with a verification function. Search‑oriented approaches generate multi‑hop questions from retrieval behavior. GUI‑and‑software agents explore unfamiliar interfaces to discover operations and turn them into tasks, sometimes perturbing parameters like time or quantity. Some systems also auto‑generate validators and setup scripts, enabling end‑to‑end task execution. RLAnything jointly rewrites tasks, policies, and rewards across diverse domains.

Feedback-Space Co-Evolution

We map feedback‑space co‑evolution approaches and expose their shared strengths and gaps.

Feedback‑space co‑evolution unifies reward‑learning, outcome‑driven, and consistency‑augmented loops, yet most works still rely on static evaluators or limited contrast.

Agents and their evaluators evolve together: the reward model, test suite, or diagnostic critic is updated as the policy changes, creating a feedback loop that reshapes both behavior and its measurement.

This camp treats the evaluator as a reward model that learns from pairwise preference comparisons. Early work (Christiano et al.) trains a neural reward from human‑generated trajectory pairs; later systems replace humans with scripted teachers and reuse stored rollouts. Methods differ in how they select informative pairs: PEBBLE relabels replay buffers when the model changes, Liang et al. and Zhu et al. query the model where its confidence is lowest, DAPPER maximizes behavioral contrast by pairing trajectories from distinct policies, and DUO focuses on high‑disagreement, diverse segments.

This camp updates the evaluator based on observed task outcomes—returns, failures, or execution traces. Policies are trained under a candidate reward, then the reward is revised using rollout returns or failure signals. Some systems co‑search reward candidates and policies (ROSKA, LaRes), while others refine decomposed rewards from visual or scalar feedback (REGOT). Additional works adjust reward parameters from policy regret (Lu et al., 2025c) or evolve unit tests and verifiers from execution failures (CURE, CoEvoSkills). Hint‑based curricula (Lu et al., 2026b) and dynamic re‑weighting of fixed terms (AHRS) further shape the feedback signal.

This camp strengthens evaluators by enforcing internal consistency across multiple signals. SURF reuses confidently ranked segment pairs to augment sparse labels. R* updates reward parameters only when several critics agree on segment rankings. ARCO co‑trains a step‑scoring rubric with the policy, requiring step scores to sum to the final outcome. NLAC trains a language critic whose next‑step predictions must stay consistent with later trajectory steps. ECHO updates a diagnostic critic only when its advice yields measurable policy improvement.

Interaction-Space Co-Evolution

Survey of approaches that let environments co‑evolve with agents.

Agents and the spaces they act in evolve together: the environment changes in response to agent behavior while agents adapt to the newly shaped environment.

Across the surveyed literature, three dominant camps emerge: evolutionary level generators, simulation‑randomization/LLM‑driven generators, and world‑model co‑training, each trading offline design effort for online adaptability.

This camp treats the environment as a genotype that evolves alongside agent policies. Evolutionary algorithms (e.g., POET) maintain a population of environment‑agent pairs, selecting and mutating environments that challenge current agents. Regret‑based methods explicitly construct levels that expose the agent’s current weaknesses. The approach is applied to gridworld navigation, continuous control, and multi‑agent games, reporting up to 30 % higher success rates on curricula compared with static baselines.

This camp augments high‑fidelity simulators or embodied environments. Domain randomization (ADR) widens physical parameter ranges as policies improve, while OMNI‑EPIC filters generated worlds by an “interestingness” metric. LLM‑POET adapts the classic POET loop to embodied simulation using an LLM to propose novel scene configurations. The methods are evaluated on robotics and embodied‑AI tasks, reporting up to 40 % reduction in sim‑to‑real gaps.

This camp replaces the real environment with a learned world model that co‑trains with the agent. The model predicts observations, rewards, or textual feedback, enabling virtual rollouts that supplement or replace costly real interactions. Reported gains include up to 2× faster convergence on text‑based games and significant sample‑efficiency improvements in visual domains such as Minecraft driving.

Meta Co-Evolution

Synthesizes the field’s consensus on meta‑co‑evolution and highlights remaining disagreements.

Meta‑co‑evolution shifts the field from static evolutionary loops to mechanisms that themselves evolve, opening a path toward open‑ended agentic systems.

**Figure 4.** Cross-paper evidence for the effect, consistency, and convergence of co-evolution. See Appendix C for data selection and aggregation details.

Methods that make the evolution mechanism itself evolvable but keep the rest of the system static. They typically evolve prompts, self‑modification code, or memory architectures while the surrounding task agents and evaluators remain unchanged.

Approaches that let the evolution mechanism itself be subject to evolution, thereby altering the co‑evolving ecosystem of agents, tasks, and environments. This creates a dynamic loop where each evolutionary step reshapes the conditions for the next.

Challenges and Future Directions

Open challenges and research directions for evaluating, scaling, and governing co‑evolutionary agentic systems.

Current benchmarks report only final agent capabilities—tool use, web browsing, software engineering, computer use, and multi‑agent interaction—leaving the dynamics of co‑evolution invisible.

PostTrain‑Bench (Rank et al., 2026) is a first step toward evolution‑aware evaluation by testing whether an agent can autonomously post‑train a base LLM.

True co‑evolutionary evaluation must answer three questions: do all evolving components improve, do gains transfer to unseen partners and environments, and how does each component contribute to joint progress.

Higher task success can mask exploitative behavior, leading to evaluator exploitation, partner overfitting, or diversity collapse.

Therefore evaluation should pair static benchmarks with process‑level tests such as historical cross‑play, component ablations, and held‑out evaluators.

Most existing work studies local loops—attacker–defender training, policy–reward adaptation, or agent–task generation—rather than full system‑wide co‑evolution.

Future research should scale to scenarios where agents, their harness, and the environment evolve together, requiring careful choices about which components adapt and how their updates interact.

Meta co‑evolution tackles this by allowing the system itself to decide evolutionary pressures, moving beyond human‑defined improvement pathways.

As autonomy grows, evolving agents may develop novel attack strategies, tool‑use patterns, communication protocols, or organizational behaviors that exceed human comprehension.

Ensuring safety therefore demands auditable, interruptible governance: sandboxed deployment, continuous monitoring, rollback to verified states, and explicit human intervention points.

Conclusion and Outlook

We conclude, discuss limitations, and provide an ethics statement for the survey.

We organize the field through a progressive taxonomy that expands the adaptive boundary from agents alone, to agents plus environments, and finally to the joint evolution mechanism, highlighting that future gains come from agents that continuously co‑evolve rather than from stronger static agents.

Meta co‑evolution remains nascent; only a few works satisfy our Stage 3 definition, so our discussion largely relies on single‑entity meta‑evolution, which shows that evolution mechanisms can evolve but are not yet coupled to lower‑level co‑evolving systems.

We flag safety and governance as unmet operational needs, noting first‑order concerns such as evaluator exploitation, partner overfitting, and diversity collapse, while emphasizing that concrete safeguards remain future work.

Our ethics statement confirms that all surveyed works are properly cited, publicly accessible, and that we performed no new data curation or human annotation, leading us to conclude the paper raises no ethical concerns.

Appendix: Distinguishing Concepts

Distinguishes co‑evolution from related ideas and surveys its coverage.

Co‑evolution is only present when multiple agents or components persistently update each other, shaping one another over successive interactions.

Multi‑agent interaction alone—communication, collaboration, or competition—does not constitute co‑evolution unless those interactions lead to lasting updates in the agents.

Agent loops and loop engineering describe how a single agent repeats a task, designing goals, feedback, verification, and stopping rules; they do not address whether the agent itself changes across tasks.

Harness engineering builds the runtime scaffolding (prompts, memory, tools, workflows) for an agent. It becomes co‑evolutionary only if the harness or the model is updated and those updates are retained across runs.

Evolutionary optimization searches over candidate prompts or designs and selects the best; co‑evolution requires that the selected units continue to adapt to each other, not just be evaluated once.

Continual learning repeatedly updates a single learner while preserving its parameters; it lacks the mutual adaptation of separate evolving units that defines co‑evolution.

Self‑play pits a learner against copies of itself. It can drive self‑evolution, but co‑evolution demands at least two distinct adaptive units that retain their changes and influence each other.

The surveyed literature (Meng et al., Guo et al., Huang et al., etc.) largely organizes around harnesses, multi‑agent systems, or self‑evolution, with only scattered mentions of co‑evolution and no dedicated taxonomy.

Our taxonomy places co‑evolution at the core, distinguishing Agent–Agent, Agent–Environment, and Meta co‑evolution as separate branches.

Self‑evolution lets an agent improve while its task, feedback, or counterpart stays fixed; co‑evolution adds a second adaptive unit whose changes reshape the first and vice‑versa.

Meta‑evolution modifies the very mechanism that governs future evolution (what, when, how, where to evolve) and is driven by a lower‑level co‑evolving system.

Iterative self‑improvement repeats a fixed improvement procedure; recursive self‑improvement also upgrades that procedure itself, enabling ever‑more effective upgrades.

Normalized trajectories map raw performance curves onto a [0, 1] progress grid, aligning lower‑is‑better metrics before scaling.

Open‑endedness describes a system that keeps generating novel, learnable outcomes indefinitely; it is a possible long‑term behavior, not a guarantee of co‑evolution.

Panels A & B aggregate matched static vs. evolving conditions across papers, compute within‑paper mean improvements, and plot the average paired gain against the number of matched settings.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers