Raven: The Harness of Harnesses for Composable Agentic Intelligence
EverMind AI
Raven is a multi-agent ecosystem that autonomously composes and evolves specialized harnesses to solve long-horizon, cross-domain tasks.
How can we compose multiple specialized AI agents into reliable, long-horizon workflows without the manual overhead of designing custom harnesses for every task?
Manual harness design for AI agents fails to scale as tasks grow more complex, forcing developers to choose between rigid, domain-specific tools and brittle, general-purpose systems. Raven treats each model-harness pair as a composable unit, using a Host Agent to decompose goals into directed acyclic graphs (DAGs) of specialized agents that exchange artifacts through typed execution contracts. On the Multi-Agent Orchestration Benchmark, Raven outperforms state-of-the-art systems, achieving Exact Match gains of over 10 percentage points in specialist selection and dependency prediction.
Paper Primer
Raven functions as a "Harness of Harnesses," providing a shared orchestration interface that connects native specialists (Research, Code, Design, Oncall) with third-party agents like Claude Code or Codex. The core mechanism is a typed execution protocol: the Host Agent maps subtasks to agents, defines their dependencies as a DAG, and enforces semantic contracts that ensure outputs from one agent are compatible with the inputs of the next.
Raven significantly improves orchestration accuracy on complex, long-horizon tasks.
Performance on the Multi-Agent Orchestration Benchmark (MAOB) across two model backbones. 10.4 and 10.5 percentage point gains in Exact Match over the strongest baseline.
Why is a "harness" necessary, and why does Raven treat it as a composable unit?
A harness—comprising tool interfaces, context management, and execution policies—determines how a model applies its capabilities. By treating these as composable units, Raven allows the system to mix and match specialized execution environments rather than forcing a single, monolithic agent to handle every stage of a complex workflow.
How does Raven handle the coordination overhead that typically plagues multi-agent systems?
Raven makes collaboration explicit through a typed dependency graph and an append-only ledger for artifacts. This structure prevents the context-loss common in free-form message passing and allows the runtime to validate dependencies and resource feasibility before any agent begins execution.
Introduction
We frame the scaling and generality challenges of harness design and introduce Raven’s automated solution.
As large language models mature, agents move from isolated, domain‑specific tasks toward long‑horizon, cross‑domain workflows. This shift exposes two intertwined problems: (1) harnesses grow more complex, making manual design hard to scale, and (2) tighter coupling to particular domains curtails the generality of any single harness.
Raven’s Host automatically builds and runs executable model–harness pairs, handling task assignment, data exchange, and execution so agents can refine their policies and reuse procedural knowledge.
How does the Raven Host differ from a conventional task scheduler?
A conventional scheduler merely queues jobs, assuming each job is self‑contained. The Raven Host, by contrast, constructs a DAG of interdependent subtasks, actively matches each subtask to a specialist with the appropriate toolset, and manages data handoffs and error recovery, turning a simple queue into a coordinated, knowledge‑rich workflow.
**Figure 1.** Raven performance overview. Selected results for multi-agent orchestration, research, coding, design and on-call tasks, compared with the strongest reported alternative in each setting.
The shift from isolated tasks to long‑horizon workflows requires automated harness design.
System Architecture and Composition
Formalizing how a Harness turns a model into a composable execution policy.
The core challenge is to guarantee reliable task coverage when many agents are stitched together under a shared budget. Without a principled composition analysis, a host could over‑allocate resources or silently drop tasks, breaking the system’s guarantees.
A Harness wraps a static model with its tool interfaces, context, memory mechanisms, and control rules, turning the model into an executable policy that can be composed with other agents.
How does a Harness differ from simply wrapping a model in a function?
Wrapping a model in a plain function yields a deterministic mapping from inputs to outputs and cannot express uncertainty, tool calls, or stateful memory. A Harness explicitly defines a stochastic policy over admissible actions, incorporates tool interfaces, and respects a typed observation history, which are all required for the composition soundness proofs.
Step 1: The Harness reads the last label (e.g., $0$) from $H$.
Step 2: It queries the tool “flip‑coin”, receiving $c \in \{0,1\}$ with equal probability.
Step 3: It forms the action distribution $\Delta(U)$ that assigns $0.5$ to “output $c$” and $0.5$ to “output $1-c$”.
Step 4: Sampling from this distribution yields the final output (e.g., $1$).
The Harness injects controlled randomness via the tool, so the same observation history can lead to different outcomes—exactly the stochastic behavior required for the later reliability analysis.
Define a plan $\pi = (G, \sigma, \Phi, \Gamma, \lambda, b)$ where $G=(V,E)$ is a DAG of worker calls.
Assign each node $v\in V$ a Harnessed agent $a_i$ via $\sigma(v)$, and attach typed contracts $\Gamma$ to every edge.
Compute a schedule $\lambda$ that orders all nodes and edges respecting data dependencies.
Check resource feasibility: $b_H + \sum_{v\in V} b_v + \sum_{e\in E} b_e \le B$.
Execute operations in order $\lambda_1,\dots,\lambda_{K_{\text{op}}}$, charging the declared cost $b_{\lambda_j}$ and updating the append‑only ledger $\ell$.
At each step verify the entry predicate $A_j$ and transition predicate $D_j$; successful transitions preserve the invariant $I_j$.
After the final node, retrieve the record $\text{rec}_\pi(\ell, \Sigma)$ and deliver it as the overall output $Y_{S_H}$.
**Figure 3.** Harness composition. (a) A model and its harness form a callable execution unit. (b) A host assigns and coordinates these units. Solid arrows carry artifacts and dashed arrows denote control. The research–code–design chain is an illustrative plan with explicit final synthesis by a specialist. All operations share one budget. Model labels need not identify different foundation models.
Multi-Agent Collaboration
How Raven structures heterogeneous agents through explicit dependency graphs.
Existing multi‑agent frameworks such as AutoGen and MetaGPT rely on a shared message interface and a simple turn‑taking loop, which leaves the system builder to configure role prompts and maintain the orchestration manually.
Raven, by contrast, builds heterogeneous harnesses: a roster that mixes built‑in specialists with third‑party agents (e.g., Claude Code, Codex) connected through execution adapters, exposing differing tools, contexts, and control policies.
The collaboration proceeds in five stages: (1) the Host loads orchestration guidance on demand and composes a typed dependency graph; (2) the runtime admits the graph after structural, capability, and environment checks; (3) the executor dispatches nodes as dependencies settle, judges each result, and returns exceptions or clarification requests; (4) a run report returns file locations and terminal outputs; (5) later graphs or single delegations reuse completed nodes by reference.
Running example: a user asks for a survey, a reproduction, and a web page. Two research nodes produce the survey and benchmark, a coding node consumes both, and a design node consumes the coding output, yielding the plan $\{ \text{valg}, \text{vbench} \} arrow \text{vcode} arrow \text{vdesign}$.
Raven treats a set of agents as a directed acyclic graph of tasks, where each node declares its required inputs and the host guarantees that those inputs are produced before the node runs.
Task decomposition starts with a Heterogeneous Agent Registry that binds a name to one of four backends and records three capability tags (stateful, file‑capable, progress‑reporting) used for planning.
Staged Orchestration Guidance supplies the host with a concise schema (placeholder syntax, concurrency limits, wiring rules) only when a request requires multiple agents; the full guide is pinned in session context for subsequent graph construction.
Graph construction works backward from the desired deliverable: the host creates nodes, assigns identifiers, and adds dependencies whenever a node’s input depends on another node’s output, producing a DAG $G_{rt} = (V_{rt}, E_{rt})$.
Each node $n_v$ stores $(\text{id}_v, \sigma(v), \text{Tpl}_v, \text{Dep}(v), \text{Bind}_v, \text{inst}_v, \text{Cfg}_v)$; admission validation checks format, graph structure, capability compatibility, status, and environment before any execution begins.
Once admitted, the Host mediates execution: nodes transition from pending to running once all predecessors are completed, respect a shared concurrency semaphore, and report outcomes to a completion judge that labels them “accomplished” or “not accomplished”.
Exceptions generate a structured report (node, agent, missing condition, dependent list) that the Host can repair, abandon, or trigger a replanning step; clarification requests are first answered by the Host using stored artifacts before involving the user.
This table outlines the access permissions for different actors regarding host libraries ($L_{host}$), user libraries ($L_{user}$), and agent libraries.
Raven organizes agents via executable dependency graphs rather than simple conversation loops.
Harness Self-Evolution
Harness Self-Evolution refines a frozen model’s policy via modular interfaces and a gene‑bank driven search.
Frozen language models cannot change their internal weights, so improving task performance requires altering the surrounding execution policy. The core pain is that a static model offers no room for incremental fixes when a task fails or a new tool becomes available. Harness Self‑Evolution solves this by evolving a lightweight wrapper that can be edited, screened, and archived without touching the frozen core.
Think of the frozen model as a black‑box engine; Harness Self‑Evolution builds a plug‑in harness that decides which tools to call, what context to keep, and how to act, then iteratively improves that plug‑in by diagnosing failures and recombining past solutions.
Apply the patch: $\text{Mem}_h$ now truncates the context to the most recent 4 tokens, leaving other components unchanged.
Run the diagnostic on a training task set $D_{\text{tr}}$ (3 attempts per task) and record a utility $\text{Util}(h')=0.62$ versus the parent utility $0.55$.
Compute the paired gain: $\bar{\gamma}=0.07$, sample variance $\sigma^2_{\gamma}=0.001$, yielding $t_{\text{pair}}=4.4$ which exceeds the cutoff $1.96$.
Screening accepts the candidate, stores it in the HarnessBank cell (Memory, “context‑overflow” failure), and marks it as a survivor for the next round.
The example shows that a tiny context‑window edit can produce a statistically significant utility gain, and the gene‑bank preserves that improvement for future recombination.
How does Harness Self‑Evolution differ from fine‑tuning the entire frozen model?
Fine‑tuning rewrites the model’s internal weights, requiring costly gradient updates and risking catastrophic forgetting. Harness Self‑Evolution leaves the model untouched and instead evolves a lightweight wrapper that can be edited, screened, and archived with orders‑of‑magnitude less compute.
The HarnessBank is a semantic archive that keeps one evaluated harness per (edit‑category, failure) cell, enabling reuse of past solutions across evolution rounds.
Algorithm 3: Candidate Screening
Algorithm 4: Semantic Gene‑Bank Search
**Figure 8.** Harness self-evolution. (a) Four strategy interfaces expose decisions around a frozen model. The shared execution shell handles dispatch and accounting. (b) The best training-evaluated harness supplies failure traces for diagnosis and candidate generation. The bank stores one complete harness per edit-category/pathology cell. P/K/R/C denote prompt, knowledge, runtime, and configuration edits. The first $N_{keep}$ candidates passing validity, activation, and paired-gain screening receive full-training evaluation before cell competition, where $N_{keep}$ is the per-round survivor cap. The final training-selected harness is frozen before held-out testing.
EverOS Memory System
EverOS Memory structures, consolidates, and retrieves past interactions for persistent agent and user knowledge.
Agent interactions generate a wealth of context that outlives a single turn. EverOS Memory captures, consolidates, and makes this information reusable across sessions.
EverOS Memory treats past dialogues as structured records that can be formed, grouped, and queried on demand.
Segment seg₁ is assigned a MemCell Cell₁ = (episode = “user asked X, agent performed search”, facts = {“search used”, “X queried”}, foresight = ∅, meta = timestamps + owner = user).
Embedding z₁ = Enc(episode) ≈ [0.2, ‑0.1, 0.5] (dₑₘb = 3) is stored.
When a new segment seg₂ arrives later, similarity sim(z₂, $\mu$₁) = 0.78 exceeds $\theta$ₛᵢₘ = 0.7, so seg₂ joins the existing scene, updating the centroid $\mu$₁ to [0.21, ‑0.09, 0.52].
The example shows how a tiny three‑message exchange is turned into a reusable memory cell and how similarity‑driven clustering grows a scene without needing the full conversation.
**Figure 7.** Shared group memory. (a) The host mediates access to owner-partitioned EverOS libraries. It retrieves experience before planning, deposits round summaries and outcomes in its shared library, and writes agent assessments to the corresponding agent libraries. Dispatch prefetch combines shared experience with recent assessments from a provenance ledger, with respective caps $k_{group}$ and $k_{verdict}$. Personal user memory is excluded, while workers retain their ordinary memory access. Offset outlines indicate one library and worker per mapped agent. (b) Assessment has two phases: a close verdict after execution and a feedback verdict on the next user turn. Attribution is extracted from the host's reply and omitted from the user-visible response and session log.
**Figure 10.** Raven's user-memory retrieval through EverOS HYBRID. (a) BM25 and dense search retrieve episodes. Reciprocal rank fusion (RRF) determines their expansion order. A separate calibrated score ranks them for final selection. (b) The query scores facts belonging to each expanded episode. Episodes and facts compete within a bounded candidate set. A selected fact is returned under its parent episode, which restores its context. The positive integer $k_{mem}$ is the backend candidate-set cap.
**Figure 9.** Memory formation and consolidation in EverOS. (a) Semantic boundaries group dialogue and tool interactions into a source segment $seg_i$. Its episode $ep_i$ anchors atomic facts. The dashed branch derives optional foresight with validity interval $[t_{p,0}, t_{p,1}]$. (b) Across interaction history Hist, episodes, shown as points, form MemScenes around centroids marked by crosses. These scenes select evidence for profile updates. The agent track consolidates execution cases $c_1, c_2, c_3$ into reusable procedures, depicted by a small dependency graph. The diagram shows representation and data dependencies. Persistence and indexing proceed separately.
**Figure.** (a) Two scores, two roles. (b) Fact Expansion and Response. A fact can displace its parent. The response restores episode context.
Skill Forge
Skill Forge curates a catalog, retrieves matching procedures, and updates them from execution experience.
Skill Forge turns raw execution cases into reusable procedures by (1) building a curated skill catalog, (2) retrieving the most relevant skills for the current task, and (3) feeding the execution experience back into the catalog for future reuse.
Compute RRF for each name using $w_j=1$ and $c_{\text{skill}}=1$: $\text{RRF}(A)=\frac{1}{1+1}+\frac{1}{1+2}=0.5+0.33=0.83$, $\text{RRF}(B)=\frac{1}{1+1}+\frac{1}{1+2}=0.5+0.33=0.83$, $\text{RRF}(C)=\frac{1}{1+2}=0.33$, $\text{RRF}(D)=\frac{1}{1+2}=0.33$.
Select representatives: for A choose $h_{11}$ (native score 0.9), for B choose $h_{31}$ (0.85), for C choose $h_{22}$ (0.8), for D choose $h_{32}$ (0.5).
Order names by RRF descending: (A, B, C, D). Take the top $k_{\text{pool}}=3$ → $\text{Pool}_0 = [$rep(A), rep(B), rep(C)$] = [$h_{11}$, $h_{31}$, $h_{22}$].
Assume no policy blocks or body‑fetch failures, so $\text{Pool} = \text{Pool}_0$.
The LLM gate receives the three candidates, discards any requiring unavailable tools, and returns identifiers $[\,\text{id}(h_{31}), \text{id}(h_{11})\,]$.
Apply the selection cap $k_{\text{sel}}=2$: $S_{\text{sel}} = [$rep(B), rep(A)$] = [h_{31}, h_{11}]$.
RRF equalizes the influence of each source, so even a weaker hub hit can win if the name appears across multiple sources, while the final representative ensures the richest body is used.
How does Skill Forge differ from a naïve retrieval‑only system that simply returns the top‑k hits from a single source?
Skill Forge first deduplicates and scores skills across three heterogeneous sources, then fuses them by exact name using Reciprocal Rank Fusion, which prevents a single source’s scoring bias from dominating. It also runs a policy‑aware LLM gate that checks tool availability and specialist overlap, and finally feeds execution experience back into the catalog, continuously improving skill quality—behaviors a plain retrieval system lacks.
Initialize the need flag and query; run the optional rewriter, falling back to the raw request on failure.
If the rewriter signals “skip”, abort early; otherwise use the (possibly rewritten) query $q$.
Query each configured source ($\text{local}$, $\text{memory}$, $\text{hub}$) concurrently for up to $2k_{\text{pool}}$ hits.
Compute name scores with the RRF formula and pick the top $k_{\text{pool}}$ names, keeping the highest‑scoring native hit as the representative.
Remove any blocked representatives, fetch missing bodies from the hub, and apply the detail policy; discard any candidate that fails.
If the resulting pool is empty, return an empty segment.
Run the LLM gate on the query, the pool, and the agent’s tool/specialist rosters; parse the identifier list or fall back on failure.
Form the final selected list $S_{\text{sel}}$ using the normal cap $k_{\text{sel}}$ or the fallback cap $k_{\text{fb}}$ as dictated by the gate outcome.
Attempt resource resolution for each selected entry under the configured policy, preserving bodies when resolution fails.
Render the selected entries with provenance and return the skill context.
Acquire the clustering lock for the owner, embed the case intent, and assign the case to an existing cluster or create a new singleton.
If embedding is unavailable or the case quality falls below $\theta_{\text{skip}}$, abort with no skill update.
Persist the cluster state together with the case identifier and embedding, then release the clustering lock.
Acquire the skill‑update lock, verify the cluster’s existence, and enumerate its Markdown skills.
Select up to $k_{\text{skill}}$ skills, using index ranking or back‑fill from the catalog when needed.
Collect supporting cases for each skill, sort them by quality then recency, and truncate to the per‑skill support cap.
Construct the LLM prompt (mode determined by case quality) and request an ordered list of operations (add, update, or none).
Iterate over the returned operations, validating indices, action types, and field contents; skip malformed or out‑of‑range entries.
Apply accepted operations to produce emitted skill records, preserving lineage $\Lambda(s^{+})$ and confidence clipping.
Write the emitted records, reconcile any renamed files, and release the update lock.
**Figure 11.** Skill selection and experience-driven updates. (a) Ranked source hits are grouped by name and fused before body retrieval, runtime policy checks, and LLM selection. The normal gate selects a bounded list, while a gate failure uses a separate ranked fallback. Resource resolution is attempted after selection. (b) A case enters an intent cluster and conditions skill operations together with existing skills and a bounded set of supporting cases. Persisted revisions become candidates for later EverOS retrieval after indexing. The positive integer $k_{pool}$ is the fused skill-pool cap. Numeric labels show the implementation defaults in Appendix B.3.
Only records that pass structural validation and deduplication reach this decision.
Experimental Results
We assess Raven’s orchestration, self‑evolution, and skill‑reuse across diverse benchmarks.
Recall that Raven composes model–harness pairs via a host that automates task assignment, information exchange, and execution, letting agents evolve policies and reuse procedural knowledge.
Raven outperforms both baselines on the Multi‑Agent Orchestration Benchmark across all four metrics.
Node F₁ $0.923$ vs $0.776$, Edge F₁ $0.897$ vs $0.813$, POA $0.950$ vs $0.854$, Exact Match $0.711$ vs $0.607$ (DeepSeek‑V4‑Flash backbone). +0.147 Node F₁, +0.084 Edge F₁, +0.096 POA, +0.104 Exact Match
**Figure 13.** Multi-Agent Orchestration Benchmark results with backbone (a) Qwen3.8-27B and (b) DeepSeek-V4-Flash-0731.
**Figure 15.** Raven-Research results on DeepResearch Mixed. (a) Accuracy per source under DeepSeek-V4-Flash. (b) Paired accuracy gain of Raven-Research over each comparison on the same backbone, with stratified bootstrap 95% intervals over all questions. The hollow marker denotes the comparison with DeepSeek-Harness on Qwen3.5-397B-A17B, with exact McNemar $p = 0.21$. (c) Accuracy against mean cost per question, with filled markers for the DeepSeek-V4-Flash harnesses and hollow markers for the commercial services, whose costs follow each provider's own accounting.
**Figure 16.** Coding results. Each row is one system, grouped by benchmark and backbone. SWE-bench reports the resolved rate, WorkBuddy-Code the reward, and SWE-Refactor the mean task score. Hollow markers are public leaderboard values.
**Figure 18.** PresentBench overall scores (0–100). Gold bars are Raven-Design and dark bars Claude Code, with the backbone in parentheses. Light bars are public leaderboard entries evaluated by the benchmark authors, whose model configurations are not given.
Raven consistently outperforms baselines across coding, research, and design tasks.
Related Work
Survey of prior approaches to multi‑agent orchestration, harness design, memory, and skill management.
Multi‑agent collaboration can be organized around conversation, fixed workflows, dynamic task states, or executable dependency graphs, each exposing different inspection points and failure modes.
Conversation‑oriented frameworks such as Auto‑Gen and MetaGPT let developers compose agents via message exchanges, but they do not provide a typed graph that can be validated before execution.
Systems like MacNet (DAG), Magnetic‑One (task ledger), MasRouter, and Dang et al. make execution structure explicit or adaptive, yet they generally assume a stable roster and a learned routing policy.
Scaling studies (Kim et al., AgentBench, MAST) demonstrate that merely adding agents does not guarantee better performance; coordination gains depend on task structure and system architecture.
Formal methods such as Hoare logic, separation logic, and assume‑guarantee reasoning provide vocabularies for making orchestration contracts testable.
Raven’s Host Agent implements this contract perspective by accepting a DAG where each node specifies an executor, prompt template, dependencies, and optional stateful handles, with runtime checks for identifiers, acyclicity, scope, and budget.
An agent harness defines the execution policy around a language model, encompassing context, reasoning loops, tool interfaces, memory access, and recovery behavior; examples include ReAct and SWE‑agent.
Several works improve only parts of this policy—GEPA evolves prompts, ADAS searches agent designs, AFlow generates workflow programs, and AgentSquare recombines modular components.
Open‑ended evolution approaches such as the Darwin Gödel Machine and Growing Harness edit executable agent code, while MAP‑Elites and HarnessBank retain high‑performing elites and apply gated screening.
Raven factorizes a harness into Memory, Planning, Capability, and Action interfaces, allowing targeted edits to be attributed to specific diagnosis categories during evolution.
Memory research distinguishes factual, experiential, and working memory; early systems like MemGPT, Generative Agents, A‑MEM, and Zep explore trade‑offs between compact context, associative retrieval, and temporal structure.
Production‑oriented memory systems (Mem0, MemoryOS, MemOS, MIRIX, Letta) address scale, asynchronous updates, and substrate separation, motivating explicit scope and temporal information for records.