From Traces to Agentic Worlds: Agentic Language World Models for Interactive Environment Simulation
Quanyu Long, Xiao Chen, Jianda Chen, Haozhen Zhang, Qisheng Hu, Jianzhu Bao, Wenya Wang
Trace2Env reconstructs non-executable environment simulators from interaction traces to support faithful, stateful agent evaluation.
How can we reconstruct a reusable, stateful environment simulator from historical interaction traces without needing access to the original system?
Realistic environment replicas are often impossible to rebuild because the original code, infrastructure, or hidden state is inaccessible. Relying on simple prompt-based language models to simulate these environments leads to inconsistent behavior, as they struggle to track state changes across long interaction sequences. Trace2Env solves this by treating environment simulation as an agentic task: it reconstructs a reusable "worldbook" from historical traces and uses a dedicated world model agent to actively inspect this knowledge. This agent maintains a persistent episode workspace, allowing it to track state changes and enforce environment constraints that persist across multiple turns. Across nine environments, Trace2Env improves next-observation fidelity and long-horizon consistency, ensuring that actions generated in simulation remain valid when replayed in the real environment.
Paper Primer
The core mechanism is a two-phase framework: an offline constructor distills raw traces into a structured worldbook, and an online agentic simulator operates that worldbook. The worldbook acts as a reference library: it contains schemas, grounded evidence, and induced behavioral rules that the world model agent queries to determine how the environment should respond to a given action.
The simulator is like a librarian: it doesn't "be" the environment, but it actively consults a catalog of past behaviors and current episode notes to answer the agent's queries, ensuring that if a file was deleted in turn one, the simulator remembers it is missing in turn ten.
Trace2Env significantly improves next-observation prediction fidelity over conventional prompt-based language world models.
Across 14 environment-backbone pairs, Trace2Env outperformed direct prompting in 11 cases and never underperformed it, achieving average score gains of 6.43 to 7.04 points. 75.94 average score (GPT-5.6-sol) vs. 69.51 for direct prompting.
Trace2Env preserves long-horizon interaction consistency, ensuring simulated behavior transfers to real environments.
In multi-turn tasks like ALFWorld and SciWorld, actions generated by Trace2Env show significantly higher success rates when replayed in the real environment compared to those generated by standard prompting. Consistency ratio (W2R/Real) improved from 0.032 to 0.914 in ALFWorld.
Why is this approach better than simply retrieving raw traces (RAG) for the model to read?
Raw retrieval provides isolated examples, but Trace2Env organizes these into structured schemas and induced rules. This allows the simulator to reason about state changes and constraints that aren't explicitly present in any single retrieved trace.
Does this method require the original environment to be executable during the simulation phase?
No. The framework is designed specifically for settings where the original system is unavailable; it relies entirely on historical interaction traces to reconstruct a behavioral specification that the world model agent then simulates.
Researchers can now build faithful, stateful environment replicas for agent training using only historical logs, bypassing the need to reverse-engineer complex, proprietary software systems.
Introduction
Introducing Trace2Env, which models environments from interaction traces instead of rebuilding them.
Realistic environment replicas are increasingly valuable for training and evaluating agents, yet many original systems are inaccessible or impractical to reproduce. Historical interaction traces, however, often remain available, recording actions, observations, and their consequences. This motivates a shift from rebuilding full executables to modeling environments directly from traces.
Trace2Env turns archived interaction traces into a structured “worldbook” that a dedicated world‑model agent can query to simulate environment responses.
**Figure 1.** From interaction traces to an agent-operated simulated environment. Trace2Env reconstructs traces into a reusable environment worldbook containing schemas, grounded evidence, and induced behavioral knowledge. During simulation, a task agent supplies actions, while a world model agent actively consults the worldbook together with the current episode state and episodic memory to infer their consequences and predict the resulting observations.
The key insight is shifting from rebuilding full environments to modeling them via trace‑derived worldbooks.
Related Work and Baselines
Contextualizes prior world‑model work and formalizes the trace‑only setting.
Prior language world models (LWMs) predict action consequences, but recent work augments them with external experience or trace‑based reconstruction.
Preliminaries formalize the LWM interface and the trace‑only reconstruction problem.
Ask the language world model directly for the next observation by embedding the current action in a concise prompt.
Retrieve relevant past trace snippets and prepend them to the prompt, letting the LWM condition on concrete historical evidence.
The Trace2Env Framework
We describe how Trace2Env builds a reusable worldbook and uses a world model agent to simulate episodes.
When reusing past interaction traces, the key difficulty is mixing reusable behavioral knowledge with episode‑specific facts, which can corrupt a new simulation if not separated.
From the traces the constructor builds a worldbook $K_E$ containing a schema for the
During a new episode the workspace $W_t$ holds $K_E$, an empty represented state $\hat{s}_t$ (no files known yet), and an empty memory $M_t$.
The agent proposes
The validation gate accepts the proposal (schema satisfied) and commits a tentative state update that
The worldbook supplies reusable action semantics, while the mutable state $\hat{s}_t$ supplies episode‑specific facts; the harness guarantees they never overwrite each other unintentionally.
The worldbook is an immutable package of schemas, evidence, abstractions, and provenance extracted from build‑time traces.
How does a worldbook differ from a conventional world model?
A conventional world model learns a monolithic function from data, while the worldbook is a curated collection of explicit schemas, concrete evidence, and abstract rules that are manually induced from traces and kept immutable for reuse.
The world model agent is a conditional generator that, given the current workspace $W_t$ and an action $a_t$, proposes a set of candidate state effects and an observation.
Why does the agent propose a transition instead of updating the state immediately?
Separating proposal from commitment lets the harness enforce schema constraints and resolve ambiguities (e.g., format‑only evidence) before any irreversible change, preserving consistency across episodes.
Offline: run the constructor $\text{Construct}_{\phi}$ on build traces $D_{\text{build}}$ to produce the immutable worldbook $K_E$.
Online rollout start: initialize the workspace $W_0 = (K_E, \hat{s}_0, M_0)$ where $\hat{s}_0$ and $M_0$ are empty.
For each timestep $t$, the world model agent receives action $a_t$ and queries $W_t$.
The agent retrieves candidate entries, runs the applicability gate $g^{\text{app}}_t$, and samples a proposal $q_t$.
The validation gate $g^{\text{val}}_t$ checks $q_t$ against schemas, abstractions, and constraints.
If accepted, the harness $H$ commits the effects, producing the next state $\hat{s}_{t+1}$ and observation $\hat{o}_{t+1}$, and updates memory $M_{t+1}$.
Agentic simulation loop – the world model agent proposes, the harness validates and commits.
**Figure 2.** Trace2Env framework. Trace2Env reconstructs historical traces into an environment worldbook and uses a world model agent to operate it as a stateful simulated environment.
Experimental Results
Trace2Env consistently outperforms baselines across diverse environments.
Trace2Env achieves the highest average score across both backbones, reaching 75.94 with gpt‑5.6‑sol and 75.75 with deepseek‑V4.1‑flash, improving over Direct Prompting by 6.43 and 7.04 points respectively.
Table 1 shows Trace2Env leading in 11 of 14 environment–backbone pairs and never underperforming Direct Prompting.
**Table 1.** Performance comparison of different prompting methods across various environments using two different world-model backbones (gpt-5.6-sol and deepseek-V4.1-flash).
**Table 2.** Task success rate in the real environment (Real), the simulated world model (WM), and real-environment replay of world-model-induced actions (W2R); higher CR indicates better transfer.
**Table 3.** Worldbook component ablations on Terminal. Effects are paired score differences relative to the corresponding variant without the added component; ± denotes standard error.
**Figure 3.** Trace scaling on Terminal. Next-observation prediction score and one-time worldbook construction cost as the number of construction traces increases.
Failure Analysis and Qualitative Results
Provides a quick guide to the appendix material and highlights Figure 4’s qualitative insight.
This section walks you through the supplemental material that supports the main experiments. It lists the appendix organization, points out the key qualitative illustration (Figure 4), and summarizes the environment‑level statistics in Table 4.
**Figure 4.** The task agent generates an unsupported put action. Direct Prompting hallucinates a successful placement, causing a false state transition and a simulated success that fails under real-environment replay. Trace2Env not only predicts the correct no-op, but also uses its worldbook and persistent state tracking to preserve the inventory state and support subsequent interaction, enabling recovery with the valid move command.
Appendix A‑E enumerate the full experimental setup, extended results, implementation specifics, case‑study details, and prompt templates. Table 4 aggregates worldbook statistics across nine benchmark environments, reporting the number of trajectories, actions, state fields, executable‑rule candidates, notes, and evidence turns for each domain.
Case Study Analysis
Ablation results show how worldbook knowledge and persistent state each boost prediction fidelity.
Trace2Env rebuilds past interaction traces into a worldbook, letting the agent simulate environment transitions without the original system.
Removing the worldbook component causes the prediction score to fall from $90$ to $30$ on the heredoc terminal task.
Trace2Env achieves a judge score of $90$, whereas Direct Prompting scores $30$.
Removing persistent package state reduces the prediction score from $100$ to $55$ on the pip‑install task.
Trace2Env receives a judge score of $100$, while Direct Prompting gets $55$.
Without Trace2Env’s failure handling, the task agent cannot recover, yielding zero reward versus a reward of $1$ with Trace2Env.
Trace2Env rollout yields reward $1$, whereas Direct Prompting fails to achieve the goal.
Experimental Setup
Details the data collection, worldbook construction, and evaluation protocols across nine environments.
Table 4 provides a compact overview of the nine environments, listing the number of rows per trajectory, the action‑to‑observation mapping, and detailed construction trace statistics.
When the original environment can be executed we collect independent construction trajectories; otherwise we rely on cross‑fitting, partitioning benchmark trajectories into folds and building worldbooks from out‑of‑fold data.
Evaluation treats whole trajectories as the statistical unit: for AgentWorldBench every record is a prefix of a trajectory, and for Android we recover 92 distinct trajectories from the merged identifiers.
EnvScaler evaluates Food, Shopping, and Benefits using the held‑out records listed in Table 4, while ALFWorld and SciWorld are assessed through multi‑turn interactions rather than single next‑observation predictions.
Construction for Terminal uses a gpt‑5.6‑sol Terminus‑2 agent on Terminal‑Bench 2.0, sampling 20 successful trajectories for the worldbook and reserving the rest for the construction‑size ablation described later.
Web’s worldbook comes from 50 trajectories collected with a gpt‑5.6‑sol task agent; of the 200 evaluation records, 151 cannot be linked to a construction task and are reported separately.
Android and SWE employ five‑fold cross‑fitting, ensuring each evaluation record is predicted using a worldbook built exclusively from trajectories outside its own fold.
EnvScaler gathers trajectories via a GPT‑5.6‑sol agent, sampling 16, 3, and 15 for Food, Shopping, and Benefits respectively; the worldbooks contain only task instructions, tool actions, and observed responses.
ALFWorld holds out the first 100 Word2World test tasks for evaluation, draws construction candidates from the next 100, filters out overlapping goal families, and retains 40 successful simulator‑confirmed trajectories (836 turns) for the worldbook.
All worldbooks are reconstructed with the Trace2Env pipeline, frozen, and reused unchanged for both gpt‑5.6‑sol and deepseek‑v4.1‑flash backbones; cross‑fitted environments select the appropriate out‑of‑fold worldbook per record.
We compare Trace2Env against four baselines: Direct Prompting follows the AgentWorldBench protocol; Harness uses the same agent loop without a worldbook; Trace RAG Prompting retrieves raw transitions as context; Worldbook Prompting supplies the static worldbook as prompt.
Scoring follows the official gpt‑5.2 judge; pairwise differences are reported as mean paired differences on a 0–100 scale, with uncertainty estimated via a paired bootstrap over trajectories (10 000 resamples) and 95 % confidence intervals.
Leakage control guarantees that construction data never includes the target observation of any evaluation record; for Terminal and Web we additionally separate tasks that appear in the construction set, and cross‑fitting inherently prevents leakage for Android and SWE.
Cross‑fitting builds worldbooks from trajectories that belong to folds different from the evaluation record, ensuring no overlap between training and test data.
**Table 4.** Environments, evaluation data, and construction data. Worldbook statistics for cross-fitted environments are ranges over the per-fold worldbooks.
Detailed Performance Metrics
Appendix B breaks down Trace2Env’s gains and isolates its component contributions.
This appendix breaks down the aggregate scores and isolates the contributions of Trace2Env’s components.
**Table 5.** Five-dimension judge scores, GPT-5.6-Sol. best total per environment in bold.
Across all 14 environment–backbone pairs, Trace2Env improves every judge dimension, with only minor drops in Format for Android and Web.
**Table 9.** Complete Terminal Ablations
These analyses demonstrate that both the reconstructed worldbook and the efficient runtime are essential for the observed gains.
Performance Trade-offs (I)
Performance tables and unseen‑task transferability for the evaluated methods.
Appendix B Part 1 presents the quantitative comparison of the five methods on a suite of environments and examines how well the models transfer to tasks unseen during construction.
**Table 6.** Five-dimension judge scores, DeepSeek-V4.1-Flash. Layout as in Table 5.
**Table 8.** Worldbook gain by construction overlap (mean ± SE, clustered 95% CI). Rows marked † report Trace2Env – Direct Prompting; the others report Trace2Env – Harness only.
Section B.3 evaluates unseen‑task transferability. The numbers show that, even when the test tasks were not present during construction, Trace2Env maintains high scores (often above 80 % and reaching 100 % in several cases), whereas the other methods exhibit larger drops.
Performance Trade-offs (II)
Aggregated performance statistics for the prompting variants listed in this appendix.
This appendix enumerates a large collection of accuracy scores for each prompting strategy across multiple evaluation settings.
Performance Trade-offs (III)
Efficiency–accuracy trade‑offs and component ablations for Trace2Env are detailed here.
The authors observe that Trace2Env’s improvements persist even when the test tasks do not overlap with the construction traces, confirming that the worldbook encodes reusable environment dynamics rather than mere memorisation of specific tasks.
Table 9 reports ablation differences relative to the full Trace2Env system. Removing the adaptive loop cuts the score by 1.79 ± 0.91 points, while dropping persistent state tracking reduces the score by 0.97 ± 0.78 points, illustrating that both the loop and the state‑tracking mechanism contribute measurable performance gains.
Implementation Overview
Implementation details for the worldbook pipeline and its artifacts.
Appendix C provides the implementation details for the worldbook pipeline, covering the four sub‑components introduced in Section 4. It enumerates the worldbook components, their reconstruction process, the simulation harness, and the experimental settings used in the reported runs.
Trace2Env implements an agentic Language World Model that predicts the consequences of actions within the worldbook.
**Table 11.** Inference cost per record across environments for the reported runs (list prices; GPT-5.6-Sol / DeepSeek-V4.1-Flash). Single-call systems cost one prompt of the stated size.
The worldbook artifacts are organized into schemas, grounded evidence, induced abstractions, state constraints, conventions, and content descriptors, each linking back to the original episodes.
**Table.** Validation, rendering, and commitment.
Worldbook Artifacts
Details the worldbook’s components, how they are rebuilt, and the simulation harness that uses them.
Section C.1 enumerates the four artifact families that compose a worldbook KE: schemas, grounded evidence, induced abstractions, and provenance.
Schemas declare variables in four groups—world (durable facts), session (interaction context), surface (currently visible state), and epistemic (uncertain or conflicting beliefs); for example, the terminal worldbook lists a file map and installed programs under world, a working directory under session, and a flag indicating whether a foreground program owns the terminal under surface.
Grounded evidence stores each aligned transition verbatim, preserving any detail that no abstraction captured; deterministic demonstrations are selected subsets of these transitions.
Induced abstractions are rules that map conditions over state and action arguments to ordered effects; each rule is marked as supported, tentative, or conflicted, and only supported rules become executable.
Provenance attaches references to the source episodes and turns that justify each abstraction and demonstration, enabling the agent to trace an abstract effect back to its concrete evidence.
Section C.2 describes the reconstruction pipeline (Figure 2): first align actions with results using call identifiers when present or adjacency otherwise, then extract facts, outcomes, candidate effects, and uncertainty while preserving the original observation.
Next, schemas and abstractions are induced from the extracted records; deterministic checks fill in missing action names, arguments, and state variables, and proposed rules are reviewed against supporting evidence and success–failure contrasts, with only those meeting compilation criteria becoming executable.
Section C.3 introduces the simulation harness that operates the workspace $W_t = (KE, \hat{s}_t, M_t)$; the state is initialized from a possibly partial seed, and worldbook examples do not seed episode facts.
Algorithm 1 (summarized in the text) executes a single step under the reference policy, using the reconstructed worldbook as a fixed source of schemas and abstractions throughout the simulation.
Simulation Harness Details
Details of the simulation harness, hyperparameters, and case‑study comparisons.
This appendix formalizes the harness that drives a single simulation step and enumerates the hyperparameters used throughout the experiments.
Algorithm 1 – Simulating one action under the reference policy.
Retrieval first combines an action‑type lookup with a lexical search; the deterministic gate $G_{\text{app}}$ then assigns applicability labels based on episode‑local operands such as terminal identifiers or page names.
With the retrieved entries, the agent builds a brief that concatenates the action, current state, memory, and applicable entries, then inspects schemas, rules, and evidence before proposing a query $q_t = (\Delta_t, \tilde{o}_{t+1})$.
Validation ($G_{\text{val}}$) checks the proposed effects against mutability, type, and rule constraints on a temporary copy of the workspace; if the check fails, the harness falls back to an observation‑only response that discards any state changes.
For next‑observation evaluation, observed turns are first loaded into memory; any rule templates that match the observed output are applied, while model‑proposed effects are schema‑checked and recorded as diagnostics rather than grounds for rejection.
**Table.** Case Study
The rollout is internally consistent with the world model's earlier false transition, but the resulting actions do not complete the task in the real environment.
Prompting Strategy
This section lists the prompts used for Direct Prompting and the Trace2Env language world model.
Direct Prompting uses the official AgentWorldBench system prompt, which supplies the terminal role, objective, and task definition to the model. The prompt includes the historical context, current terminal state, and user action, and asks the model to predict the next observation. Repetitive command‑specific sections and few‑shot examples are omitted for brevity.
Trace2Env’s Language World Model treats the terminal as a stateful environment, representing the visible screen, prompt, and implicit filesystem state. It updates this state on command execution, interactive program changes, process completions, and signal handling, while tracking working directory, environment variables, and file modifications across turns. The model assembles output by echoing the command, emitting stdout/stderr, and producing a new prompt, preserving exact formatting and logical consistency.
Baseline Prompting Details
Defines single‑call baselines and their prompt templates for reproducible evaluation.
The section first outlines quality criteria, emphasizing output completeness, sufficient detail for downstream reasoning, and inclusion of necessary diagnostics.
It then defines the input space, describing how agents interact with a terminal environment via sequences of JSON‑encoded actions.
Each action follows the Action Format: a list of objects containing a $keystrokes$ string and a $duration$ number, where $duration$ defaults to 1.0 seconds if omitted.
The field table specifies three columns—$keystrokes$, a raw terminal command; $duration$, the wait time before capturing output; and their types, with $keystrokes$ treated verbatim and $duration$ expressed in seconds.
Predicted observations must return four items: the echoed command, the full stdout/stderr, the next prompt (if the command finished), and exact whitespace preservation.
Few‑shot examples illustrate multi‑turn terminal interactions, showing how commands, delays, and outputs are concatenated in chronological order.
Section E.2 introduces two single‑call baselines that augment the official AgentWorldBench prompt with an extra context block.
Trace RAG Prompting appends a “# Retrieved examples …” block containing up to five previously recorded turns whose commands resemble the current action, preserving their raw observations for reference.
Worldbook Prompting instead appends a “# Reconstructed environment package …” block that statically presents the reconstructed worldbook’s schemas, induced knowledge, demonstrations, and retrieved evidence.
Worldbook Construction Details
Describes the pipeline that turns raw traces into a reusable Worldbook.
Worldbook construction proceeds in stages over the collected traces: transition extraction, schema induction, rule induction, renderer‑contract induction, and note induction.
Transition extraction runs once per recorded transition, prompting a forensic analyst to isolate the normalized action, outcome, minimal state mutations, preconditions, observation facts, latent variables, and ambiguities, while preserving provenance and explicit uncertainty.
Schema induction takes the local evidence produced by the previous stage and induces the smallest domain‑independent action and state interface, merging aliases, declaring argument types, and preserving path prefixes such as world., session., surface., or epistemic..
Rule induction creates contrastive transition rules by aggregating multiple evidence records, specifying applicability conditions, effects, outcomes, renderer contracts, confidence, scope, and provenance, while separating success and failure branches and treating repeated correlations as tentative unless supported by an invariant.