Qwen-AgentWorld: Language World Models for General Agents

Yuxin Zuo, Zikai Xiao, Li Sheng, Fei Huang, Jianhong Tu, Yuxuan Liu, Tianyi Tang, Xiaomeng Hu, Yang Su, Qingfeng Lan, Yantao Liu, Qin Zhu, Yinger Zhang, Bowen Yu, Haiquan Zhao, Haiyang Xu, Jianxin Yang, Jiayang Cheng, Junyang Wang, Lianghao Deng, Mingfeng Xue, Tianyi Bai, Yang Fan, Yubo Ma, Yucheng Li, Zeyu Cui, Zhihai Wang, Zhihui Xie, Zhuorui Ye, An Yang, Dayiheng Liu, Jingren Zhou, Ning Ding

Qwen-AgentWorld introduces language world models that simulate environment dynamics to improve agent reasoning and planning.

How can we train a single language model to act as a high-fidelity simulator for diverse agent environments, from terminal commands to GUI interactions?

Current LLM agents focus on policy (state-to-action) but lack a general-purpose world model to predict environment feedback (action-to-state), leaving them unable to anticipate consequences or handle complex, multi-step interactions reliably. The authors introduce Qwen-AgentWorld, a foundation model trained to simulate seven agentic environments—including Terminal, Android, and Software Engineering—by predicting next-state observations through long chain-of-thought reasoning. Qwen-AgentWorld outperforms existing frontier models on the new AgentWorldBench, while enabling scalable, controllable environment simulation for agentic reinforcement learning.

Paper Primer

The method hinges on a three-stage training pipeline: Continual Pre-training (CPT) injects environment dynamics and world knowledge, Supervised Fine-tuning (SFT) activates explicit next-state prediction as a reasoning pattern, and Reinforcement Learning (RL) sharpens simulation fidelity using a hybrid rubric-and-rule reward system. This approach treats environment simulation as a conditional text generation task, where the model learns to map interaction history and actions to the subsequent environment state.

Qwen-AgentWorld significantly outperforms existing frontier models in environment simulation fidelity.

Evaluation on AgentWorldBench, a benchmark covering seven domains (e.g., Terminal, Android, SWE) using ground-truth rubric judging across five dimensions. The model achieves superior performance across all seven domains, with factuality showing an 11.3% relative improvement during RL training.

Beyond foundation modeling, the authors demonstrate two practical paradigms: using the world model as a decoupled simulator to scale agentic RL without real-world infrastructure, and using world-model training as a warm-up stage to improve downstream agent performance on benchmarks like SWE-Bench and Terminal-Bench.

Why is a language-based world model necessary if real environments are already available?

Real environments are often slow, irreversible, or proprietary. A world model provides turn-level scalability and precise controllability, allowing agents to train against targeted perturbations and edge cases that are rare or impossible to trigger in live systems.

How does the model avoid "reward hacking" during the RL stage?

The authors combine open-ended rubric rewards with rule-based verifiers that provide binary correctness signals. They also use content-type classification to ensure deterministic outputs are judged by exact match, preventing the model from inflating scores through self-praise.

The paper assumes that world modeling is a necessary prerequisite for general-purpose agents, positioning next-state prediction as a "meta-level thinking pattern" that allows agents to refine action selection by simulating future outcomes.

Researchers can now treat environment simulation as a first-class training objective, using Qwen-AgentWorld to bootstrap agentic RL or as a standalone simulator to stress-test agents in controlled, reproducible settings.

Introduction and Motivation

We expose the missing internal simulation in LLM agents and present Qwen‑AgentWorld as a language‑based world model.

LLM agents excel at mapping states to actions, yet they lack a core component: an internal model that predicts how the environment will respond to those actions. Without such simulation, agents must react to feedback after the fact, limiting their ability to plan and reason about future outcomes.

Current language‑model agents operate purely reactively—they see a state, choose an action, and only then observe the next state, never anticipating the consequences beforehand.

The model receives the current observation and the agent’s action, then predicts the next observation as the next token in a long text sequence, turning environment dynamics into a language modeling problem.

To measure simulation quality, the authors construct AgentWorldBench, aggregating real‑world interaction data from five frontier models across nine established benchmarks (e.g., Tool Decathlon, Terminal‑Bench 1.0 & 2.0, OSWorld‑Verified) and evaluating five rubric dimensions.

Empirical results show that Qwen‑AgentWorld consistently surpasses existing frontier models on AgentWorldBench, confirming that language‑based world modeling can materially improve simulation fidelity.

The paper explores two complementary ways to exploit the world model: (i) decoupling it as an external simulator for scalable, controllable training of thousands of environments, and (ii) unifying it with the policy to form a single agent foundation that benefits downstream RL.

**Figure 1.** Building the Foundation Model for Agentic Environment Simulation

The key shift is moving from purely reactive agents toward agents that reason with an internal world model.

Formalizing the Language World Model

Unify seven domains under a single textual trajectory format for a language world model.

Existing agents treat each environment separately, forcing a bespoke data pipeline for every domain. This fragmentation prevents knowledge transfer and inflates engineering effort. The trick is to force every interaction into a single, text‑based trajectory so a single language model can learn across domains.

LWM is a conditional text generator that, given a full interaction history and the agent’s current action, writes the next environment observation as plain text.

How does an LWM differ from a vanilla language model that predicts the next token?

A vanilla model conditions only on the preceding token stream, whereas an LWM conditions on a structured prompt, the full history of (action, observation) pairs, and the current action. This extra conditioning lets it generate an entire environment response rather than a single token.

Model receives $c$ (task description, action space, initial state) concatenated with $(a_1, o_1)$.

It predicts $\hat{o}_2$ = "test\n".

Because the history includes the created folder, the prediction correctly lists the new directory.

The LWM’s prediction depends on the cumulative state encoded in the prior action–observation pairs, not just the last command.

The schema packs a static system prompt together with an ordered list of (action, observation) turns, yielding a single text sequence that any language model can ingest.

Why concatenate the system prompt with the turn list instead of feeding them as separate inputs?

Language models operate on a single token stream; concatenation preserves the causal order and lets the model attend to the prompt while processing each turn, ensuring the prompt’s constraints influence every prediction.

Concatenate $c$ with $(a_1, o_1)$ to form the prefix for turn 2.

The model receives this prefix and predicts $\hat{o}_2$ = HTML of order‑confirmation screen.

Because $c$ already encoded the UI hierarchy, the model can infer the correct DOM transformation after the second tap.

The same textual format lets the model reuse its reasoning about UI structure across both turns without any architectural change.

**Figure 2.** Qwen-AgentWorld unifies seven categories of interactive environment simulation within a single language world model.

**System Prompt — Terminal LWM RL** **[1] Task Description** You are a Terminal World Model — a precise terminal state simulator. Your task is to predict the exact output of a Linux/Unix terminal after executing a given command or sequence of commands. Your goal is to be as faithful as possible to real terminal behavior while maintaining consistency and logical correctness across the interaction sequence. *Given:* (1) Historical Context (optional); (2) Current Terminal State; (3) User Action (keystrokes). *Predict:* the exact next terminal state after all actions are executed. Core Responsibilities: State Prediction | Context Maintenance | Behavioral Fidelity [...state transition rules, side-effect tracking, error handling, output formatting ...] **[2] Action Space** User actions are JSON arrays of command objects: `{"keystrokes": "ls -la\n", "duration": 0.1}` Keystrokes: `\n` = execute; `C-c` = SIGINT; `C-d` = EOF; `C-z` = SIGTSTP; `C-l` = clear screen Shell constructs: pipes, redirects, command chaining; Programs: shell builtins, editors (vim/nano), REPLs, pagers **[3] Initial State** The initial container snapshot of the terminal environment is: OS: Ubuntu 22.04.3 LTS (`x86_64`) Kernel: Linux 5.15.0-134-generic Packages: Python 3.10.12, pip 23.2.1, gcc 11.4.0, git 2.34.1, Docker 24.0.5 Disk: 64 GB (41 GB free) RAM: 16 GB Working dir: /workspace The initial terminal state is: root@6b254155-5503-4b86-837b-fd0f080ab297:/workspace# **[4] Demonstrations** Turn 1 | Action: "git clone https://github.com/expressjs/express.git /tmp/express\n" Observation: root@6b254155:/workspace#git clone https://github.com/expressjs/express.git /tmp/express Cloning into '/tmp/express'... Turn 2 | Action: "", duration=3.0 Observation: remote: Enumerating objects: 32841, done. remote: Counting objects: 100%(1205/1205), done. Receiving objects: 100%(32841/32841), 12.76 MiB | 8.53 MiB/s, done. Resolving deltas: 100%(21567/21567), done. root@6b254155:/workspace# [...6 more turns demonstrating directory navigation and build operations ...] **[5] Simulation Instruction** Simulate a system with CUDA 11.8 drivers and only 2GB free disk space. pip install torch==2.1.0 should complete the download phase normally (progress bar reaching 100%), then fail during unpacking of `libtorch_cuda`.so with OSError: [Errno 28] No space left on device. After the failure, pip cache list should report the partially downloaded .whl file still present. df -h should show /tmp at 100%usage. See §6.1.2 for the controllable simulation capability.

**Table.** The table categorizes various domains by their associated actions, observations, and core capabilities. It lists seven domains: MCP, Search, SWE, Terminal, Android, Web, and OS. Each row maps a specific domain to its primary interaction method (e.g., "JSON Tool Call" for MCP, "Touch / Swipe / Type / ..." for Android) and the corresponding data observed by the agent, alongside the required core reasoning capability, such as "Factual world knowledge" or "Visual state reasoning."

The Three-Stage Training Pipeline

The section details the three‑stage training pipeline and the data processing that powers it.

Training a world‑modeling language model at scale runs into two bottlenecks: insufficient, diverse environment trajectories and an unwieldy objective that mixes raw token prediction with control‑oriented behavior.

**Figure 5.** Three-stage training pipeline of Qwen-AgentWorld. Stage 1 CPT injects world knowledge; Stage 2 SFT instills next-state-prediction thinking patterns; Stage 3 RL sharpens output quality.

The model first learns a generic simulation of environments, then learns to reason about the next state, and finally refines its predictions with task‑specific rewards.

CPT: the model sees 20 tokens total, learns generic environment dynamics.

SFT: the model receives a 12‑token prompt that includes a system instruction to predict the next observation, reinforcing the next‑state pattern.

RL: the model is given an 8‑token prompt and a reward of 22 (mean rubric = 4.4 × 5) plus a rule‑based bonus of 2, total 24.

After the three passes, the model’s parameters have been updated three times, each time focusing on a different learning signal.

The pipeline isolates knowledge acquisition, reasoning activation, and reward‑driven refinement, so improvements in one stage do not overwrite the gains of the others.

How does this three‑stage pipeline differ from a single “pre‑train‑then‑fine‑tune” workflow?

In a single workflow the model would see only one objective and one data distribution, mixing world knowledge with task‑specific signals. Here each stage sees a pure signal (knowledge, reasoning, reward) and a dedicated data pool, preventing catastrophic forgetting and allowing targeted curriculum design.

Training data are assembled from three complementary sources so that the model observes both controlled and wild‑type interactions.

First, dedicated agent infrastructure runs sandboxed code, persistent terminals, and GUI environments, continuously generating synthetic queries and their full execution traces.

Second, open interaction traces harvested from public repositories provide long‑tail, noisy examples such as real terminal recordings and tool‑call logs.

Third, in‑house agentic trajectories collected during model development are converted into the unified trajectory format.

All raw sources are normalized to a common schema: a system prompt followed by alternating user (action) and assistant (observation) turns.

Trajectory‑to‑Turn Expansion turns each multi‑turn trajectory into many turn‑level prediction samples by pairing every turn t with its preceding context as input and the observation at t as target.

Data filtering removes short trajectories, tool‑mismatches, and broken GUI runs, then applies two domain‑specific filters.

System prompts consist of five components; the action space and demonstrations are static for most domains, while the initial state and simulation instruction are dynamic per trajectory.

Prompt engineering is automated via the AutoResearch pipeline: an optimizer agent proposes prompt variants, the model evaluates them on held‑out trajectories, a judge scores predictions, and the optimizer refines the prompt iteratively.

Stage 1 (CPT) trains on next‑token prediction over the expanded turn samples, mixing environment trajectories with large factual corpora to inject broad world knowledge.

To focus learning on informative turns, a turn‑level information‑theoretic loss mask computes Overlap, Novelty, Jaccard, and length‑ratio statistics and assigns each turn to one of seven semantic categories with keep ratios (Table 3).

Stage 2 (SFT) diversifies system‑prompt templates (10 variants) and applies rejection sampling: three rollouts per query are judged, the best is kept if it exceeds a quality threshold, yielding a 69 % retention rate (Table 4).

Stage 3 (RL) uses GSPO with a hybrid reward: a five‑dimensional rubric (mean × 5) combined 9:1 with a binary rule‑based verifier, and caps prompts at 128 k tokens to keep compute dominated by context processing.

Training stability is achieved by limiting turn expansion to a single target per trajectory (avoiding shared‑prefix collapse) and by shaping the reward to balance rubric feedback with the strict verifier.

Overall, the three‑stage pipeline, together with rigorous data curation, loss masking, and reward design, yields a controllable world‑model that can be fine‑tuned for downstream agent tasks.

Evaluating Simulation Fidelity

AgentWorldBench quantifies long‑context simulation fidelity across seven domains.

AgentWorldBench comprises 2,170 evaluation samples spanning seven interaction domains, providing a large‑scale, long‑context testbed for language world models.

Samples are drawn from frontier‑agent trajectories on nine benchmark query sets, with real‑environment ground truth for each turn.

AgentWorldBench is a curated suite that asks a language world model to predict the next environment observation given the full prior interaction history, measuring how faithfully it can simulate real‑world dynamics over many turns.

Main Evaluation Results

Qwen-AgentWorld tops AgentWorldBench, showing sizable gains from world‑model training.

Qwen‑AgentWorld‑397B‑A17B achieves the highest overall average score on AgentWorldBench.

Table 5 reports 58.71 versus 58.25 for the runner‑up GPT‑5.4, a +0.46‑point lead.

**Figure 7.** Main results on AgentWorldBench: five-dimensional rubric mean per domain. Qwen-AgentWorld-397B-A17B achieves the highest overall average among all evaluated models, with consistent advantages on text-based domains and competitive performance on GUI domains.

Scaling Agents with Simulated RL

How Qwen‑AgentWorld makes simulation scalable, controllable, and useful for agents.

Qwen‑AgentWorld treats environment dynamics as a next‑state prediction task, letting a language model simulate what will happen after an agent’s action.

This section shows why naïve simulation fails—real environments cannot scale or be steered arbitrarily—and introduces the single trick that solves both problems: Simulated Reinforcement Learning (Sim RL), a training loop that uses a learned world model as a controllable simulator.

Instead of collecting costly real‑world interactions, we let the language‑model world model generate synthetic trajectories on‑the‑fly, then train the policy against those trajectories just like ordinary RL.

Seed trajectory: start with $f=0$, agent takes “flip”, reward = 1, next state $f=1$.

Simulate 3 new trajectories by prompting the LWM with “If the flag is 0 and the agent flips, what happens?” – it returns $f=1$ and reward = 1 each time.

Policy receives the simulated (state, action, reward) triples and updates its probability of choosing “flip” when $f=0$.

After a few updates the policy always flips the flag, achieving expected reward = 1.

Even with a trivial state space, the loop demonstrates how a learned world model can replace real interaction while still providing a learning signal.

Collect a modest corpus of real interaction traces (state, action, next‑state, reward).

Fine‑tune the Language World Model on these traces to predict next states given (state, action) pairs.

Generate a large pool of synthetic environments by prompting the LWM with variations of the seed states.

Optionally add controllable instructions (e.g., “inject intermittent API errors”) to the prompts.

Run the RL agent inside the synthetic environments: at each step the agent proposes an action, the LWM returns a predicted next state and reward.

Update the agent’s policy with the simulated experience using any standard RL algorithm (e.g., PPO).

Using this pipeline, Qwen‑AgentWorld achieves three concrete benefits.

First, environment scaling. Table 6 shows that when the simulator is the high‑fidelity Qwen‑AgentWorld‑397B‑A17B, Sim RL lifts Claw‑Eval from 65.4 to 69.7 (+4.3) and QwenClawBench from 47.9 to 55.0 (+7.1) despite the OpenClaw tasks being completely out‑of‑distribution.

Second, controllable simulation. Adding natural‑language control instructions yields large gains: Table 7 reports +12.3 on MCPMark and +16.3 on WideSearch, far exceeding uncontrolled Sim RL (which shows no improvement). Figure 8 visualizes the same trend across three held‑out domains (MCP +5.0, SWE +11.5, Search +11.8).

Third, fictional‑world construction. By synthesizing entirely invented yet internally consistent worlds, Sim RL improves WideSearch F1‑Item by +16.29 and F1‑Row by +10.49 for the 35 B model (Table 8), confirming that agents can learn effective search behavior without ever seeing a real search engine.

Beyond simulation, LWM RL warm‑up (Section 6.2) transfers the next‑state prediction skill to a unified agent. Table 9 shows consistent improvements across seven benchmarks, e.g., WideSearch F1‑Item +12.79, Terminal‑Bench accuracy +6.30, and out‑of‑domain Claw‑Eval +11.3.

**Figure 8.** Cross-domain generalization when training Stage 3 (RL) on Terminal data alone. (a) Terminal (in-domain) improves by +14.2 points over the SFT baseline. (b) All three held-out domains improve without receiving any domain-specific training signal: MCP (+5.0), SWE (+11.5), and Search (+11.8).

**Figure 9.** Controllable Sim RL vs. Real RL (trained against a live search engine) on WideSearch during the first 60 training steps. Both experiments use Qwen3.5-35B-A3B-SFT as the base model.

This bar chart compares prediction accuracy before and after applying LWM RL. The "Before LWM RL" bar shows an accuracy of 69.9%, while the "After LWM RL" bar shows an accuracy of 78.3%, representing an improvement of +8.4%.

**Figure 1.** Multi-Step Causal Reasoning, Information Leakage Prevention, and Epistemic Boundary Awareness. The figure displays three distinct panels illustrating agent reasoning capabilities: 1. **Multi-Step Causal Reasoning**: Shows a terminal command sequence and a six-step causal chain analyzing why a `curl` command failed due to a missing Node.js process. 2. **Information Leakage Prevention**: Demonstrates a search query scenario where the agent identifies that the query topic (ocean acidification) is unrelated to the provided reference text (Nordic health expenditure) and explicitly avoids leaking information. 3. **Epistemic Boundary Awareness**: Illustrates a Python coding scenario where the agent recognizes its inability to compute specific random values beyond a known seed, opting to describe the output format instead of fabricating data.

Mechanism Analysis

We dissect reasoning patterns and RL‑driven fidelity gains to reveal each component’s impact.

This section probes how Qwen‑AgentWorld attains accurate predictions and what changes when RL fine‑tunes the model.

Deliberative self‑correction appears in roughly 10.4 interrupts per turn on average.

Across 129 turns, 1,347 explicit “Wait!” interrupts were observed.

The maximum observed self‑correction count in a single turn reaches 56 interrupts.

A single SWE turn exhibited 56 “Wait!” tokens.

Overall, the model generates 1,347 self‑correction interrupts.

Aggregated over 129 thinking traces across four text‑based domains.

In the Terminal domain, self‑correction occurs at 16.9 interrupts per turn.

Measured across its 32‑turn trajectories.

In the MCP domain, self‑correction occurs at 12.7 interrupts per turn.

The model detects when the agent’s query diverges from the known reference answer and deliberately omits that answer from its generated snippet, preserving the agent’s knowledge gap.

The model builds a chain of cause‑and‑effect steps spanning system components, allowing it to anticipate downstream failures before they occur.

The model computes the exact byte count of 53 for the given file content.

In the terminal example, the thinking trace enumerates characters—including newline bytes—yielding 53 bytes.

Across nine sequential Notion API calls, the model maintains 100% field‑level consistency.

All user identifiers and parent.`page_id` fields match across calls, reproducing the full ≈20‑field schema without omission.

Related Work

We situate Qwen‑AgentWorld among prior world‑model and language‑world‑model research.

World Models have become a cornerstone of general‑intelligence research, where an internal simulator predicts future states conditioned on actions. Early visual and embodied work such as IRIS (Micheli 2022) showed Transformers can act as sample‑efficient simulators for Atari, while DreamerV3 (Hafner 2023) and Dreamer 4 (Hafner 2025) trained agents entirely inside learned dynamics, achieving strong control performance without environment interaction. Recent video‑centric models—Cosmos (Ali 2025), Genie 3 (Ball 2025), Vid2World (Huang 2025a)—scale prediction to high‑resolution scenes, and diffusion‑based GameNGen (Valevski 2025) enables real‑time game simulation. Self‑supervised approaches like V‑JEPA 2 (Assran 2025) learn predictive representations without pixel reconstruction, while UniSim (Yang 2023) unifies diverse visual datasets into a single simulator. Hou et al. 2026 provide a comprehensive survey of robot‑learning world models.

Parallel to visual simulators, a growing body of work investigates Language World Models (LWM), where large language models (LLMs) simulate text‑based environments. Li et al. 2025b introduced a three‑level evaluation showing fine‑tuning improves fidelity, whereas Wang et al. 2024 highlighted the unreliability of off‑the‑shelf LLMs. Theoretically, Richens et al. 2025 proved any sufficiently general agent must contain a world model, extended to stochastic settings by Cifuentes 2026. Empirically, RLVR‑World (Wu 2026a) used RL to train a unified simulator across text, web, and video, outperforming SFT‑only baselines. In the web domain, WebDreamer (Gu 2024), WMA (Chae 2025), WebWorld (Xiao 2026), and WebATLAS (Cheng 2025) pioneered model‑based planning with LLM‑simulated web interactions. Subsequent systems—Shen et al. 2026b, Imagine‑then‑Plan (Liu 2026), DynaWeb (Ding 2026), WebSynthesis (Gao 2025)—combined world‑model guidance with search or MCTS. Works such as Simia (Li 2025c), RWML (Yu 2026), DyMo (Guo 2025b), and Wang et al. 2025a further demonstrated that LLM‑based simulators can scale to diverse digital tasks, while training innovations like VAGEN (Wang 2026a), BehR (Huang 2026), Ren et al. 2026, and SWIRL (Qiu 2026) refined the learning objectives for better alignment.

Beyond web agents, LWM research spans GUI, code, and software‑engineering domains. Code2World (Lehrach 2025) translates game rules into executable Python simulators; Text2World (Hu 2025a) and Agent2World (Hu 2025b) generate symbolic PDDL world models. GUI‑focused efforts include Code2World (Zheng 2026), CUWM (Guan 2026), MobileDreamer (Cao 2026a), and generative visual‑code models (Koh 2026). UISim (Xiang 2025b) builds image‑based UI simulators, while SWE‑World (Sun 2026) removes Docker dependencies for software‑engineering agents. CWM (Copet 2025) mid‑trains on observation‑action trajectories, and Rahmani 2026 analyzes failure modes in code world models. Complementary lines such as WorldLLM (Levy 2025) explore curiosity‑driven theory‑making, and Qian et al. 2026 report that current agents still struggle to invoke world models strategically, underscoring the need for native integration.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers