Agentic Game Development as a Verifiable Trajectory Data Engine for Scaling World Models

Pengfei Zhou, Hexin Wang, Zhengfeiyang Zhang, Yixing Ma, Zhenglin Wan, Kaipeng Zhang, Wangbo Zhao, Yang You

Game development provides a recursive, verifiable data engine for training spatial world models.

Can we improve world model scaling by using game engine feedback loops as a verifiable data source instead of relying on noisy human or model-based reward proxies?

Spatial world models currently rely on fuzzy proxies like CLIP scores, which are noisy and biased, preventing the kind of iterative reinforcement learning that made code agents successful. The authors propose Agentic World Model (AWoMo), which treats game development as a recursive data engine where game engines provide dense structural verification and human developers provide final acceptance. On UnitySceneBench, this Reinforcement Learning with Human-Engine Verification (RLHEV) approach achieves the highest asset classification performance, significantly outperforming fuzzy-proxy baselines.

Paper Primer

The core move is to treat 3D world construction as an executable program rather than a static media artifact. AWoMo uses a "propose, render, verify, repair, and review" loop: the model generates scene edits, the engine automatically checks for structural failures like collision or navmesh errors, and the human reviewer provides the final acceptance signal.

This framework, the Unified World-Development Protocol (UWDP), stores these interactions as structured trajectories—linking design intent, engine diagnostics, and human critiques—which serve as high-fidelity training data for the next generation of models.

Full human-engine verification (RLHEV) maximizes asset classification accuracy.

Comparison against fuzzy proxies, SFT, and engine-only RLVR on the UnitySceneBench dataset. Outperforms the strongest non-full baseline by +0.098 in primary score and +0.120 in accuracy.

Pretraining on source-engine development traces improves out-of-distribution (OOD) generalization.

Cross-engine transfer experiments from Unity to Unreal and Godot. Unity-to-Unreal performance increased from 0.25 to 0.35, and Unity-to-Godot from 0.15 to 0.35 relative to scratch training.

Why is game development a better data source than simply scraping more video?

Video scraping lacks a grounded reward signal, forcing models to rely on fuzzy proxies. Game engines provide executable, deterministic feedback on structural correctness (e.g., physics, collision), which allows for reliable reinforcement learning post-training.

Does this approach require manual annotation for every single training instance?

No. The engine provides dense, automated structural rewards, while human review is reserved for final acceptance. This makes human feedback more efficient by focusing it on high-level utility rather than low-level structural debugging.

Spatial intelligence research should shift from scaling passive video data to instrumenting executable development workflows that generate their own verifiable training signals.

Introduction: The Scaling Bottleneck

We expose why scaling world models with more video data is inefficient and propose a game‑engine verification loop.

Current practice scales world models by training on ever larger crawled video datasets, assuming that more data and compute will close the performance gap.

Training on more video alone is inefficient because it provides no grounded, verifiable reward signal; without such signals, post‑training reinforcement learning cannot reliably improve spatial world models.

Code agents demonstrate why this recipe fails: executable code yields dense compiler and test feedback, while spatial generation still depends on fuzzy proxies like CLIP similarity, which are noisy and gameable. Game development, by contrast, records human‑engine verification loops that turn scene edits into executable specifications, providing both dense engine checks and sparse human acceptance signals.

**Figure 1.** From subjective reward proxies to a human-engine verification loop. Conventional spatial-data pipelines convert sparse human or model annotations into noisy final-output rewards. A game-development loop can instead pair human intent and feedback with engine execution, development trajectories, rendered evidence, and localized verifier failures, allowing future builders to improve from both grounded checks and developer judgment based on traces rather than final artifacts alone.

Scaling world models by merely crawling more video is a dead‑end; without a cheap, reliable reward engine, larger datasets cannot replace the need for verifiable supervision.

The Thesis of Verification

We introduce Human‑Engine Verification, a cheap verifier plus human judgment to train world models.

Automatic verification is cheap, but full task correctness still requires human intent. Without a cheap, reliable verifier, training world models on raw video wastes compute on ungrounded rewards.

Instead of asking a model to guess a single scalar reward, we split the evaluation: the game engine instantly checks structural constraints, while a human reviewer gives the final accept/reject decision.

Engine evaluates y₁: C₁(y₁)=0 (no collisions), C₂(y₁)=1 (one script warning).

Engine evaluates y₂: C₁(y₂)=2 (two collisions), C₂(y₂)=0 (no script warnings).

Human reviewer inspects y₁ and y₂ and accepts y₁ ($U_H$=1) but rejects y₂ ($U_H$=0) based on overall feel.

Combined protocol stores (x, y₁, C₁=0, C₂=1, $U_H$=1) and (x, y₂, C₁=2, C₂=0, $U_H$=0).

Engine checks filter out obvious failures cheaply; the human only needs to evaluate the few remaining plausible candidates.

How does RLHEV differ from standard reinforcement learning that learns a reward model from human feedback?

Standard RL learns a scalar reward function from human labels, which still requires a human judgment for every sampled output. RLHEV keeps the human judgment only for the final acceptance decision; all intermediate structural checks are performed automatically by the engine, dramatically reducing the number of human evaluations needed.

The model proposes a candidate output y for a given input x.

The game engine computes all diagnostics $C_i$(x, y) and evaluates hard gates $G_j$(x, y).

If any hard gate fails, the candidate is discarded without human involvement.

For candidates that pass, the engine‑derived penalties $\lambda_{i}$ $\phi_{i}$($C_i$) are summed and subtracted from the human utility $U_H$.

A human reviewer inspects the remaining candidates and assigns $U_H$ (accept/reject).

The resulting reward r = $U_H$ − ∑ $\lambda_{i}$ $\phi_{i}$ is fed back to update the model parameters.

The updated model generates stronger candidates in the next iteration.

The thesis is that progress is limited not by data or compute but by the availability of a scalable, cheap feedback channel. Human‑Engine Verification provides exactly that channel for spatial creation tasks.

Why Current Spatial Data Fails

Current spatial data lacks reliable verifiers, forcing reliance on fuzzy reward proxies.

Progress in spatial world models stalls because there is no efficient verifier for physical and geometric correctness. Without such a verifier, training relies on indirect, noisy signals.

Video generation models produce realistic clips, yet no fast, reliable check tells whether the physics or perspective are coherent. Evaluation is limited to Fréchet Video Distance and human raters—both fuzzy proxies that can be costly and gameable.

Text‑to‑3D pipelines inherit the same problem: curated 3D asset collections are tiny (≈ $10^{7}$ objects) and high‑quality assets still require expensive manual verification. Consequently, 3D generation often collapses to distilling 2D priors, inheriting fuzzy signals from an unverifiable domain.

Learned world simulators need dense ground‑truth depth, geometry, and dynamics, which are prohibitively expensive to annotate at scale. This annotation wall becomes the dominant constraint on spatial intelligence.

When a model is asked to judge its own output, the reward is derived from an imperfect proxy—e.g., a CLIP score or a language model—rather than an objective verification of the underlying world state.

How does a fuzzy reward proxy differ from a deterministic engine verifier?

A fuzzy proxy predicts a scalar from learned correlations and can be fooled by inputs that trigger high scores without satisfying physical constraints. In contrast, an engine verifier runs the generated scene through a physics engine or collision system, producing a binary pass/fail that directly reflects the world’s rules.

**Figure 2.** Fuzzy reward in unverifiable spatial generation versus human-engine feedback in game development. The top pipeline illustrates how model-as-a-judge and human aesthetic variance can produce ambiguous rewards and performance bottleneck. The bottom pipeline turns assets into executable worlds: game engine and internal test suites expose local failures, rendered evidence supports review, and developer supplies the final signal for acceptance.

The ambiguity of model‑as‑a‑judge rewards makes scaling spatial intelligence impossible without reliable engine verification.

Game Development as Verification

We introduce Human‑Engine Verification to turn game engine checks into dense training signals.

Game development already embeds a practical verifier: the engine runs the scene and the developer inspects the result. This loop supplies dense, grounded feedback that avoids the fuzzy reward proxies of pure video‑based training.

How does RLHEV differ from standard reinforcement learning that uses sparse, hand‑crafted rewards?

Standard RL relies on a single scalar that often reflects a proxy (e.g., image similarity), which can be noisy and easy to game. RLHEV replaces that proxy with concrete engine checks that are deterministic, low‑variance, and directly tied to the physical and logical constraints of the game world.

Engine computes the axis‑aligned bounding boxes of both cubes.

Collision query detects overlap because the new $y$ coordinate of $obj_2$ becomes $0.5$, intersecting $obj_1$.

Engine returns a “collision” flag as the RLHEV reward signal.

Model receives the negative feedback and proposes an alternative edit (e.g., moving $obj_2$ to $(0,0,2)$).

Collision query now reports no overlap, yielding a positive RLHEV signal.

Engine checks provide an immediate, binary signal that tells the model exactly which geometric constraint was violated, something a pixel‑level loss cannot reveal.

Execution loop that drives RLHEV data collection.

UWDP turns ordinary game‑development actions into a typed trace of intent, edit, engine check, and human review, making every edit a supervised training example.

In what way does UWDP differ from typical dataset collection pipelines that simply dump screenshots and logs?

Typical pipelines treat raw assets as unlabeled data, losing the causal link between an edit and its verification outcome. UWDP preserves that link by recording the full $state\!\to\!action\!\to\!check\!\to\!review$ chain, turning each development iteration into a supervised example rather than an isolated snapshot.

The engine spawns the bridge and runs a static‑load check, producing $g_t$ = “no interpenetration”.

It then runs a physics stability simulation, yielding $g_t$ = “stable under gravity”.

Rendered image $v_t$ shows the bridge from a top‑down view.

The reviewer inspects $v_t$ and decides $h_t$ = “accept” because the bridge matches the brief and is aesthetically appropriate.

The full tuple $u_t = (b, o_t, s_t, a_t, g_t, v_t, h_t, \rho_t)$ is stored for training the next‑edit predictor.

By keeping the engine checks ($g_t$) alongside the human decision ($h_t$), the trace tells the model exactly which physical constraints were satisfied and which human‑only criteria still mattered.

The situation mirrors compiling code: the engine is the compiler, automated playtests are the test suite, and the developer’s final approval is the code review. Just as passing tests does not guarantee user‑facing quality, engine checks alone are insufficient—human review anchors the loop.

Experimental Results

Experiments demonstrate RLHEV’s superiority on UnitySceneBench across understanding, generation, and transfer.

Full RLHEV achieves a 0.681 primary score on UnitySceneBench, beating the next best method by +0.098.

Figure 4 shows Full RLHEV primary score 0.681 versus Engine‑based RLVR 0.583.

A benchmark that measures how well a model can both classify existing Unity assets and generate new ones from multimodal prompts.

**Figure 4.** Unity asset classification evaluation on UnitySceneBench. Bars represent the primary score, and markers denote accuracy and AUC. This figure reports the best performance for each method across eight random seeds. Full RLHEV obtains the highest best-of-eight primary score (0.681), with 0.665 accuracy, 0.665 balanced accuracy, 0.733 F1, and 0.690 AUC.

**Figure 5.** Scaling with different numbers of training samples and tested on *UnitySceneBench*. (a) reports the primary score for Fuzzy Proxies Baseline, SFT Baseline, Offline RLHF, Engine-based RLVR, and Full RLHEV on the test set, as mean±std over eight seeds. (b) Unity assets generation uses the same training budgets and reports mean±std over eight seeds per method and budget.

**Figure 6.** Generalization and embodied diagnostic evaluations. (a) Generalization under distribution and engine shifts. Bars report the MLLM-as-a-judge score in [0, 1] for training from scratch and target-adapted from a pretrained checkpoint. Pretrained on source data and adapted on the target distribution achieve significant gain; the cross-engine settings show smaller but measurable positive signal. (b) Embodied diagnostic evaluation on R2R, Gymnasium MuJoCo, and D4RL Gym-MuJoCo. Values are relative improvements over the original baseline after normalizing scores (positive is better for all benchmarks). Naive aug. denotes standard data augmentation, whereas AWoMo-augmented denotes the policy performance trained with AWoMo-generated data.

The provided image contains a table comparing various methods across different numerical settings (40, 80, 160, 320, 640, 720).

**Table 3.** Unity assets generation quality for Figure 5. Values are mean quality over eight seeds per method for each training budget.

The table presents performance metrics across different transfer categories, including Distribution and Cross-engine scenarios. It evaluates models based on Assert, Proxy, Engine acc., and Loss metrics under various conditions such as zero-shot, target, and scratch.

**Table 5.** Target-adapted transfer results for each target engine training split.

RLHEV consistently outperforms proxy‑based baselines on UnitySceneBench.

Methodological Details

We detail the Agentic World Model and protocol traces that enable verifiable training.

Crawled video provides only weak supervision, making it hard for world models to learn actionable dynamics.

AWoMo is a world‑model core embedded in an agent loop that proposes edits, gets engine verification, and records the whole process as a trace.

The engine checks for collisions; none are found, so it returns a pass flag.

The reviewer accepts the edit, finalizing the scene and storing the trace entry.

The trace records: intent “move cube”, action “translate to (0,1,0)”, verification “no collision”, review “accept”.

Even this trivial edit illustrates how the trace links the proposed action to its verification outcome, enabling supervised learning of edit‑success prediction.

How does AWoMo differ from a conventional world‑model that simply predicts next frames?

AWoMo does not output raw predictions; instead it proposes concrete scene edits, receives engine‑generated feasibility checks, and records the full edit‑verification‑review cycle as a trace, providing dense, verifiable supervision rather than ambiguous pixel‑level loss.

A protocol trace is an ordered list of all intermediate states, checks, and repairs that led from an initial intent to the final scene.

Step 0: Scene v0 is generated from the task brief.

Step 1: Engine detects a collision failure; the trace records the failing object.

Step 2: The agent issues a repair edit moving the object to resolve the collision.

Step 3: Engine verifies the navmesh is now reachable; the trace logs a pass.

Step 4: A bounded playtest runs; the engine rewards the scene, and the reviewer accepts it.

The trace captures not only the final successful scene but also the intermediate failure‑repair cycle that produced it, providing supervision for both diagnosis and generation.

What information does a protocol trace capture that a final‑snapshot approach completely misses?

A final snapshot only records the terminal scene state. A protocol trace additionally records the original intent, each intermediate edit, the specific engine checks that failed, the exact repair actions taken, and the reviewer’s acceptance decision, giving a full causal history.

Propose: the agent generates an edit $a_t$ given intent $b$.

Render: the edit is applied to produce an intermediate scene $s_t$.

Verify: the engine returns check outputs $g_t$ and rendered evidence $v_t$.

Repair: if any check fails, the agent issues a repair action to modify $s_t$.

Review: a human reviewer accepts or rejects the scene, producing decision $h_t$.

Loop until the reviewer accepts, rejects, or requests further revision.

If the implemented reward deviates from the intended reward by at most $\epsilon$, the value gap of an $\eta$‑optimal policy under the implemented reward is bounded by $2\epsilon+\eta$ under the intended reward.

**Figure 7.** Protocol traces preserve the process data that is absent from final snapshots. A final scene records only the terminal state, whereas a trace records intent, intermediate scene states, failed checks, repair edits, validation outcomes, and engine rewards. This object-linked sequence provides a training objective for asset diagnosis and repair.

**Figure 8.** Source-side trace transfer under an 8-label target budget. Values are held-out Spearman correlations over 8 seeds and two target engines. Protocol-trace features provide substantially stronger target-engine ranking signal than coarse final-snapshot features; increasing the number of Unity source instances gives a smaller additional gain.

**Table 1.** Example fields in one UWDP instance. The values illustrate how stable object IDs connect intent, scene state, engine checks, repairs, rendered evidence, and review.

Limitations and Future Work

We discuss the main limitations of our approach and outline future research directions.

Games are not reality, so the sim‑to‑real gap remains the central test of our method. Current experiments lack real scans, robots, or a full real‑to‑sim‑to‑real loop, so they cannot be taken as evidence that game‑engine rewards already transfer to the physical world.

The next logical test is to adapt the approach to a real‑world environment, fine‑tune with a small amount of real data, and measure which executable checks survive the domain shift.

Video generation is advancing quickly, but its progress is limited by imitation and compute constraints, relying on fuzzy aggregates rather than a self‑improvement loop. Typical failures involve physical inconsistencies and long‑horizon coherence.

Engine rewards can be gamed: a loose collision, navigation, or budget check may be optimized around if used in isolation. This does not invalidate verifiers; instead it motivates ensembles of checks, randomized probes, held‑out validations, and human review.

Cheaper real‑world 3D data does not equate to cheaper verification. Scans capture appearance without annotation and cannot confirm the correctness of synthesized outputs; the engine provides the missing automatic check.

Source‑engine traces risk overfitting to a single engine’s semantics. Cross‑engine studies (Unity‑to‑Unreal, Unity‑to‑Godot) show measurable positive transfer after target adaptation, supporting the use of source traces as initialization data and calibration steps.

Our experiments give bounded evidence: Human‑engine feedback improves UnitySceneBench tasks, and target adaptation boosts generalization and embodied diagnostics, but they do not yet prove full game quality, human‑subject validity, or closed‑loop deployment.

The long‑term goal is a recursive game‑building loop where an agentic world model generates executable games, playtest agents expose failures, and those failures become training signals for the next model. Trustworthiness hinges on executable specifications, robust playtest agents, and human review of design intent.

A practical path starts with bounded playable levels, adds playtest agents and failure localization, then scales to multi‑scene games with new engines, agents, and human reviewers.

Broader impacts are indirect: grounding spatial generation in verifiable engine rewards could lower training costs for embodied AI and make pipelines easier to audit, but also enable more convincing synthetic media, raising dual‑use concerns.

Engine‑grounded pipelines require careful data‑governance: opt‑in collection, project‑level filtering, redaction of proprietary content, and controlled training paths for sensitive projects.

We reinterpret recursive self‑improvement as a hybrid loop where AI systems participate in human‑centric production pipelines, learning from executable traces generated during real development.

Whether such socially grounded feedback loops scale beyond game development to broader engineering and scientific domains remains an open research question.

In the long term, cheaper software and content creation frees human creators to build virtual worlds, generating a growing economy of verified construction traces that in turn accelerate future world‑model scaling.

Discussion and Conclusion

Human‑engine verification proves a practical feedback source for scaling world models.

The central premise—that video‑only training lacks grounded reward signals—was revisited here. We now examine how human‑engine verification supplies the missing feedback.

On UnitySceneBench, the best‑of‑eight full‑budget configuration reaches a primary score of 0.681.

Measured with human acceptance combined with engine checks; the same run reports 0.665 accuracy.

**Table 6.** Final embodied generalization results. Values are mean±std where available. $\Delta$ is the direction-normalized relative improvement of AWoMo-augmented training over the original baseline.

Human‑engine verification is a practical feedback source for scaling.

Experiment and Benchmark Details

Supplementary details on baselines, evaluation protocols, and generalization experiments.

Section B.1 lists all baselines and benchmarks used in the experiments. The zero‑shot visual baseline applies CLIP similarity thresholds without any task‑specific learning, while the MLLM‑as‑a‑judge harness (Qwen3.6‑35B‑A3B) follows the same accept/reject rubric as the human‑review channel.

UnitySceneBench instances are built from the developer workflow (see Appendix A.1) at a reduced scale: prompts and assets are turned into candidate edits, imported into Unity, rendered, engine‑checked, and finally labeled by a reviewer.

The evaluation suite also includes R2R, Gymnasium MuJoCo, D4RL Gym‑MuJoCo, and SAME/DUET‑style VLN trajectory metrics, providing a broad set of embodied diagnostics across three game engines.

Table 2 enumerates the concrete method definitions used on UnitySceneBench, ranging from the zero‑shot CLIP proxy to the full RLHEV model that fuses human‑review and engine rewards.

Section B.2 details the full‑RLHEV validation. The task is binary asset classification: the model predicts acceptance of edited Unity assets, using prompts, asset features, reference images, and optional CLIP embeddings, while the target label and engine verdict are withheld.

Rewards are weighted 0.65 for human feedback and 0.35 for engine verification; this mixture is the only RL component beyond the supervised baseline.

Evaluation follows a best‑of‑eight protocol (Figure 4): each method is trained on the 720 training instances with eight random seeds, and the highest test performance across seeds is reported.

Figure 5 instead shows the mean ± standard‑deviation across seeds for each training budget, illustrating seed‑level stability.

The generation side runs the full‑RLHEV model on the same 720 instances, producing 640 fresh Unity plans that all exit with code 0 and contain no missing artifacts; Table 3 breaks down quality across budgets.

In this best‑of‑eight setting, Full RLHEV achieves a +0.098 primary‑score gain and a +0.120 accuracy (balanced‑accuracy) improvement over the strongest non‑full baseline.

Section B.3 describes the generalization experiments. Two transfer categories are examined: distribution transfer (Unity → held‑out Unity) and cross‑engine transfer (Unity → Unreal or Godot).

Training conditions compare target‑only scratch (training solely on the target domain) with target‑adapted transfer (pre‑training on the source domain then fine‑tuning on the target).

The primary metric is a normalized MLLM‑as‑a‑judge score in [0, 1]; higher values indicate better alignment with the accept/reject rubric.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers