Scaling Game: Converting Pre-trained Models for Agent Inverse

Kuan Zhang, Yukun Chen, Zhihao Yang, Yue Su, Xiangnan Wu, Zirong Chen, Run Luo, Jinkun Hou, Tao Tan, Yinhe Zheng, Yiming Li

A controlled scaling study reveals that embodied agents gain general control primitives early, but struggle to transfer to distant virtual worlds.

How do model size, data volume, and world diversity interact to determine the performance of agents trained on diverse virtual game worlds?

Continued pre-training (CPT) is the standard way to turn general foundation models into game-playing agents, but it remains unclear whether diverse experience helps or merely causes interference. The authors conduct a controlled grid study across 120 configurations, isolating the effects of model size, data volume, and mixture diversity on performance across three levels of virtual worlds. They find that while scaling reliably improves in-domain performance, generalization to distant, unseen worlds remains a persistent bottleneck that data volume alone cannot solve.

Paper Primer

The authors isolate the three axes of embodied scaling—model capacity, data volume, and mixture diversity—to determine if heterogeneous game data creates a generalist agent or just ruins performance in individual worlds. They fit an empirical scaling law to predict validation loss, revealing that CPT follows a two-phase dynamic: an early "interface phase" where the model learns universal action primitives, followed by a "world phase" where gains are tied to specific training distributions.

Larger models are significantly more data-efficient for embodied tasks.

Larger models achieve comparable or superior performance to smaller models using up to 32× less gameplay data. Large models trained on 50 hours of data outperform small models trained on 1,600 hours.

Real-time interaction exposes a critical performance gap in frontier models.

In unpaused, real-time environments, frontier models suffer a ~90% score reduction due to inference latency, while the authors' models retain 88% of their time-paused performance. The authors' 9B model outperforms the best-tested frontier model by ~30% in real-time settings.

Why does this study matter for current agent development?

Existing research typically scales model size, data, and diversity simultaneously, making it impossible to know which factor drives improvement. This study isolates these variables to provide a reliable reference for building generalist embodied agents.

What is the primary limitation of continued pre-training for these agents?

While CPT improves performance on in-corpus and related worlds, it fails to drive consistent gains in distant, visually or mechanically distinct worlds, suggesting that imitation-based training is insufficient for true generality.

For embodied agents, scaling data diversity is effective for related environments but insufficient for distant ones; developers should prioritize matching training distributions to target domains while accounting for the severe real-time latency penalties of large models.

Motivation and Scaling Dynamics

We frame the scaling bottleneck and two‑phase learning dynamics that limit virtual embodied pre‑training.

One physical world versus diverse virtual worlds illustrates the core gap: agents trained on a single, homogeneous environment may not transfer to the myriad dynamics of virtual games. Existing studies conflate data volume, diversity, and model size, leaving the effect of each factor unresolved, so we conduct a controlled grid study across four model sizes (0.8 B–27 B), six data budgets (50–1 600 h), and five mixture settings (Minecraft‑only to broad multi‑game corpora).

Related work shows early agents were environment‑specific specialists, while recent foundation models adopt CPT to generalize across games. However, no prior work isolates the three axes—model capacity, data volume, and mixture diversity—while measuring held‑out loss and transfer to both related and distant worlds, leaving a critical empirical gap.

The performance of embodied agents plateaus when model size, data amount, and mixture diversity are not balanced, creating a bottleneck that limits further gains.

Training proceeds in an early interface phase where agents acquire basic control primitives, followed by a world phase where those primitives are refined and specialized for each target environment.

**Figure 1. One physical world versus diverse virtual worlds.** Left: Agents learn from varied physical scenes sharing common dynamics, where they can scale and generalize with broader experience (Wang et al., 2026a). Right: Agents play diverse games with varying dynamics. We want to investigate the question that whether continued pre-training scales and generalizes in virtual worlds.

The shift from single‑world to multi‑world agent training is the key lever for unlocking scalable, generalizable embodied intelligence.

Experimental Setup

Describes the data construction, training setup, and evaluation protocol for the experiments.

The model receives a prompt, the current visual observation, and the full interaction history, then produces a textual reasoning trace followed by a structured action command.

Compose $x_t = (c,\,o_t,\,H_t)$.

Sample $r_t$ → “The path ahead is unobstructed; moving forward is safe.”

Condition on $x_t$ and $r_t$ to sample $a_t$ → $\langle\text{action\_start}\rangle\;10\;0\;0;\;K^{(1)}=\text{mouse\_left}\rangle\langle\text{action\_end}\rangle$.

The environment interprets the action as a 10‑pixel rightward mouse move with the left button held for 33 ms.

This walk‑through shows how the model separates high‑level reasoning from the low‑level command format, enabling the same reasoning to drive different concrete actions.

How does the action token differ from the reasoning trace?

The reasoning trace $r_t$ is free‑form text that the model uses internally; it is never executed. The action token $a_t$ follows a strict syntax that the environment parses into concrete device commands.

Training data blends gameplay trajectories with general vision‑language material, and we vary the proportion of Minecraft trajectories to create five distinct mixtures.

From the 27 Minecraft samples, 20 are taken from the VPT corpus and 7 from OpenP2P.

The remaining 73 non‑Minecraft samples are split 50 % vision‑language (≈36) and 50 % embodied (≈37) data.

During training, 5 % of the Minecraft turns (≈1 per 20) are replaced by GPT‑5.6‑Luna‑generated reasoning strings.

This concrete split illustrates how mixture S₃ balances domain‑specific gameplay with broader visual‑language exposure, while the GPT augmentation adds reasoning depth without changing the action distribution.

Why augment a fraction of gameplay turns with GPT‑5.6‑Luna reasoning?

The augmentation injects high‑quality textual reasoning into otherwise silent gameplay steps, ensuring the model learns to produce coherent thought traces even when the original data provide only actions.

Select a model size $N$ from $M=\{0.8\text{ B},\,2\text{ B},\,9\text{ B},\,27\text{ B}\}$.

Choose a data budget $T$ (training hours) from $\{50,\,100,\,200,\,400,\,800,\,1600\}$.

Pick a data‑diversity mixture $S_i$ ($i\in\{1,\dots,5\}$) with the corresponding Minecraft proportion.

Train the model with a causal next‑token prediction loss over interleaved histories, using AdamW and a Warmup–Stable–Decay schedule.

Evaluate on three validation sets (Minecraft‑only, in‑corpus games, unseen games) and record key, mouse, and format losses.

Compute total FLOPs via the “6N D rule” (≈ $2.2\times10^{22}$ for the full grid) and total training tokens (≈ $3.5\times10^{11}$).

**Figure 2.** Adaptation to the keyboard-mouse interface during CPT. Smoothed training losses for key, mouse, and format tokens, and eight-direction mouse prediction accuracy for three model capacities trained on mixtures S1 and S4.

Scaling Laws and Learning Phases

Key scaling results, loss trends, and empirical law across models and data.

Mean Minecraft validation loss drops by about 10.4 % from 50 to 1,600 hours across four model sizes under the Minecraft‑heavy mixtures S1–S3.

Observed across all four backbones (0.8 B–27 B) when training on diverse Minecraft trajectories.

**Figure 3.** Loss scaling across data volume, model size and data diversity. Rows report the smoothed training loss and three-level validation loss. Columns 1–4 denote the 0.8B-27B backbones; colors denote the training mixtures S1–S5; markers denote the six checkpoints from 50 to 1,600 hours. The base model loss is annotated in each validation panel. All plots in a given row shares a common y-axis scale. Column 5 plots the loss predicted by the fitted scaling law (Section 4.4) against the observed loss.

**Figure 4.** Benchmark performance across data budget, model size and data diversity. Rows report MCU, ViZDoom, OmniGameArena, and GameWorld performance. Columns denote the six checkpoints from 50 to 1,600 hours. Colors denote S1-S5 and the base-model reference, while each curve spans the 0.8B-27B backbones. All plots in a given row shares a common y-axis scale.

The quantitative relationship between data, model size, and loss is captured by a high‑fidelity three‑factor scaling law, but its predictive power breaks down for out‑of‑distribution worlds.

Benchmark Evaluation

Details of benchmark setups, scoring protocols, and deployment configurations for the ablations.

This appendix documents how we evaluate all models across the three benchmark families and the serving setups used for the ablation studies.

MCU is a suite of 30 long‑horizon Minecraft tasks that span navigation, survival, GUI interaction, combat, and crafting, providing a comprehensive test of an agent’s world understanding.

**Table 1.** Comparison with baselines on MCU. We report average steps and success rate (ASR, %) across 30 comprehensive tasks. Baselines are quoted from GameWAM. Within each model group, the red and blue indicate the best and second, respectively. “–” denotes an unavailable result.

**Table 1.** Performance comparison of various models across MCU, ViZDoom, OmniGameArena, and GameWorld benchmarks, evaluating both time-paused (TP) and real-time (RT) performance.

**Table 8.** Serving layouts for the GameScaling models. All time-paused benchmark jobs reserve eight H200 GPUs. Real-time vLLM instances use TP = 1 for 0.8B/2B and TP = 4 for 9B/27B so that the reported score includes the latency of the stated deployment.

Latency exceeding 200 ms causes more than a five‑fold drop in real‑time performance.

When model latency surpasses 200 ms, the environment continues advancing while the agent waits, leading to severe degradation (over 5×) as described in the real‑time interaction analysis.

Supplementary Scaling Experiments

Additional ablations explore scaling trends, loss behavior, and inference efficiency.

Figure 6 shows that successive GPT and Qwen releases achieve larger gains than our game‑specific CPT on a fixed backbone, especially on the GameWorld benchmark where CPT adds little.

**Figure 6. Frontier progress on the four benchmarks against release date.** Circles: GPT series; squares: Qwen series; stars: GameScaling S3 models at 1,600 h, all under time-paused interaction.

Figure 7 replots loss versus model size, revealing that larger models reduce Minecraft and in‑corpus losses across all training budgets, while OOD loss behaves differently.

**Figure 7. Loss scaling against model size.** Rows: training loss at the checkpoint, and held-out validation loss on Minecraft, on the game-balanced in-corpus set, and on the out-of-distribution set; columns: 50, 100, 200, 400, 800, and 1,600 gameplay hours; colors: mixtures S1–S5.

Figure 8 visualizes benchmark scores versus gameplay hours, highlighting monotonic gains for 0.8 B and 2 B models, flattening for 9 B/27 B after 100–400 h, and declines for S1 at larger scales on OGA and ViZDoom.

**Figure 8. Benchmark scaling against gameplay hours under time-paused interaction.** Rows: MCU, ViZDoom, OGA, and GameWorld (all in %); columns: 0.8B, 2B, 9B, and 27B backbones; points span 50 to 1,600 gameplay hours; colors: mixtures S1–S5; dashed gray: the untrained Qwen3.5 backbone.

Figure 9 evaluates test‑time scaling on MCU, showing that extending the step cap from 600 to 3 000 increases success rates from ~42 %/~51 % to 59.3 %/69.3 % for 0.8 B and 9 B models, while untrained backbones remain near zero.

**Figure 9. Test-time scaling on MCU.** Cumulative success rate of GameScaling-0.8B-S1 and GameScaling-9B-S1 (1,600 h, solid) and their untrained Qwen3.5 backbones (dashed) on the 30 MCU tasks as a function of the maximum steps allowed, on a logarithmic axis. The standard protocol uses 600 steps.

Table 9 compares inference latency, revealing that frontier models take 4.0–23.7 s per step, while GameScaling models run in 0.13–0.69 s, enabling sub‑200 ms decision making for the 0.8 B and 2 B agents.

Comparison with Scaling Literature

We compare GameScaling’s training scale and scope to prior language‑model, GUI, and robotics studies.

**Table 15.** Training scale of language-model studies and GameScaling. Budgets cover the experiments listed, excluding original backbone pre-training for CPT. Related-work budgets are estimates; GameScaling uses the reported totals in Section 3.

**Table 16. Scope of GUI, robotics, and virtual-world scaling studies.** GameScaling cost covers the full grid; other costs are per run. †: time estimate at unchanged recipe and throughput. Data use the original units; GameScaling hours include warmup, stable training, and decay.

Discussion and Conclusion

Training agents first adapts to the interface, then scales to world‑specific tasks.

Early training yields cross‑world improvements by learning universal controls and action patterns, but later gains depend heavily on the training distribution. By 1,600 hours, mixed‑world training achieves a mean OGA score about 2.8× higher than Minecraft‑only training across model sizes. These benefits do not consistently transfer to distant virtual worlds, so scaling model capacity must be paired with strategic corpus curation.

GameScaling models are competitive with both Minecraft specialists and frontier models, yet their performance hinges on whether the environment pauses during inference. In real‑time (unpaused) evaluation, GameScaling‑9B‑S3 scores 25.3 on average, surpassing the best frontier model’s 19.2. Delayed actions become obsolete in dynamic worlds, making response latency as critical as gameplay capability.

CPT provides a strong foundation for generalist game agents, but gains in related environments do not always translate to distant worlds; the 27B‑S3 checkpoint improves time‑paused OGA by 16.0 points yet drops 4.0 points on GameWorld. This gap likely stems from CPT’s reliance on behavior cloning without direct environmental feedback. Future work should explore scaling world diversity, integrating online feedback, or combining both to enhance cross‑world transfer.

Our gameplay training data come from the publicly released VPT, D2E, and OpenP2P datasets, which were cleaned by their providers to protect player privacy. We sample existing trajectories and add retrospective reasoning annotations without collecting new recordings or personal data, and we identify no additional ethical concerns.

Future generalist agents should pair larger models with diverse, strategically curated training worlds and incorporate online feedback to improve cross‑world transfer.

Dataset Details

Details of how the training mixtures, diversity metrics, and window construction are built.

Training tokens are split 85 % to gameplay trajectories and 15 % to a fixed vision‑language anchor. The gameplay pool merges VPT, D2E, and OpenP2P, while the anchor aggregates LLaVA‑OneVision, Zebra‑CoT, GameQA, Embodied‑Reasoner, and Cambrian‑S data. The anchor never contains game content, ensuring that changes in gameplay diversity are isolated.

Diversity is varied by shifting the Minecraft share against a fixed set of 79 other worlds. Across mixtures S2–S5 the non‑Minecraft pool remains unchanged, so the only effect is the redistribution of gameplay experience. Running the full 120‑condition grid required roughly 45 000 H200 GPU‑hours.

Each recording is normalized, sampled at 5 Hz, and sliced into 20‑turn windows with a 10‑turn stride. A window supervises 4.0 seconds of actions while spanning 3.8 seconds between its first and last observations, and actions are encoded into a unified six‑substep format. Idle‑dominated windows are downsampled to avoid over‑representing non‑interactive periods.

Mixture diversity is quantified by the effective number of worlds $D = \exp\!\bigl(-\sum_w p_w \log p_w\bigr)$, the Hill number of order one. The inverse Simpson index $1/\sum_w p_w^2$ is also reported to highlight dominant worlds. Both metrics increase as the Minecraft share is reduced, reflecting a more even distribution of experience.

**Table 3.** Mixture diversity of S1–S5. Shares are computed over gameplay segments after cross-source aliases are merged into canonical world labels. $D$ is the effective number of worlds (Hill number of order one); the inverse Simpson index emphasizes dominant worlds.

**Table.** List of games used in the recording pool, including their category, spatial dimension, number of sessions, and number of clips.

**Figure 5. A training window with sparse reasoning.** Five consecutive 200 ms turns from an Apex Legends recording in S4. Each turn begins with the current frame. Only $o_t$ and $o_{t+4}$ carry a GPT-5.6-Luna rationale; the other three turns emit an empty `<think>` span and act directly. Every action packet contains a mouse displacement ($\Delta x, \Delta y$) and the keys held during six 33 ms sub-steps; key icons show the distinct keys in each packet. Image tokens receive no loss, while thought and action tokens are trained autoregressively. Rationales are abridged excerpts of the stored annotations.

Validation Environments

Describes the three held‑out validation sets and the OOD set composition.

We evaluate action loss on three fixed held‑out sets: a Minecraft set from the VPT test split, an in‑corpus set that mixes game‑balanced recordings from D2E and OpenP2P with a size‑matched Minecraft subset, and an out‑of‑distribution (OOD) set containing only titles absent from any S1–S5 mixture. All sets share the same observation and action representation as training, and overlapping windows are kept separate to prevent leakage.

**Table 1.** Comparison of model parameters, latency, and output tokens per step.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers