MAI-Gym: An Open All-In-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments

Panrong Tong, Quyu Kong, Pengxiang Li, Tianyu Xia, Hanzhang Zhou, Lei Wang, Chenglin Cai, Long Li, Long Chen, Jianan Zhang, Zhengwen Zeng, Xu Zhang, Gongjie Zhang, Yue Wang, Steven Hoi

MAI-Gym unifies GUI agent training and evaluation across mobile, computer, and web with a scalable, shared rollout architecture.

How can we build a unified, scalable infrastructure to train and evaluate GUI agents across mobile and computer environments using online reinforcement learning?

GUI agent research is fragmented: training and evaluation often use different harnesses, causing performance drift, while scaling to thousands of interactive environments remains prohibitively expensive and unstable. MAI-Gym provides a unified platform that shares a single rollout path across evaluation, data collection, and online reinforcement learning (RL), using automated synthesis to generate resettable mocked apps and cross-app tasks. This infrastructure enables training on 20,000 parallel instances, producing a 9B-parameter agent that achieves state-of-the-art performance among open models of its size.

Paper Primer

The core mechanism is a shared rollout harness that ensures the agent's observation processing, context construction, and action execution are identical during both training and evaluation. By decoupling the environment manager from the agent harness, the system allows researchers to scale to 20,000 parallel instances while using window-wise history packing to minimize redundant computation in multi-turn GUI trajectories.

MAI-Gym-9B achieves state-of-the-art performance among open GUI agents under 10B parameters.

The model scored 78.4% on MobileWorld and 76.2% on OSWorld-Verified, outperforming similarly sized baselines like UI-Venus-2-9B and SCALECUA-Qwen3.5-9B. It improved success rates on MobileWorld by 60.4 percentage points over the base Qwen3.5-9B model.

Training in synthesized real-world clones transfers effectively to physical devices.

RL training in a mocked train-ticketing app improved task success on the real-world counterpart (on a physical phone) from 28.8% to 48.1%. A 19.3 percentage point gain in real-world task success.

Why is a shared rollout path necessary for GUI agents?

When evaluation and training use separate agent implementations, they inevitably drift in how they preprocess screenshots or parse actions, making it impossible to know if a performance gain comes from the model or the harness.

How does MAI-Gym handle the high cost of GUI environment resets?

It uses container-based virtualization (e.g., ReDroid for Android) to restore golden snapshots of the system state, achieving median reset times as low as 6 seconds for web and 37 seconds for desktop environments.

MAI-Gym shifts GUI agent research from isolated, benchmark-specific training to a unified, scalable RL workflow, enabling researchers to train on synthesized tasks that directly improve real-world application performance.

Introduction and Motivation

We expose the bottleneck of GUI‑agent training and introduce MAI‑Gym as a unified platform.

Online reinforcement learning (RL) has propelled advances in math, coding, and tool use, yet scaling these gains to GUI agents stalls because interactive environments are expensive to run and realistic training tasks are few.

MAI‑Gym addresses this gap by providing a single platform that unifies GUI‑agent evaluation, data collection, and online RL training across mobile, computer, and web environments; a shared rollout pipeline guarantees consistent agent behavior, it supports ten established benchmarks, and it can run up to 20,000 parallel environment instances.

**Figure 1.** **Overview of MAI-Gym.** Left: online GUI environments. Center: core capabilities. Bottom: the evaluation–diagnosis–synthesis–retraining loop. Right: performance comparison of MAI-Gym-9B with models of different sizes on MobileWorld and OSWorld-Verified.

The cost of interactive environments is the primary bottleneck for scaling GUI agents.

Introduction

We expose why scaling GUI agents via online RL is fundamentally hindered by environment cost and task diversity.

Training GUI agents via online RL faces two fundamental obstacles. First, interactive GUI environments are expensive to scale: each episode must run on an isolated device because the agent’s actions modify the entire screen and underlying file system, preventing concurrent reuse. Second, realistic GUI learning demands diverse, coherent tasks that span multiple applications and maintain a resettable state, something current task synthesis pipelines struggle to provide.

A GUI Agent is a model that perceives a graphical interface (screenshots, UI elements) and issues low‑level actions (taps, clicks, keystrokes) to achieve a goal.

Online RL continuously collects experience from the environment while the policy updates, rather than separating data collection and training into distinct phases.

**Table 1.** Comparison of MAI-Gym with existing agent benchmarks.

System Foundations

Defines executable tasks, the agent harness, and online‑RL formulation for GUI agents.

Training GUI agents is hampered by costly, fragmented interactive environments that must be reset and isolated for every rollout.

An Environment Contract bundles everything a task needs: the software system, its initial state, the natural‑language goal, and the verifier that judges success.

Instantiate the environment: launch the file‑manager with $s_0$ (empty folder).

Policy receives $c_0$ (instruction “create foo.txt”) and outputs action $y_0$ = “type foo.txt ↵”.

Harness executes the GUI click to open the save dialog and the CLI command “touch foo.txt”.

Episode ends; $V$ scans the folder and returns $b=1$ because “foo.txt” is present.

The contract makes the task fully reproducible: swapping $s_0$ for a folder that already contains “foo.txt” would change the success outcome even though $g$ is unchanged.

**Table 2.** Agent observations, context, and actions.

How does the Environment Contract differ from a standard RL task definition?

Standard RL tasks usually specify only an MDP (states, actions, reward). The Environment Contract adds an explicit initial snapshot $s_0$ and a separate verifier $V$, making the task fully deterministic and verifiable across heterogeneous GUI/CLI environments.

Architecture and Task Synthesis

MAI-Gym unifies GUI‑agent training, evaluation, and data collection via a single harness and manager.

Online RL for GUI agents typically requires a separate stack for each platform, which leads to hidden drift between the agent used for training and the one used for evaluation. MAI‑Gym solves this by collapsing all workloads onto a single harness and rollout path.

All three workloads—evaluation, data collection, and RL training—share the exact same agent instance, so any change to the harness instantly propagates to every use case.

Worker A runs step 1: harness queries $π( s_0 )$, receives action $a_1$, sends it to the environment, receives $s_1$.

Worker B runs step 1 on its own instance: the same query $π( s_0 )$ yields the identical action $a_1$ because the policy parameters $θ$ are shared.

Both workers write their trajectories $(s_0,a_1,s_1)$ to the same buffer; the buffer now contains two identical entries.

The RL trainer samples the buffer, sees duplicate trajectories, and updates $θ$ once; the updated $θ$ is instantly visible to both workers on their next query.

Because the policy is shared, any improvement or bug affects every workload simultaneously, guaranteeing perfect alignment between training and evaluation.

How does this differ from the common practice of running separate training, evaluation, and data‑collection agents?

In the usual setup each workload maintains its own copy of the policy, so a change to preprocessing or action parsing in one copy never reaches the others. MAI‑Gym’s single‑harness design forces all three workloads to use the exact same code path, eliminating hidden inconsistencies.

The manager acts as a universal adapter that translates five high‑level operations into the native API of any GUI environment, shielding the agent from platform‑specific quirks.

Why does the manager expose only five operations instead of exposing the full native API of each environment?

The five operations capture exactly what the RL loop needs—instance provisioning, observation, task reset, step execution, and reward evaluation. Anything beyond that is either irrelevant to training or handled internally by the manager, keeping the interface minimal and portable.

Obtain a GUI instance from the EnvClient via the Environment Manager.

Run the single harness: query the inference engine, dispatch the action, and receive the next observation.

After each step, invoke the verifier (code or VLM) to compute a reward.

Write the (state, action, reward, next‑state) tuple to the shared trajectory buffer.

When the episode ends, signal the scheduler to schedule the next task attempt.

The pipeline automatically creates new RL tasks and corresponding code verifiers that target the current model’s weaknesses, turning a discovered failure into a concrete training signal.

World synthesis: folder A = {doc1.txt, Report.pdf}, folder B = {image.png, notes.docx}, folder C = {music.mp3, video.mp4}.

Task generation: RL TaskSpec = (environment = MobileWorld, start state = folder A, goal = Report.pdf).

Verifier synthesis runs the golden script, reaches the goal state, and records the final file list.

The model’s rollout is executed; the verifier compares the model’s final file list to the golden list and returns a reward of 1 if Report.pdf is present, 0 otherwise.

The synthesis pipeline turns a single failure (“model missed Report.pdf”) into a repeatable training example with an automatically computed reward.

How is this synthesis different from manually writing new tasks and verifiers?

Manual authoring requires a human to design the world state, craft the instruction, and implement a verifier for each edge case. The automated pipeline generates all three components programmatically, scales to thousands of tasks, and guarantees that each verifier matches the exact goal state produced by the golden implementation.

**Figure 1.** System architecture diagram illustrating the components of an online reinforcement learning training and evaluation framework, including environment management, GUI instance handling, and the training backend.

Scalable Rollout Infrastructure

Scalable rollout infrastructure enables fast, reliable, and efficient GUI agent training.

GUI rollouts cost far more than text rollouts because each attempt needs a fresh device state, thousands of instances run concurrently, and each step triggers many multimodal inference calls.

The trick is to separate resetting, provisioning, and inference so that each can be parallelized and reused, turning a per‑step cost into an amortized one.

Turn 1–4 form Window 1; the KV cache stores the prompt prefix for turns 1‑4.

Turn 5 starts Window 2, overlapping turn 4; only the new screenshot for turn 5 is appended, and the cached prefix from Window 1 is reused.

Turn 6 continues Window 2; the cache now covers turns 2‑5, so only turn 6’s new content is processed.

Turn 7 begins Window 3, overlapping turn 6; the prefix changes only once every $K\!-\!O=3$ steps.

This overlapping scheme reduces cache misses from every step to once every three steps, cutting inference latency by roughly a factor of three for long trajectories.

How does the $K$‑$O$ windowing differ from the naïve “keep the last $k$ screenshots” policy?

The naïve policy discards the oldest screenshot at each step, which changes the prompt’s start token every turn and forces the engine to rebuild the KV cache each time. In contrast, $K$‑$O$ windowing only replaces the prefix when a new window begins, so the cache is reused for $K\!-\!O$ consecutive turns, dramatically lowering the number of full recomputations.

Start instance creation asynchronously as soon as a request arrives.

Place the new instance into a warm pool once its initialization and registration complete.

Route incoming rollout tasks to the first available instance in the pool.

Transfer screenshots via object storage, keeping control‑plane messages lightweight.

Monitor each session with leases; if a worker fails, its lease is revoked and the session is retried on a healthy instance.

Record environment failures separately from zero rewards so that infrastructure glitches never corrupt the training signal.

Benchmark Performance

MAI-Gym unifies online RL for GUI agents, cutting environment costs and boosting performance.

MAI‑Gym‑9B attains the highest Task OOD score on MobileWorld among comparable GUI agents.

Table 4 shows a 78.4 % success rate, surpassing UI‑Venus‑2‑9B by 3.2 points and GUI‑Owl‑1.5‑8B by 40.2 points.

Across all seven benchmarks, MAI‑Gym‑9B also outperforms larger agents such as GUI‑Owl‑1.5‑32 B and EvoCUA‑1.5‑32 B, and comes within 2.5 points of GPT‑5.5 on OSWorld‑Verified.

**Figure 6.** Gains from SFT warm-up and joint RL.

**Figure 7.** Efficient behaviors learned by MAI-Gym-9B. Top: adaptive zoom in MobileWorld. Middle: batched GUI actions in OSWorld. Bottom: CLI tool use in OSWorld.

**Figure 8.** Synthesized task scaling.

**Figure 9.** Clone-to-real transfer (12306, 52 tasks).

**Figure.** Initial title slide. Task. Read the job ticket, handwritten notes, and confirmation email to consolidate the printing settings for three proofs in Calc. Add one slide per proof to the comparison deck. Export the spreadsheet to CSV and the completed presentation to PDF.

Table 7 shows that joint RL (78.4 % MobileWorld, 76.2 % OSWorld‑Verified) exceeds all expert‑only and merged baselines, confirming the benefit of learning from combined rewards.

Efficiency and Reliability Analysis

Ablation studies quantify the impact of design choices on efficiency and performance.

This section isolates the contribution of each component by removing or altering it and measuring the resulting performance drop.

Window‑wise history cuts the training update time from 2166.4 s to 810.9 s.

Table 6 shows the reduction while only modestly decreasing rollout throughput.

Joint RL attains 78.4 % (MobileWorld) and 76.2 % (OSWorld‑Verified), surpassing the best expert‑merge baselines.

Table 7 reports the joint‑RL success rates versus weighted parameter averaging merges (76.4 % and 74.8 %).

Scaling synthesized tasks to 5 000 improves MobileWorld success from 69.2 % to 78.4 %, a +9.2 pp gain.

Figure 8 illustrates the monotonic increase in success as the task pool grows.

Clone‑only RL raises clone success to 73.1 % (↑36.6 pp) and real‑app success to 48.1 % (↑19.3 pp).

Results on 52 paired tasks show the transfer gap widens despite the gains.

RL increases unfamiliar‑app success to 45.2 % (↑19.4 pp) and reduces mean trajectory length to 16.4 steps.

Evaluation on 62 held‑out tasks under a 60‑step budget shows fewer budget‑exhaust attempts (5 vs. 27).

Comparing the GRPO recipe with SAO shows similar early task success, but SAO’s extra critic updates offset its reduced sampling wait time, yielding no clear overall training‑time advantage.

**Figure 12.** **RL enables efficient task completion in a novel app.** Before RL, repeated ineffective gestures exhaust the 60-step budget; after RL, the agent moves the folder and reports unchanged free space, passing all four checks in five steps.

Related Work

Survey of GUI‑RL systems and platforms, highlighting MAI‑Gym’s unifying role.

Recent work on GUI agents spans a wide spectrum of domains and training recipes. Researchers have built domain‑specific environments, reward functions, and RL pipelines for Android, web, and desktop interfaces, while others focus on reusable platforms that unify evaluation and training. MAI‑Gym joins this landscape by offering a single open platform that runs online RL across mobile, computer, and web benchmarks.

DigiRL is an Android‑focused RL framework that experiments with reward design, asynchronous data collection, and adaptive training schedules.

OSWorld is a benchmark suite of verified desktop GUI tasks that evaluates agents on realistic operating‑system interactions.

Unifies web benchmarks behind a single interface, enabling reproducible evaluation of web‑based agents.

Runs online RL on hundreds of parallel web environments, supporting large‑scale training of web agents.

Provides an online‑RL platform for mobile GUI tasks, handling parallel execution on Android devices.

Framework for training, evaluating, and deploying GUI agents, focusing on reproducible evaluation pipelines.

Gym that supplies computer GUI environments and tasks but does not release training code.

Data‑focused gym that contributes new GUI environments without providing training implementations.

Mobile GUI benchmark that supplies a suite of tasks and mock apps for RL research.

Gym for language‑model agents operating in text‑based environments, supporting tool use and reasoning.

Gym for LLM agents in tool‑based environments, focusing on execution of external commands.

Policy‑optimization library used by MAI‑Gym to implement scalable online RL.

Task‑generation system that scores success with a vision‑language model (VLM) judge.

Explores an app with a VLM before proposing tasks, using the VLM as a verifier.

Generates trajectories in reverse order to synthesize tasks for GUI environments.

Models GUI environments as finite‑state machines, enabling verifiable task generation.

Scalable web‑task synthesis framework that produces verifiable tasks for RL agents.

Derives rewards from mobile UI sub‑states, offering a more reliable but narrower reward signal.

Synthesizes desktop tasks and initial states around application‑specific verifiers.

Transforms recurrent agent failures into capability‑targeted environments for focused training.

Co‑evolves environments together with the agent, allowing the task distribution to adapt during training.

Joint evolution framework where environments and agents are updated in tandem.

**Figure 11.** Relative-date reasoning after clone-only RL. Top: incorrect dates before RL. Bottom: the correct Monday is selected and the reminder is set in both the clone and real app after RL.

Implementation Details

Provides detailed tables and descriptions of the MAI‑Gym dashboard, interface contract, and action space.

The appendix supplies concrete reference material: a screenshot of the MAI‑Gym dashboard, a concise summary of the environment‑adapter API, and the full list of actions supported by the platform.

**Table 8.** Responsibilities across the environment contract.

The table lists various actions categorized into GUI Actions, CLI Actions, API Actions, and Interaction and Control Actions, providing a definition for each.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers