Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?
Yibo Li, Jinhang Qiu, Zhi Zheng, Qianyun Guo, Jiaying Wu, Shuo Ji, Bryan Hooi
Learn2Play Bench evaluates how LLM agents learn from experience in novel, rule-hidden environments.
How effectively do current LLM agents learn from experience when placed in unfamiliar, dynamic environments?
Existing benchmarks often rely on tasks where agents can lean on pre-existing knowledge, making it impossible to tell if an agent is truly learning new rules or just recalling training data. The authors introduce Learn2Play Bench, a suite of text-based games with counterintuitive, hidden rules that require agents to discover mechanics through interaction. Agents play repeated episodes, and the benchmark measures their ability to improve performance and transfer knowledge to new instances. Top-performing humans consistently outperform agents, who often struggle to balance exploration with exploitation and frequently repeat suboptimal strategies.
Paper Primer
The benchmark uses 20 game templates with deterministic feedback and automatic scoring, split into fixed instances and re-shuffled instances. The core move is to force agents to learn from scratch: they must issue actions, observe textual feedback, and retain experience across episodes without any weight updates.
Raw interaction history outperforms summarized experience.
Methods that retain full, unmodified interaction logs (MEMORY) consistently lead performance metrics compared to methods that distill experience into rules, workflows, or reflections. MEMORY leads all four metrics (Max, Mean, Learning Gain, Learning Slope) when using the Kimi K3 backbone.
Agent harnesses significantly impact learning efficiency.
Holding the backbone model constant, switching to a more specialized harness (e.g., Claude Code or Codex) improves performance while reducing inference costs. Claude Code with Opus 5 outperforms the baseline OpenCode harness on all four metrics while reducing token usage.
Why is it difficult to evaluate "learning" in current LLM agent benchmarks?
Many existing tasks are based on real-world settings already represented in pretraining, meaning agents may improve by recalling familiar knowledge rather than discovering new rules through interaction.
How do human players differ from agents in these environments?
Humans explore more varied strategies and recover from setbacks more frequently, whereas agents tend to exploit discovered strategies consistently, often stopping exploration prematurely even when better strategies exist.
Researchers should prioritize retaining raw interaction evidence over aggressive compression and focus on improving agent exploration strategies, as current agents are significantly less effective than humans at discovering rules in unfamiliar settings.
Introduction and Motivation
We expose the gap between static benchmarks and agents’ ability to learn dynamically.
LLM agents excel on static tasks, yet their capacity to learn from experience in unfamiliar, dynamic settings is rarely quantified. Existing benchmarks embed the rules in instructions, conflating pretrained knowledge with genuine learning. This section isolates the missing piece: a systematic way to measure how agents acquire and reuse knowledge across repeated episodes.
Learn2Play Bench is a suite of text‑based games whose hidden rules are novel or counterintuitive, forcing agents to discover them through interaction rather than relying on pretrained knowledge.
**Figure 1.** Learn2Play Bench overview. The left panel shows 20 manually designed game templates, each of which generates multiple instances using different random seeds. The highlighted POISONER template illustrates the interaction protocol on the right: the environment describes the state and enumerates valid actions, the player selects one action, and the environment returns the resulting observation. The episode ends with an automatically computed score (Appendix D.16).
The core gap is that static evaluation overlooks agents’ ability to learn from experience in unfamiliar, dynamic environments.
Related Work and Evaluation Design
We situate our benchmark among prior self‑evolving agents and outline its design criteria.
Agents that keep their internal model fixed but improve behavior by reusing experience gathered during test‑time interactions.
Identifying hidden game rules from interaction feedback, enabling the agent to predict future consequences of actions.
Reuses past experience by having the model generate a textual reflection and then condition subsequent actions on that reflection.
Extracts insights from previous trajectories and retrieves them to inform future decisions.
Provides reusable executable skills that the agent can invoke during test‑time interaction.
Induces structured action workflows from observed trajectories.
Curates compact strategy snippets and code fragments for rapid reuse.
Incrementally maintains structured playbooks rather than raw transcripts.
Distills trajectories into reusable reasoning patterns.
Distills trajectories into heuristics that guide future decisions.
Retrieves past trajectories to estimate action advantages and adjusts output logits at inference time.
Evolves prompts through trajectory‑level reflection.
Revises both prompts and agent code via specialized collaborative roles.
Extends evolution to agent configuration, including memory and tool‑use routines.
Evolves the adaptation policy that determines how experience updates the agent.
Generates training examples from uncertain cases and fine‑tunes the agent at test time.
Evaluates agents over input–feedback streams to measure incremental improvement.
Studies learning from accumulated user feedback across task streams.
Evaluates memory updates across task streams.
Tests skill accumulation across related tasks.
Evaluates learning across successive coding issues.
Tests memory use across interdependent sessions.
Creates new environments to evaluate learning across diverse settings.
Measures repeated improvement on Jericho games.
Evaluates interaction with unfamiliar visual environments.
Evaluation Design follows four principles: rule novelty, automatic scoring, reproducibility, and controlled variation across episodes.
We build the Learn2Play Bench with 20 text‑based game templates that expose hidden rules, some deliberately counterintuitive, and provide deterministic scores for each episode.
Games are split into ten fixed instances (identical across episodes) and ten re‑shuffled instances (visible layout changes while the core rule stays constant).
Pilot runs identified five challenge games; we evaluate them over ten episodes, while the remaining games use a five‑episode window.
Experimental Results
Experiments quantify agent performance and learning dynamics across diverse backbones and methods.
Each game’s episode timeline is stretched onto a common 0–100 % progress axis so that learning trajectories can be plotted side‑by‑side despite differing episode counts.
How does this differ from simply normalizing scores without rescaling episode indices?
Score normalization removes scale differences between games, but without progress rescaling the curves would be misaligned in time—early episodes of a 10‑episode game would be plotted alongside later episodes of a 5‑episode game, obscuring true learning speed.
Claude Opus 5.5 achieves the highest peak (Max) and average (Mean) scores, while Qwen 3.8 Max records the strongest learning gain (LG) and learning slope (LS) among backbones.
Table 1 shows Opus 5.5 with Max = 80.1 and Mean = 61.0, and Qwen 3.8 Max leading the LG and LS columns.
**Figure 2.** Normalized learning curves across all 20 games for (A) OpenCode backbones, (B) self-evolving methods with Opus 5, and (C) human Top-k groups and selected agents.
**Figure 3.** Max score versus mean cost per episode. The dashed line shows the performance–cost frontier for configurations with standard-price estimates.
Most agents fail to show consistent rule discovery despite performance gains.
Additional Results by Research Question
Key extra findings on agent learning, harness effects, and cost efficiency.
Self‑evolving agents retain most of their learning when game instances change, with MEMORY agents losing only 0.88 in learning‑slope (LS) compared to a 3.76 LS drop in the static control condition.
Table 9 shows LS = 3.76 for MEMORY in the control setting and LS = 2.88 after instance shuffling, yielding a drop of 0.88.
The experiment holds the backbone model constant (Opus 5) while only swapping the sequence of game instances, isolating the effect of instance variability.
**Figure 4.** Learning curves for MEMORY with four backbone models. Scores use the shared normalization and game weights. Relative progress aligns the five- and ten-episode scoring windows.
Additional case studies (Sections B.1.1–B.1.4) illustrate rule discovery, exploration limits, and feedback reuse, but they are anecdotal and not meant to quantify frequency.
RQ2’s harness comparison (Table 2) shows that disabling persistent auto‑memory (Claude Code) or using an --ephemeral setting (Codex) reduces cross‑episode transfer, confirming the importance of session continuity.
Human versus agent behavior (Table 8) indicates that top‑human players achieve higher Max scores (84.3) than any agent, while agents excel in learning‑slope (LG) and learning‑gain (LG) metrics.
Transfer experiments (Table 9) reveal that NAIVE agents suffer a negative LS drop (‑0.08), whereas MEMORY, Reflexion, and ReasoningBank retain most of their learning when instances are shuffled.
Budget sensitivity (Table 10) shows that including extended‑budget trajectories slightly adjusts LS estimates (e.g., MEMORY LS changes from 3.76 to 3.34), but the relative ordering of methods remains stable.
Cost analysis (Table 11) highlights that MEMORY incurs the highest input token volume (1.26 M per episode) and that generation costs dominate the USD per episode budget.
Efficiency ratios (Table 12) demonstrate that configurations with Max ≥ 70 achieve the best Max‑per‑USD values, while Astra’s billed cost is excluded from ratio calculations due to missing token usage.
Aggregate Results and Trial Variability
Aggregate performance and trial variability across all configurations.
Performance spans a wide range across the 31 configurations, with the highest‑scoring model reaching a Max of 80.1 ± 2.7 and the lowest scoring 34.0 ± 3.2.
Table 13 (20‑game aggregate) reports these extreme values.
Trial variability measures how much the reported scores change when the same agent configuration is run on a fresh random seed.
Why report standard deviations instead of standard errors?
Standard deviations describe the spread of performance across independent runs, which reflects the agent’s robustness to random factors. Standard errors would shrink with more trials, but with only three runs they would give a misleading impression of precision.
**Table 13.** Performance and learning over 20 games. Values are point estimates ± standard deviations estimated from within-game trial variability.
**Table 14.** Performance and learning over 10 fixed games. Values are point estimates ± standard deviations estimated from within-game trial variability.
**Table 15.** Performance and learning over 10 re-shuffled games. Values are point estimates ± standard deviations estimated from within-game trial variability.
Per-Game Results: PrimordialSoup to Catnip
Per‑game results and rule‑discovery evidence across the benchmark games.
Self‑evolving agents consistently achieve higher maximum scores than backbone models while rule‑discovery completeness varies across games.
See the performance tables (Tables 16, 19, 22) and rule‑discovery evidence tables (Tables 18, 21, 24).
PrimordialSoup challenges agents to discover reaction recipes within a fixed budget; the learning curves and performance table (Table 16) summarize normalized scores and trial variability, while Tables 17 and 18 list the rule‑discovery criteria and evidence.
Gemforge requires agents to uncover multi‑tier recipes and manage limiting ingredients; Table 19 presents normalized performance, and Tables 20 and 21 provide the corresponding rule‑discovery criteria and evidence.
Catnip focuses on learning action–target matches and optimal exit timing; Table 22 reports normalized performance, while Tables 23 and 24 detail the rule‑discovery criteria and evidence for each target.
**Figure 5.** PrimordialSoup: raw-score learning curves within the scoring window.
Per-Game Results: Dungeon to DreamGarden
Benchmark results show how different agents learn and discover rules across three environments.
The three environments differ in task structure, yet all share the same evaluation pipeline: fixed game, scored seed, first‑N episodes, and a reference score for normalization.
Claude Code + Opus 5 attains the highest maximum score (85.2) in DreamGarden, outpacing all other configurations.
Table 31 reports a Max of 85.2 ± 0.0 for this configuration, the top entry among 30 models.
**Figure 6.** GemForge: raw-score learning curves within the scoring window.
**Figure 7.** Catnip: raw-score learning curves within the scoring window.
**Figure 8.** Dungeon: raw-score learning curves within the scoring window.
Per-Game Results: Ecosphere to Castaway
Ecosphere results: top scores, learning gains, and raw‑score curves across configurations.
Claude Code + Opus 5 attains the highest maximum raw score on Ecosphere.
Table 34 shows a max of 85.0, while the next best configuration reaches 10.7.
The task fixes the Ecosphere game, uses seed 1, evaluates the first five episodes, and compares against a reference score $C = 339$.
**Figure 11.** Ecosphere: raw-score learning curves within the scoring window.
Per-Game Results: Roadside to Plague
OpenCode + Opus 5.5 dominates the Roadside Observatory benchmark, while other configurations show varied learning gains.
OpenCode + Opus 5.5 attains the highest maximum score (96.8) on the Roadside Observatory benchmark.
Table 46 reports a Max of 96.8 ± 5.6, outpacing the next best configuration by several points.
**Figure 15.** Roadside Observatory: raw-score learning curves within the scoring window.
Per-Game Results: Patch Reality & Butterfly
Patch Reality shows agents achieving near‑reference raw scores, far surpassing earlier benchmarks.
The Patch Reality benchmark asks agents to remember diagnosis‑repair mappings, handle decoys, and respect prerequisites across five episodes of a fixed game (seed 1). Scores are reported as raw points with a reference ceiling of 960.
Agents attain raw scores an order of magnitude higher than on the earlier Plague benchmark.
Peak scores on Patch Reality reach ≈850, whereas the best Plague scores top out near 100.
**Figure 19.** Patch Reality: raw-score learning curves within the scoring window.
Table 58 lists eight rule‑discovery targets (T1–T8) that define the diagnostic‑repair grammar, from per‑level action specifications to audit‑only constraints.
Table 59 records evidence for each target across backbone and agent‑harness configurations; every backbone (e.g., Claude Opus 5, Qwen 3.8 Max) achieves complete evidence (S) on all targets, while the two agent harnesses also attain full evidence.
Table 60 aggregates performance metrics (First, Max, Mean, LG, LS) for all configurations, showing that the top‑performing backbone (Claude Opus 5) attains a Max of 850 ± 0.0 and a Mean of 845 ± 2.1, with comparable LG and LS gains.
Per-Game Results: RedEye to Model Auditor
RedEye, Butterfly, and Model Auditor results reveal performance peaks and rule‑discovery evidence.
This section reports the quantitative outcomes for the three benchmark tasks—RedEye, Butterfly, and Model Auditor—highlighting top scores and rule‑discovery success.
RedEye configurations reach the reference score of 100, achieving the highest Max performance.
Table 67 shows OpenCode + Opus 5 attaining Max = 100 and the strongest Learning Gain (+68.9 ± 16.8).
**Figure 22.** RedEye: raw-score learning curves within the scoring window.
Discussion and Future Directions
We outline open questions and research directions for improving LLM agents’ learning ability.
The paper’s premise is that LLM agents are evaluated on static tasks, yet their capacity to learn in dynamic, unfamiliar environments remains under‑quantified.
Human top‑1 players achieve higher peak scores than any agent configuration because humans explore a broader set of strategies across episodes, whereas agents often reuse strategies that have already succeeded.
Future research should develop mechanisms that let agents deliberately test new strategies even when their current policy yields high rewards, thereby improving the exploration–exploitation balance.
MEMORY, the Kimi K3 self‑evolving method, outperforms other approaches by preserving the full interaction history instead of compressing it into distilled rules.
Other methods that distill interactions into abstract rules can lose critical evidence, as illustrated by EvoTest in GEMFORGE, which tried a single gem combination and then, via a prompt, refused to test unseen combinations.
Linking each learned rule back to the concrete interactions that generated it would allow agents to revisit and revise rules when new feedback becomes available.
Our transfer evaluation reveals that altering surface details—such as layouts or characters—reduces agents’ learning slopes, even when the underlying rules remain unchanged.
Future methods should enable agents to compare experiences across differing situations, isolate invariant rules from instance‑specific details, and verify whether those rules still apply under new conditions.
Appendix Overview
Details the benchmark games, scoring pipeline, and reference data used throughout the appendix.
The appendix first lists the benchmark games and the learning objectives each game encodes, separating games that reshuffle their environment from those that keep a fixed instance.
This table lists ten games and the corresponding learning objectives for the player.
**Table 5.** Fixed games: knowledge accumulated about a recurring instance.
Evaluation proceeds by normalizing raw episode scores, computing per‑game metrics, and finally averaging those metrics across all games.
A.2.1 defines a trial as a sequence of episodes with no cross‑trial experience; each configuration runs three trials, five episodes for normal games and ten for challenge games.
A.2.2 converts a raw score $x_{g,r,t}$ for game $g$, trial $r$, episode $t$ to a normalized score $z_{g,r,t}= \frac{100\,x_{g,r,t}}{C_g}$ using the game‑specific reference $C_g$, then derives metrics such as Max, Mean, LG, and LS from these normalized values.
A.2.3 weights challenge games by 3 and others by 1, then aggregates a metric $M_g$ as $\frac{\sum_g w_g M_g}{\sum_g w_g}$; for curve plots the episode axis is interpolated to ten equally spaced points per game.
**Table 6.** Per-game reference scores used for normalization and their supporting evidence.