StudentSim: Training LLM-based Student Simulators
Ke Yang, Chenglong Wang, Michel Galley, Chandan Singh, Jeevana Priya Inala, ChengXiang Zhai, Jianfeng Gao
STUDENTSIM builds personalized student simulators that mirror individual learner behavior and respond to tutor guidance.
How can we train LLM-based student simulators that accurately predict individual learner behavior and provide reliable reward signals for optimizing AI tutors?
AI tutors need to adapt to individual students, but collecting real-world feedback is slow and expensive. Existing simulators either fail to capture a specific student's unique competence or cannot meaningfully update their behavior when given tutor guidance. STUDENTSIM solves this by training a base model on pooled student data, then specializing it for each individual using their specific records. This two-stage pipeline creates simulators that are both faithful to a student's baseline and responsive to pedagogical intervention. Across chess, English writing, and math, these simulators outperform frontier LLMs on both behavioral fidelity and guidance responsiveness, providing a reliable reward signal for training better AI tutors.
Paper Primer
The core challenge is the "competence paradox": prompted LLMs are fluent but unreliable, often solving problems the student cannot or ignoring guidance entirely. STUDENTSIM addresses this by formalizing two metrics: behavioral fidelity ($F$), which measures how well the simulator matches a student's actual response, and guidance responsiveness ($R$), which measures how well it updates toward a correct answer after receiving tutor feedback.
The method uses a two-stage training pipeline: Stage 1 pools data across all students to learn domain-wide patterns like common mistakes and guidance-following pathways; Stage 2 specializes the model for each individual using their sparse, personal records. This is like a student learning the general rules of a game from a textbook (Stage 1) before refining their specific playstyle through a coach's feedback on their own past games (Stage 2).
STUDENTSIM simulators significantly outperform GPT-5.4 on both fidelity and responsiveness across all three domains.
In chess, STUDENTSIM achieves $F=0.51$ and $R=0.91$, compared to GPT-5.4's $F=0.23$ and $R=0.72$. The framework enables a 2x improvement in behavioral fidelity in chess and a 3x improvement in guidance responsiveness compared to specialized move-prediction models like Maia2.
Trained simulators serve as effective reward models for reinforcement learning (RL) in AI tutor training.
Expert human raters ranked tutors trained with STUDENTSIM rewards as more accurate, better-guided, and more personalized than those trained with GPT-5.4 rewards or no RL. Accuracy of tutor guidance improved from 75.7% (no RL) to 90.5% (STUDENTSIM reward).
Why is a two-stage training process necessary instead of just fine-tuning on individual data?
Individual student records are too sparse to train a model from scratch without overfitting. The pooled Stage 1 provides the necessary shared structure—such as domain-specific mistake patterns—that a single student's limited data cannot supply.
What is the primary limitation of existing "LLM-as-student" approaches?
Existing models are prompted to role-play but lack the underlying training to map a student's cognitive state to specific behavioral consequences. They often exhibit "surface mimicry," where they sound like a student but fail to reproduce the specific errors or competence level of the target individual.
The framework assumes that a student's learning dynamics can be captured by their response to one-step guidance; it does not yet model long-term knowledge retention or forgetting over multiple sessions.
By providing a reliable, teachable proxy for human students, STUDENTSIM enables scalable reinforcement learning for AI tutors, shifting the bottleneck from data collection to model optimization.
The Student Simulation Problem
StudentSim turns sparse per‑student data into individualized simulators that supply fast, high‑frequency feedback for AI tutor training.
Real‑student feedback arrives at human timescales, making it slow, costly, and sparse—hard to collect at the scale needed for training adaptive AI tutors.
Effective simulators must meet two criteria: behavioral fidelity (F), i.e., how closely the simulator reproduces a student’s own responses, and guidance responsiveness (R), i.e., how readily it updates its response when a tutor provides guidance.
Prior work satisfies only one of these criteria—state‑tracking models capture fidelity but cannot ingest tutor explanations, while LLM‑prompted role‑players follow guidance but fail to reproduce individual student competence.
StudentSim builds a personalized simulator for each learner by first learning shared patterns across many students, then fine‑tuning on that learner’s own sparse records.
**Figure 1** Real-student feedback is hard to scale; a student simulator supplies it at machine timescales.
**Figure 2. Scoring a student simulator.** Education tracks a learner’s progress through what they produce independently and how far they advance once support arrives, the zone of proximal development [VYGOTSKY, 1980] that dynamic assessment operationalizes [Grigorenko and Sternberg, 1998]. We score whether a student simulator can serve as a feedback source for a tutor, which rests on two abilities mirroring those signals: *behavioral fidelity*, behaving like the student it stands for, and *guidance responsiveness*, moving where the tutor’s guidance indicates (a). On chess (b), STUDENTSIM leads on both, the prompted LLM on responsiveness alone, and the state-tracking model on fidelity alone, and the same ranking holds in L2 and math (Table 2 and Table 3).
**Figure 3.** Two-stage training pipeline for STUDENTSIM. Stage 1 pools the sparse records of many students in a domain and trains one base simulator on what they share. Stage 2 specializes that base to a single student on that student's own records, yielding one simulator per student.
STUDENTSIM simulators consistently outperform both a strong LLM‑prompted baseline (GPT‑5.4) and a state‑tracking baseline (Maia2) on the F and R metrics across chess, L2 English, and math.
When a StudentSim simulator serves as the reward model for reinforcement‑learning‑based tutor training, the resulting chess tutor is judged by expert humans to be more accurate, better guided, and more personalized than tutors trained without RL or with a GPT‑5.4 simulator reward.
The key shift is moving from human‑in‑the‑loop feedback to simulator‑in‑the‑loop tutor optimization.
Prior Work and Framework
We situate prior simulators, define per‑student metrics, and introduce the Maia2 baseline.
Prior work on student simulation clusters into three strands. First, cognitive‑state and behavior‑prediction models (e.g., knowledge tracing) capture population‑level patterns but lack a natural‑language guidance channel. Second, LLM‑prompted simulators condition on textual descriptions and have been deployed as dialogue‑tutoring corpora, multi‑agent classrooms, and domain‑specific learners. Third, tutor‑side research either curates expert dialogues for supervision or queries a generic LLM judge against rubric‑based quality metrics.
Models such as knowledge tracing and human‑play move predictors estimate learner behavior at the aggregate level, using latent cognitive states to forecast next actions.
These simulators are conditioned on natural‑language prompts, enabling them to engage in conversational tutoring, generate learner data, and populate multi‑agent classroom environments.
Approaches either curate expert dialogues to supervise tutor explanations or employ a generic LLM judge to assess output against a rubric, sometimes using a knowledge‑tracing model as a proxy for student response.
To our knowledge, STUDENTSIM is the first work to (i) formalize per‑student simulation as a measurable target jointly defined by behavioral fidelity and guidance responsiveness, (ii) train simulators on real per‑individual data so they both match a student’s own behavior and react to natural‑language tutor guidance, and (iii) employ the trained simulator as the reward source for AI‑tutor optimization.
Maia2 is a prior per‑student simulator that adapts a generic language model to each learner using only their observed responses, without explicit guidance conditioning.
Problem formulation introduces the student set $\\{\\pi_1, \\dots, \\pi_N\\}$ and the corresponding simulators $\\{M_1, \\dots, M_N\\}$. For each student $i$, single‑turn records $S_i = \\{(x, m)\\}$ capture a problem $x$ and the student’s response $m$, while multi‑turn records $T_i = \\{(x, m, \\tau, m^*)\\}$ also include tutor guidance $\\tau$ and the target corrected response $m^*$. Orthogonality holds: high behavioral fidelity $F$ and high guidance responsiveness $R$ are jointly required for a useful simulator.
Experimental Design
Details datasets, tasks, guidance types, and two‑stage training for chess, L2, and math.
Each domain supplies single‑turn records (problem → student response) to learn baseline behavior and multi‑turn records (problem → wrong response → tutor guidance → corrected response) to learn how a student updates after guidance. The chess data come from Lichess, L2 from EFCAMDAT, and math from a foundational‑assistance corpus, all collected from real human learners.
Stage 1 mixes multi‑turn records at a ratio of 0.2 in the training mix.
The multi‑turn ratio is set to 0.2 when combining single‑turn and multi‑turn examples.
First learn population‑level behavior, then specialize each student’s adapter on their own data.
How does this two‑stage pipeline differ from standard fine‑tuning on each student separately?
Standard fine‑tuning starts from the base model for every student, discarding any shared behavior learned across the population. The two‑stage pipeline first builds a domain‑wide adapter that captures common patterns, then only refines the adapter per student, which reduces over‑fitting and speeds up convergence on the limited per‑student data.
The three domains—chess, second‑language writing, and mathematics—demonstrate that the framework generalizes across distinct learning contexts.
Predicting Student Behavior
Behavioral Fidelity results showing StudentSim outperforms baselines across domains.
StudentSim achieves per‑player top‑1 fidelity $\mathcal{F}=0.51$, surpassing Maia2 (0.45), GPT‑5.4 (0.23), and GPT‑4o (0.22).
Figure 4 shows the per‑player fidelity for each model on the held‑out 30‑player chess set.
F quantifies how often the simulator predicts the exact move or answer a specific student actually made, evaluated per individual.
How does Behavioral Fidelity differ from standard accuracy metrics?
Standard accuracy aggregates predictions over all users, rewarding the most common answer; Behavioral Fidelity evaluates each individual’s exact response, so a model must reproduce each learner’s idiosyncratic behavior to score well.
**Figure 4.** Behavior fidelity on a held-out chess position. Three real players in our held-out set choose three different moves at this board (panel a): Player A plays e4 (gold, central push), Player B plays e3 (rose, solid), Player C plays Bg5 (teal, pin on Nf6). Maia2 (panel b) collapses all three onto the ELO-modal e4. GPT-5.4 (panel c), prompted in-context with each player's recent history, misses all three (Bg5 / e4 / Bh6). STUDENTSIM (panel d) reproduces all three actual moves. Solid arrows are ground truth; dashed arrows are predictions; ✓ / ✗ mark per-player match. On the full 30-player held-out set, STUDENTSIM reaches per-player top-1 fidelity $\mathcal{F} = 0.51$ vs. Maia2 $\mathcal{F} = 0.45$, GPT-5.4 $\mathcal{F} = 0.23$, and GPT-4o $\mathcal{F} = 0.22$.
Guidance Responsiveness and RL
StudentSim‑based reward yields the highest tutor scores across all human‑rated metrics.
The StudentSim‑based reward produces a tutor that outperforms all baselines on human‑rated accuracy, guidance quality, and personalization.
Table 4 shows the StudentSim condition achieving the highest scores across the three axes.
The StudentSim reward also leads in personalization (3.93) and accuracy (90.5%), compared with 75.7%/2.99 for No RL and 71.6%/3.08 for the GPT‑5.4 reward.
R measures how effectively a simulator revises its answer after receiving tutor guidance, reflecting the simulator’s ability to follow instructions.
The tutor policy is trained with reinforcement learning where the reward comes from a frozen StudentSim that evaluates how much the guidance improves the student’s move.
Why use a frozen StudentSim as the reward source instead of a larger closed‑source model?
Because the frozen simulator is small, open, and its internal representation can be probed to add personalization and perception heads, whereas a closed model offers no such extensibility and incurs high API cost.
**Figure 6** Tutor RL with a frozen student simulator as the reward source. Each episode replays a real student's wrong answer, the tutor policy proposes guidance, the frozen simulator revises, and the improvement in move quality drives the RL update.
Simulator-Grounded Rewards
Ablation analysis of reward components and data pipelines reveals their impact on fidelity and responsiveness.
This section isolates each reward‑generation component to see whether its removal harms tutor quality, as measured by Behavioral Fidelity (F) and Guidance Responsiveness (R).
When the frozen STUDENTSIM, trained on real learner data, is replaced by a knowledge‑tracing model that predicts only answer correctness, expert raters report a noticeable decline in both F and R, indicating that the simulator’s grounding in authentic behavior is essential.
Substituting the frozen STUDENTSIM with a prompted LLM student—i.e., a generic LLM that imitates a learner—produces a similar drop: the tutor’s guidance becomes less accurate and less personalized, because the LLM’s persona‑following is unstable and does not reflect real‑world response patterns.
Removing the pedagogy judge and relying solely on a generic rubric eliminates the nuanced assessment of tutor utterances; the resulting tutors are judged as less guided, showing that the rubric alone cannot capture the quality of instructional feedback.
Omitting the reinforcement‑learning loop entirely (training the tutor without any reward signal from the simulator) yields tutors that are markedly less effective, confirming that the RL feedback—when sourced from the frozen STUDENTSIM—is a critical driver of improvement.
**Table 1.** Summary of the per-domain Stage 1 and Stage 2 training data scale.
No caption provided.
Grounding the reward signal in a frozen simulator trained on real learner data is the decisive factor that makes the tutor both accurate (high F) and responsive (high R).
Performance Metrics and Baselines
Ablation sweeps on chess reveal how pooling, modality, turn‑mixing, and guidance composition affect fidelity and responsiveness.
Across all three domains, STUDENTSIM beats the strongest baseline on every guidance mode, including the hardest, most indirect ones such as chess Socratic and math conceptual settings.
Cross‑student pooling in Stage‑1 raises both fidelity and responsiveness compared with training on a single student’s repeated records.
Table 11 shows the pooled arm reaches higher scores than the single‑student arm.
Switching from a text‑only to a vision‑language encoder adds negligible fidelity while doubling training cost.
Varying the multi‑turn mixing ratio $\rho$ shows that any non‑zero amount of guidance data lifts responsiveness into a high‑value plateau while fidelity remains essentially flat.
Guidance‑mode composition experiments reveal that the four modes are largely complementary; dropping most modes does not harm, and sometimes improves, the evaluation score of the omitted mode.
**Table 11.** Cross-student pooling is what the pooled stage contributes. In Table 11, the reference arm reaches $\mathcal{F} = 0.5131$ and $\mathcal{R} = 0.9003$, while the repeated-single-student arm reaches $\mathcal{F} = 0.4602$ and $\mathcal{R} = 0.8276$. The same pattern holds for each player individually: fidelity drops from 0.4558 to 0.3992, from 0.4660 to 0.4186, and from 0.6174 to 0.5628, while responsiveness drops from 0.9093 to 0.8363, from 0.8928 to 0.8238, and from 0.8990 to 0.8228.
**Table 12.** Comparison of modality performance and training cost.
**Table 13.**
The table presents training statistics across three domains: chess, L2, and math. It includes columns for "Domain", "training instances", "total grad steps", and "wall-clock time".
**Table 15** Single-mode training on chess. Each row trains the pooled-training mix on one guidance mode only and evaluates on all four. Even one-mode training reaches $\mathcal{R} \in [0.72, 0.80]$, against 0.836 for the four-mode default. Comparative-only transfers best to the other three modes, and error-remediation-only transfers worst.
**Table 16** Weight-scaling sweep on chess: scaling the Socratic mode while keeping the other three at 1. Multiplying the Socratic weight raises the Socratic per-mode score from 0.611 to 0.670 (+5.9 pp at 4×). The other three modes shift by a few points at the strongest setting, and overall $\mathcal{R}$ is close to flat.
Per-Mode Weighting Analysis
Details of training configurations, per‑mode weighting, and hyperparameters for the reference simulators.
The per‑mode weighting handle lets practitioners tilt the simulator toward stronger or weaker responses to Socratic guidance without altering the training pipeline.
By scaling the contribution of each training mode in the pooled corpus, one can produce a simulator that either amplifies or dampens its reaction to tutor guidance.
Training uses the Qwen3‑4B‑Instruct‑2507 model in bf16 precision on a single node with eight A100 GPUs, applying LoRA adapters in two stages.
The two‑stage design shares all hyperparameters except learning‑rate‑related settings and domain‑specific batch sizes, ensuring that differences across chess, L2, and math stem only from dataset scale.
The table presents training parameters across three domains: chess, L2, and math. Columns include "Domain", "students", "per-student instances", "Stage-2 $\rho$", "epochs", "effective batch", and "total steps".
**Table 18.** Per-domain pooled-training-stage corpus size and realized optimization length. Corpus sizes match Table 1 in Section 5.1 the multi-turn fraction within each corpus is the shared $\rho = 0.20$ from Table 17. Wall-clock times are rough estimates.
**Table 20.** Per-domain per-student-specialization-stage settings. “Per-student instances” is the training-set size used for one student’s specialization run; for a given domain this size is constant across the per-student adapters in our reference family. “Total steps” is the number of optimizer steps in one student’s specialization run. The chess and math effective batch of 256 is realized as per-device batch 4 × gradient accumulation 8 × 8 GPUs; the L2 effective batch of 8 is per-device batch 1 × no accumulation × 8 GPUs.
Extended Related Work
A survey of prior student‑simulation models, LLM‑prompted simulators, and tutor‑side evaluation.
Section A.1 surveys cognitive‑state and behavior‑prediction models, beginning with classic knowledge tracing and its Bayesian, deep, and attentive variants, then extending to key‑value memory, transformer‑based KT, auxiliary‑task DKT, IRT families, and performance‑factor analyses.
LLM‑based knowledge tracing emerged in 2024, adding pretrained language‑model representations of question text and dialogue, and later uncertainty‑aware embeddings, to improve cold‑start generalization and handle tutor‑student exchanges.
The line from correctness prediction to response generation includes Option Tracing, which predicts distractor choice, and Open‑Ended KT, which generates free‑form answers—both precursors to StudentSim’s response‑generation objective.
Human behavior cloning in chess is represented by Maia and its successor Maia2, which condition move‑prediction networks on rating descriptors; however, these models update state only from labeled outcomes, lack any pathway for free‑form guidance, and consequently achieve high $F$ but near‑zero $R$.
Section A.2 reviews LLM‑prompted student simulators, which prepend a textual description of a target learner to a frozen LLM prompt and let the model continue the conversation.
The critical observation is that a frozen LLM, trained for plausible language, often solves problems beyond the described student’s level or mimics surface style while missing the student’s actual errors, leading to lower $F$ despite higher $R$ in some cases.
Section A.2.1 discusses student‑simulation validity, highlighting the competence paradox—LLMs over‑apply their general reasoning when asked to act as partially knowledgeable learners—and empirical work that finds systematic solve‑rate and error‑type mismatches.
Proposed architectural fixes include adding cognitive prototype modules or controlled imperfect reasoning to force student‑like errors, and teacher‑facing studies report that LLM students appear overly attentive, linguistically sophisticated, and inconsistent across turns.
Section A.2.2 surveys educational simulators that vary in granularity (persona, profile, group, individual), conditioning interface (prompt, retrieval, fine‑tuning), and evaluation target (stylistic plausibility, downstream task, response fidelity); StudentSim uniquely combines per‑individual weight‑level training with the $R$ guidance‑response axis.
Section A.3 covers tutor‑side evaluation and optimization, describing tutor models, judges, and benchmarks, proxy‑reward and LLM‑as‑judge methods, multi‑dimensional pedagogical evaluation, classroom‑scale studies of tutor assistance, LLM‑tutor instruction‑tuning, and conditioning tutor output on student descriptors or pedagogical strategies.
Mathematics Corpus Statistics
Dataset statistics and evaluation metric definitions for fidelity and responsiveness across domains.
Behavioral Fidelity (F) measures how often a simulator’s top‑1 prediction matches the student’s recorded response; it is computed as the mean of per‑instance indicator scores across a held‑out set.
Guidance Responsiveness (R) measures whether a simulator updates its top‑1 prediction to the canonical corrected move after receiving tutor guidance; it is the mean of per‑instance indicator scores over guidance‑augmented records.
The table provides a summary of the dataset statistics used in the study. It lists various fields such as Source corpus, Curriculum, Total students, Total interactions, Unique problems/skills, Interactions per student, and Per-record response space, along with their corresponding values.
**Table.** Three findings come out of this chess composition sweep.
**Table 9** L2: $\mathcal{R}$ by guidance mode across the two L2 tutor modes (point-based, rule-based). Population mean across the 15 L2 learners’ held-out multi-turn records, 20 records per mode within each learner.
**Table 10.** Math: $\mathcal{R}$ by guidance mode across the three math tutor modes (error remediation, Socratic, conceptual). Population mean across the 15 math students' held-out multi-turn records, 293 records per mode in the 879-record total (modes evenly represented within each per-student $T_i$).
**Table 5.** Chess corpus statistics. Stage-1 / Stage-2 training counts and per-student held-out evaluation counts ($\mathcal{F}$ test, $\mathcal{R}$ test) are reported in Table 1.
This table provides statistics regarding the EFCAMDAT error-coded subcorpus, including counts of essays and learners, as well as metrics for essays per learner and the categories used for LanguageTool issue-type buckets.
Training Infrastructure and Costs
Appendix F details compute usage, metric definitions, corpus construction, and evaluation scope.
Guidance Responsiveness ($R$) measures how well a simulated learner follows tutor guidance. It scores whether the simulator emits the canonical corrected response $m^{*}$ after reading the tutor turn $\tau$, using domain‑specific targets (best chess move, correct math answer, or teacher‑annotated correction). This metric supplies the feedback signal needed for tutor reinforcement learning.
Multi‑turn tuples $(x, m, \tau, m^{*})$ are built by pairing problems with student errors, tutor guidance, and canonical corrections. In L2, real teacher annotations provide $m^{*}$ and templated $\tau$; in chess and math, an LLM generates $\tau$ conditioned on the error and $m^{*}$, ensuring controlled guidance styles. This decoupling lets the simulator consume guidance without producing it, enabling per‑style $R$ analysis.
Behavioral Fidelity ($F$) quantifies how closely the simulator’s response matches the recorded student answer within each domain. Chess uses top‑1 accuracy against the recorded move, L2 uses error‑density and issue‑type profile matching, and math uses top‑1 accuracy on a four‑way multiple‑choice formulation. Reporting relative improvements over domain baselines preserves meaningful within‑domain interpretation while allowing cross‑domain comparison of gains.