FlowBalance: Verifier-Grounded Self-Improvement from on-Policy Reasoning Experience

Zixun Huang, Kishan Panaganti, Haitao Mi, Leowei Liang

FlowBalance uses verifier-grounded trajectory balance to turn dense self-guidance into a stable, normalized policy update.

How can reasoning models improve from their own experience without collapsing into false confidence or over-relying on noisy self-guidance?

Reasoning models often struggle to improve from their own experience because dense self-guidance can reinforce false confidence, while sparse verifier feedback provides insufficient signal to guide long-horizon reasoning. FlowBalance addresses this by combining verifier-derived advantages and privileged-hindsight guidance into a single trajectory energy, which is then used to fit a normalized distribution over complete responses via trajectory balance. This approach improves mathematical reasoning accuracy and training stability across multiple model scales, while avoiding the response-length collapse common in direct imitation methods.

Paper Primer

FlowBalance treats self-improvement as a distribution-learning problem: it maps a group of sampled reasoning trajectories to a target distribution that balances verified outcomes with dense, same-model hindsight. The core move is to use the verifier's group-relative advantage to gate the hindsight signal: positive guidance is retained for successful trajectories, reversed for failures, and disabled when the verifier provides no preference.

FlowBalance consistently outperforms verifier-only and dense-guidance baselines on mathematical reasoning benchmarks.

On a core four-benchmark average (AIME24, HMMT25, MATH500, OlympiadBench), FlowBalance improves over GRPO by 2.44 points and RLSD by 4.18 points on the Qwen3-8B backbone. +2.44 to +5.83 points improvement over existing methods depending on the baseline and model scale.

The method maintains higher semantic strategy diversity in correct solutions compared to standard reinforcement learning.

In a controlled AIME24 diagnostic, FlowBalance achieved a Simpson strategy diversity of 0.2194, compared to 0.1017 for GRPO and 0.1456 for RLSD.

Why is a normalized distribution over complete responses better than just using a token-level imitation loss?

Token-level imitation can suppress uncertainty and force the model to collapse toward a single, locally preferred reasoning mode. By fitting a normalized distribution over complete responses, FlowBalance preserves relative probability contrasts across the entire rollout group, allowing the model to maintain multiple successful reasoning strategies.

How does FlowBalance prevent the model from learning from its own confident but incorrect reasoning?

The method uses the verifier's group-relative advantage as a sign-gate for the hindsight guidance. If a trajectory is rejected by the verifier, the positive hindsight signal is reversed, effectively converting false-positive confidence into a correction that favors verified successes.

FlowBalance demonstrates that verifier-grounded distributional updates are more stable and effective than direct local imitation, providing a principled way to integrate dense self-guidance without sacrificing reasoning diversity or length.

Introduction and Motivation

We expose why self‑improvement loops for reasoning models are fragile and preview FlowBalance as a remedy.

Reasoning models can improve by training on their own on‑policy trajectories, but the loop is fragile: terminal verifiers give reliable yet sparse supervision, while dense self‑guidance from a frozen copy of the same model can reinforce false confidence or over‑focus on a narrow solution mode.

FlowBalance addresses this fragility by grounding dense guidance in verifier outcomes: a frozen policy scores each sampled token, the scores are aggregated, and sign‑gating uses the verifier’s group‑relative advantage to keep positive guidance on verified successes, reverse it on failures, and disable it when the group provides no preference.

**Figure 1** FlowBalance as a verifier-grounded self-improvement cycle. The current policy generates on-policy reasoning experience (1); a verifier supplies sparse but reliable outcome feedback (2); a frozen privileged-hindsight view of the same policy supplies dense self-guidance on the sampled tokens (3); sign gating grounds that guidance in verified outcomes (4); and profiled trajectory balance internalizes the resulting normalized response distribution (5). Refreshing the frozen snapshot repeats the inner loop.

The core takeaway is that self‑improvement loops are intrinsically fragile: without verifier‑grounded calibration, dense self‑guidance can amplify false confidence and collapse reasoning quality.

Reasoning Experience Framework

Defines on‑policy reasoning experience and privileged‑hindsight guidance that enable FlowBalance.

The preliminaries formalize the on‑policy reasoning experience and the privileged‑hindsight signals that FlowBalance later normalizes.

We treat each reasoning prompt as a fresh on‑policy episode: the current policy generates a batch of candidate answers, which are later evaluated by a verifier.

Sample $y^{(1)}$: the policy draws token “Yes” with probability $0.7$.

Sample $y^{(2)}$: the policy draws token “No” with probability $0.3$.

Freeze the current parameters $\theta^{-}$; the rollout policy $\pi_{\theta^{-}}$ will reproduce the same probabilities for this prompt.

Compute advantages later using verifier scores (see Equation 5).

The on‑policy batch contains both correct and incorrect answers, enabling the later FlowBalance target to re‑weight them based on verifier outcomes.

How does this on‑policy experience differ from standard reinforcement‑learning episodes?

Standard RL episodes usually involve a single trajectory per environment step, whereas here we generate a *group* of complete answer trajectories for the same prompt before any reward is observed. This group‑wise sampling lets FlowBalance compute a normalized advantage across the batch.

Privileged‑Hindsight Self‑Guidance supplies a dense feature by evaluating the same frozen policy with access to a training‑only context $c$ after the answer has been generated.

The FlowBalance Method

FlowBalance turns group outcomes into a normalized policy target that blends verifier signals with dense hindsight guidance.

Training reasoning models on their own outputs is fragile: sparse verifier feedback can be over‑ruled by dense self‑guidance that reinforces wrong predictions.

FlowBalance converts a rollout group of sampled responses into a single normalized target distribution, anchoring dense hindsight scores to the verifier’s outcome direction.

Compute energies: $E(y^{(1)})=0.5\cdot(+1)+0.2\cdot0.4\cdot(+1)=0.5+0.08=0.58$.

For the failure response, the sign flips: $E(y^{(2)})=0.5\cdot(-1)+0.2\cdot0.3\cdot(-1)= -0.5-0.06=-0.56$.

Exponentiate: $\exp(E(y^{(1)})/\tau)=e^{0.58}\approx1.79$, $\exp(E(y^{(2)})/\tau)=e^{-0.56}\approx0.57$.

Assume reference probabilities $\pi_{\text{ref}}(y^{(1)})=0.6$, $\pi_{\text{ref}}(y^{(2)})=0.4$. Unnormalized scores: $\tilde p(y^{(1)})=0.6\times1.79\approx1.07$, $\tilde p(y^{(2)})=0.4\times0.57\approx0.23$.

Partition $Z=1.07+0.23=1.30$. Normalized targets: $p(y^{(1)})=1.07/1.30\approx0.82$, $p(y^{(2)})=0.23/1.30\approx0.18$.

Positive verifier signals amplify dense hindsight scores, while negative signals invert them, preventing a failed trajectory from being reinforced.

How does FlowBalance differ from a standard token‑level imitation loss that directly matches $\pi_H$?

Standard imitation treats the privileged hindsight policy as a ground‑truth target for each token, ignoring the verifier’s binary outcome. FlowBalance instead uses the verifier to flip the sign of the dense hindsight gain when the outcome is negative, and it normalizes the whole distribution so the dense branch cannot dominate the loss independently.

Self‑improvement loop (Algorithm 1)

By fitting the policy to the normalized FlowBalance target, dense hindsight guidance is safely anchored to verifier outcomes, eliminating the self‑reinforcement pathology of naive self‑guidance.

Mechanism and Stability

We explain why FlowBalance enables stable verifier‑grounded self‑improvement.

Section 3 introduced FlowBalance as a partition‑normalized update over a rollout group. Here we examine four properties that make it suitable for a self‑improvement inner loop: (1) the group partition preserves all relative evidence, (2) the target moves conservatively from the reference, (3) the verifier stays an explicit control even with dense self‑guidance, and (4) false‑positive guidance on rejected responses cannot become self‑reinforcement.

FlowBalance updates the policy by blending verifier‑derived advantage with self‑generated guidance, while normalizing across the sampled rollout group to keep the update conservative.

Compute signed guidance: $\operatorname{sgn}(A) = (1,\,-1,\,1)$, so $ \beta_G G_H \operatorname{sgn}(A) = (0.12,\,-0.10,\,0.04)$.

Form combined energy $E_i = \eta_A A_i + \beta_G G_H \operatorname{sgn}(A_i)$ giving $E = (0.5\cdot0.4+0.12,\;0.5\cdot(-0.3)-0.10,\;0.5\cdot0.1+0.04) = (0.32,\;-0.25,\;0.09)$.

Reference probabilities (e.g. from $\pi_{\text{ref}}$) are $\pi_{\text{ref}} = (0.4,\,0.35,\,0.25)$. Compute unnormalized weights $w_i = \pi_{\text{ref}}(y^{(i)})\exp(E_i/\tau)$ yielding $w = (0.4e^{0.32},\,0.35e^{-0.25},\,0.25e^{0.09}) \approx (0.54,\,0.27,\,0.28)$.

Normalize: $p^{\star}_i = w_i / \sum_j w_j$, giving $p^{\star} \approx (0.54/1.09,\,0.27/1.09,\,0.28/1.09) = (0.50,\,0.25,\,0.23)$.

Sign‑gating flips the contribution of $G_H$ on the negative‑advantage response, reducing its target probability while boosting the verified positive response.

How does FlowBalance differ from simply adding a verifier reward term to the loss?

A plain reward term would increase the probability of all high‑scoring responses, including those the verifier rejects. FlowBalance first normalizes across the rollout group, then applies the sign of the verifier advantage to the self‑guidance term, so negative‑advantage responses receive a negative contribution, preventing self‑reinforcement of false positives.

Snapshot the current policy parameters $\theta^{-} \leftarrow \theta$.

Sample a minibatch $\{(x_b,c_b)\}_{b=1}^{B}$ from the dataset $D$.

Generate $N$ responses $y^{(i)}_b \sim \pi_{\theta^{-}}(\cdot \mid x_b)$ for each prompt.

Evaluate verifier rewards and compute stopped group‑relative advantages $A_{b,i}$ (Eq. 5).

Score tokens under the current policy $\pi_{\theta}$, the reference $\pi_{\text{ref}}$, and the frozen hindsight view $\pi_H$.

Compute self‑guidance scores $G_H(y^{(i)}_b \mid x_b,c_b)$.

Profile $\log Z$ for FlowBalance using Eq. (22).

Form the combined energy $E_i$ and target distribution $p^{\star}$, then compute the update $\Delta_{TB}$ (Eq. 18).

Apply the gradient update to $\theta$.

Proposition 1 states that the profiled trajectory‑balance loss is zero exactly when every relative probability contrast matches the exponential of the energy differences, meaning the group partition preserves all $N\!-\!1$ contrast directions while discarding only a common offset.

Proposition 2 shows that, among all distributions attaining at least the expected FlowBalance energy, the target $p^{\star}$ is the unique minimum‑reverse‑KL displacement from the reference, guaranteeing the smallest conservative tilt.

Proposition 3 proves that increasing the verifier coefficient $\eta_A$ monotonically raises the expected verifier reward of the target distribution, so the verifier remains an explicit control knob even when dense self‑guidance is present.

Proposition 4 demonstrates that sign gating flips the effect of positive self‑guidance on a verifier‑rejected response, turning what would be self‑reinforcement into a probability‑ratio correction that favors verified successes.

**Figure 4.** Mechanism view of verifier-grounded self-guidance. FlowBalance uses $+G$ on positive-advantage responses and $-G$ on negative-advantage responses. It retains useful guidance among successful modes without converting positive local confidence on failed responses into self-reinforcement.

**Figure 5. Exact anti-self-confirmation advantage.** The left panel shows target success probability as positive self-guidance on a failed response increases. The right panel plots the closed-form probability-ratio margins in Eq. (69); no fitted decision boundary is used.

**Figure 6. Reliability-strength map for self-guidance.** Left: gain in verified-success mass over reward-only shaping. Middle: conditional Simpson diversity among successful modes; the white contour marks the reward-only level. Right: reverse KL to the reference. The reliability parameter is a synthetic interpolation, not an empirical calibration estimate.

Empirical Evaluation

FlowBalance delivers the highest average reasoning score while speeding up training.

Recall that training reasoning models on their own output is fragile; FlowBalance grounds self‑guidance in verifier outcomes by mapping rollout groups to a normalized target.

FlowBalance achieves the highest average score of $67.61$ on Qwen3‑8B, surpassing all baselines.

Table 1 reports step‑180 results over five seeds; FlowBalance’s average exceeds GRPO by $2.12$ points, OPSD by $26.45$ points, RLSD by $3.49$ points, and FlowRL by $1.76$ points.

**Table 1.** Mathematical reasoning results across Qwen3-4B and Qwen3-8B. AIME24 uses Pass@16; all other benchmarks use Pass@1. Entries are percentages reported as mean $\pm$ sample standard deviation over five seeds at step 180. "Avg." averages the five reported benchmark means.

**Figure 2.** Training dynamics on mathematical reasoning with Qwen3-8B. (a) Training acceleration: FlowBalance reaches 0.5 AIME24 validation accuracy in about 100 steps, compared with roughly 143 steps for GRPO (1.43x faster). (b) Training stability: over 400 training steps, FlowBalance remains near its peak performance, whereas GRPO degrades sharply after approximately step 180. (c) Response length: direct OPSD rapidly collapses to substantially shorter responses, whereas FlowBalance maintains longer reasoning trajectories. In the accuracy panels, solid curves show smoothed trends and lighter curves show the corresponding per-step measurements.

**Table 2.** Ablation over verifier coefficient $\eta_A$.

**Table.** Content (boxed = reasoning actions; “...” = omitted)

**Figure 3.** LLM-judged strategy diversity. Correct-only Simpson diversity on AIME24; protocol and scope are given in Appendix D.3.

The table presents performance metrics across various benchmarks (AIME24@16, HMMT25@1, Minerva@1, MATH500@1, Olympiad@1) for different values of $\beta_G$ (1, 2, and 3), along with an average score for each row.

Related Work

Survey of prior approaches to outcome‑grounded and distributional LLM training.

We review prior methods that provide outcome grounding, privileged self‑guidance, and distributional policy updates for LLM reasoning.

RLVR trains models using sparse binary rewards that can be automatically verified, giving a reliable signal for reasoning tasks.

GRPO estimates advantage by comparing multiple responses to the same prompt, using the group of rollouts as a baseline without a separate critic.

Transfers behavior from a larger teacher model to a smaller student by matching output distributions.

Evaluates student‑generated trajectories under a teacher model, using the teacher’s feedback on the student’s own outputs.

Derives an efficient KL‑regularized target with a tunable reference–teacher trade‑off for self‑distillation.

Stabilizes a Challenger–Solver loop with a small pool of human anchors and an online difficulty‑based curriculum.

Learn stochastic construction policies that sample objects proportionally to an unnormalized reward.

Applies trajectory balance to LLM reasoning with a prompt‑conditioned partition term.

Extends Flow of Reasoning by profiling a reference‑supported distribution over rollout groups.

Replaces the auxiliary partition network with an in‑batch rollout‑group estimate.

Detailed Balance Analysis

Provides a detailed‑balance and trajectory‑balance perspective on FlowBalance.

To interpret FlowBalance we examine two classic equilibrium notions from Markov theory: detailed balance and trajectory balance.

Detailed balance requires that at equilibrium the probability flow from any response $y$ to $y'$ equals the reverse flow, guaranteeing that the Markov kernel $K$ is reversible with respect to the target distribution $q$.

How does detailed balance differ from merely having a stationary distribution?

Stationarity only requires that the distribution be unchanged after one transition, while detailed balance enforces symmetric flow for every pair of states, a stricter condition that guarantees reversibility.

Trajectory balance imposes a global equality between the model’s policy (scaled by the partition function) and the reweighted reference distribution for each complete response, avoiding the need to define pairwise transitions.

Why not use detailed balance directly instead of trajectory balance?

Detailed balance requires an explicit transition kernel over intermediate states, which is infeasible for long language sequences. Trajectory balance replaces the pairwise constraint with a global equality on complete trajectories, enabling tractable learning.

**Figure 7. Conservative change and all-contrast fitting.** Left: FlowBalance is the minimum reverse-KL target at its attained composite-energy level. Right: profiling one nuisance intercept per rollout group preserves all $N-1$ contrast directions and yields the exact local Gaussian risk reduction.

Numerical Diagnostics

Ablation diagnostics quantify how FlowBalance improves success rates and efficiency over baseline guidance.

We evaluate exact numerical diagnostics of the FlowBalance target on enumerable response spaces, isolating how each component affects verified success mass and distributional efficiency.

FlowBalance raises verified success mass to 0.900, compared with 0.818 for reward‑only shaping.

Success mass values reported in the Mechanism View (Section C.1.1).

FlowBalance raises verified success mass to 0.900, compared with 0.832 for ungated self‑guidance.

In the mixed‑outcome group, FlowBalance achieves a success probability of 0.894, versus 0.817 for reward‑only shaping.

Pairwise View results at $G^{-}=0.5$ (Equation (69)).

In the same setting, FlowBalance achieves 0.894 versus 0.807 for ungated shaping.

FlowBalance’s reverse‑KL distance to the reference is 0.973, 3.6× larger than the baseline’s 0.273.

Structural View analysis of the exponential‑tilt path.

Parameter‑risk for FlowBalance is 2.92 % of a one‑contrast‑per‑group estimator, roughly 34× lower.

Structural View calculation for group size $N=32$.

Theoretical Proofs I

Formal proofs of the FlowBalance theoretical results.

We work with the same fixed prompt–context pair and realized rollout group as in Section 4, using the main‑text notation $E_i = \eta A A_i + \beta G G H\bigl(y^{(i)} \mid x, c\bigr)\,\operatorname{sgn}(A_i)$ and $p_i^{\star}= \frac{\pi_{\text{ref}}\bigl(y^{(i)}\mid x\bigr)\,\exp\!\bigl(E_i/\tau\bigr)}{\displaystyle\sum_{j=1}^{N}\pi_{\text{ref}}\bigl(y^{(j)}\mid x\bigr)\,\exp\!\bigl(E_j/\tau\bigr)}$.

When KL is computed over the realized rollout group, $\pi_{\text{ref}}(\,\cdot\,\mid x)$ denotes its restriction and renormalization to that group.

Proposition B.1 (Profiled balance matches all within‑group contrasts).

Define the realized‑group trajectory‑balance loss $L_N(\pi, z)=\frac{1}{2N}\sum_{i=1}^{N}\bigl(\tau z+\tau\log\frac{\pi\bigl(y^{(i)}\bigr)}{\pi_{\text{ref}}\bigl(y^{(i)}\mid x\bigr)}-E_i\bigr)^{2}$.

For fixed $\pi$, $L_N(\pi, z)$ is a convex quadratic in $z$.

The unique profiled intercept is $\hat{z}= \frac{1}{N}\sum_{i=1}^{N}\Bigl(E_i-\tau\log\frac{\pi(y^{(i)})}{\pi_{\text{ref}}(y^{(i)}\mid x)}\Bigr)$.

Zero profiled loss occurs iff every pair $i,j$ satisfies $\tau\log\frac{\pi(y^{(i)})}{\pi(y^{(j)})}-\tau\log\frac{\pi_{\text{ref}}(y^{(i)}\mid x)}{\pi_{\text{ref}}(y^{(j)}\mid x)}=E_i-E_j$.

Exponentiating and rearranging yields $\frac{\pi(y^{(i)})}{\pi(y^{(j)})}= \frac{\pi_{\text{ref}}(y^{(i)}\mid x)}{\pi_{\text{ref}}(y^{(j)}\mid x)}\exp\!\bigl((E_i-E_j)/\tau\bigr)$ for all $i,j\in[N]$.

Conversely, if the above ratio holds for all pairs, the quantities $\frac{\pi(y^{(i)})}{\pi_{\text{ref}}(y^{(i)}\mid x)}$ are equal to a constant, and choosing $z$ as the negative log of that constant divided by $\tau$ makes every residual zero.

Proposition B.2 (Minimum reference displacement).

Recall the definition $p_i^{\star}= \frac{\pi_{\text{ref}}(y^{(i)}\mid x)\exp(E_i/\tau)}{\sum_{j=1}^{N}\pi_{\text{ref}}(y^{(j)}\mid x)\exp(E_j/\tau)}$.

For any $p\in\Delta_N$, $\mathrm{KL}(p\parallel\pi_{\text{ref}})=\sum_{i}p_i\log\frac{p_i}{\pi_{\text{ref}}(y^{(i)}\mid x)}$.

Decompose $\mathrm{KL}(p\parallel\pi_{\text{ref}})=\mathrm{KL}(p\parallel p^{\star})+\sum_{i}p_i\log\frac{p^{\star}_i}{\pi_{\text{ref}}(y^{(i)}\mid x)}$.

From the definition of $p^{\star}$, $\log\frac{p^{\star}_i}{\pi_{\text{ref}}(y^{(i)}\mid x)}= \frac{E_i}{\tau}-\log\!\Bigl(\sum_{j}\pi_{\text{ref}}(y^{(j)}\mid x)\exp(E_j/\tau)\Bigr)$.

Summing $p_i$ times the expression above gives $\sum_{i}p_i\log\frac{p^{\star}_i}{\pi_{\text{ref}}(y^{(i)}\mid x)}= \frac{1}{\tau}\sum_{i}p_i E_i-\log\!\Bigl(\sum_{j}\pi_{\text{ref}}(y^{(j)}\mid x)\exp(E_j/\tau)\Bigr)$.

Similarly, $\mathrm{KL}(p^{\star}\parallel\pi_{\text{ref}})=\frac{1}{\tau}\sum_{i}p^{\star}_i E_i-\log\!\Bigl(\sum_{j}\pi_{\text{ref}}(y^{(j)}\mid x)\exp(E_j/\tau)\Bigr)$.

Combining the pieces yields the decomposition $\mathrm{KL}(p\parallel\pi_{\text{ref}})=\mathrm{KL}(p^{\star}\parallel\pi_{\text{ref}})+\mathrm{KL}(p\parallel p^{\star})+\frac{1}{\tau}\sum_{i}p_i E_i-\frac{1}{\tau}\sum_{i}p^{\star}_i E_i$.

Among all $p$ with $\sum_i p_i E_i \ge \sum_i p^{\star}_i E_i$, the term $\frac{1}{\tau}\sum_i p_i E_i$ is minimized by $p=p^{\star}$, and $\mathrm{KL}(p\parallel p^{\star})\ge 0$ with equality only at $p=p^{\star}$.

Theoretical Proofs II

Formal proofs of FlowBalance’s verifier weighting and sign‑gating properties.

Proposition B.3 states that increasing the verifier coefficient $\eta_A$ monotonically raises the expected reward under the FlowBalance target.

Only the term $\eta_A A_i$ in $p^{\star}_i$ depends on $\eta_A$.

The derivative of the expected reward equals a covariance term.

For the GRPO‑normalized advantage, the covariance reduces to a variance factor.

Substituting the covariance yields the monotonicity result.

Proposition B.4 shows that sign gating flips the probability ratio between a verified success and a verifier‑rejected response, correcting self‑guidance pressure.

Start from the gated target ratio expression.

Write the analogous ungated ratio.

Subtract the two exponents to isolate the gating effect.

Exponentiating the difference yields the ratio correction.

Finally, because KL divergence $\mathrm{KL}(p\parallel p^{\star})$ is non‑negative and zero only when $p=p^{\star}$, $p^{\star}$ is the unique minimum‑reverse‑KL displacement from the reference among all admissible distributions.

Experimental Protocols

Details the experimental setup, ablations, and extra case studies for FlowBalance.

Section D.1 defines the experimental protocol used to compare policy‑update objectives. All methods share identical prompts, rollout‑group size, response length cap, verifier, optimizer schedule, checkpoint cadence, and evaluation script, ensuring a fair comparison.

**Table 1.** Content (boxed = reasoning actions; “...” = omitted)

**Table 7** Full ablation over self-guidance coefficient $\beta_G$ on Qwen3-8B at step 180. The verifier coefficient is held fixed at the default value $\eta_A = 15$. Entries are percentages reported as mean $\pm$ sample standard deviation over five seeds. "Avg." averages the five benchmark means.

Baseline implementations reuse the same sampled responses, answer verifier, and response‑length cap. GRPO optimizes a standard reward‑policy objective; OPSD performs clipped forward‑KL self‑distillation; RLSD combines verifier and teacher signals; FlowRL follows the original outcome‑only trajectory‑balance algorithm.

Evaluation uses Pass@16 for AIME24 (any of 16 samples correct) and Pass@1 for HMMT25, Minerva, MATH500, and OlympiadBench. All reported numbers are means ± standard deviation over five random seeds at step 180.

Section D.2 expands the coefficient sweeps introduced in the main text. The $\eta_{A}$ sweep varies the verifier coefficient while fixing $\beta_{G}$; the $\beta_{G}$ sweep varies the self‑guidance coefficient while fixing $\eta_{A}$. Each point is evaluated with the same five‑benchmark average.

Section D.3 introduces a semantic diversity diagnostic that clusters full reasoning trajectories using a GPT‑5.5 judge. The protocol extracts a strategy summary per trajectory and then clusters anonymized summaries within each problem‑method pair.

The reported metric is the correct‑only Simpson diversity $D_{\text{Simpson}} = 1 - \sum_k p_k^{2}$, measuring the probability that two randomly chosen correct trajectories use different strategies.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers