LLMs Get Lost in Evolving User Intent

Jihoon Tack, Philippe Laban, Jennifer Neville

LLMs struggle to track and act on user intent when it evolves through revisions and task switches during multi-turn conversations.

Do LLMs maintain accuracy when user intent shifts dynamically during a multi-turn conversation, or does performance degrade compared to static, single-turn benchmarks?

Current LLM benchmarks evaluate agents on static, fully-specified tasks, failing to capture how real users disclose, revise, and redirect their goals during long-horizon interactions. The authors introduce a framework that transforms static, verifiable single-turn datasets into dynamic multi-turn conversations by retrospectively synthesizing preceding turns that lead to the original task as a final anchor. Across multiple domains, even frontier models show substantial performance degradation—up to 30%—when intent evolves, revealing a fundamental gap in how agents maintain and update their internal belief states.

Paper Primer

The framework models intent evolution through three controllable dynamics: argument reveal (disclosing missing info), argument revision (changing previously stated values), and function switch (pivoting to a related task). By anchoring the final turn to a verifiable single-turn problem, the system preserves ground-truth evaluation without requiring new human annotations.

Strong static-setting performance does not transfer to evolving-intent scenarios.

Across four domains (math, SQL, search, software engineering), models consistently show significant accuracy drops when moving from single-turn to multi-turn interactions with six intent transitions. GPT-5.5 accuracy on math problems drops from 99.0% to 80.5% after six intent transitions.

Function switching is the most challenging transition type for LLM agents.

Ablation studies show that function switches cause steeper performance declines than argument reveals or revisions, as they require the agent to discard prior context rather than simply accumulate it. Performance degradation is most pronounced when function switches are combined with other transitions, often leading to near-zero accuracy in complex scenarios.

Why is this framework necessary if multi-turn benchmarks already exist?

Existing multi-turn benchmarks often rely on unreliable LLM-as-judge protocols and lack the scalability and verifiable ground truth provided by static, single-turn datasets.

Does providing more tool-call opportunities help agents adapt to evolving intent?

No; in domains like software engineering, agents often exhaust their tool-call budgets on exploration rather than execution, suggesting that accumulated interaction context can act as a distractor rather than a benefit.

The Challenge of Evolving Intent

LLMs struggle to track evolving user intent, revealing a gap between static benchmarks and dynamic interaction.

Current evaluations treat LLMs as static responders, but real users continuously reshape their goals as conversations progress.

Users rarely present a complete goal up front; instead they reveal, revise, and sometimes redirect their objectives over multiple turns.

We therefore propose a framework that retrofits any verifiable single‑turn dataset into a multi‑turn setting, preserving the original answer as the final‑turn target while synthesizing preceding turns that exhibit disclosure, revision, or task switching.

**Figure 1.** **LLMs get lost in evolving user intent.** Utilizing a single-turn instance (e.g., SWE-Bench; Jimenez et al., 2024), we simulate a multi-turn conversation with evolving user intents while preserving the original ground-truth evaluation. Even frontier models degrade substantially after only 6 intent transitions.

The gap between static benchmarks and dynamic agentic interaction reveals a critical blind spot in current LLM evaluation.

Related Work

We survey multi‑turn evaluation and introduce a structured model of evolving user intent.

Recent work highlights that LLM agents are increasingly used in multi‑turn dialogs, which expands the evaluation space from a single query to a combinatorial set of interaction trajectories. Building benchmarks for this setting is costly because each new task distribution requires fresh conversations, and static dialogues provide little control over user behavior, making systematic probing of interaction patterns difficult.

Multi‑turn evaluation measures how well an LLM agent performs across a sequence of exchanges rather than a single, fully‑specified query.

Simulated users are synthetic agents that generate interaction data, allowing researchers to evaluate LLM agents without costly human annotation.

We treat user intent as a structured state that can be explicitly manipulated at each turn, giving precise control over what information the user has revealed.

Turn 1: No arguments revealed, so $C^{\text{rev}}_1 = \emptyset$.

Turn 2: User reveals the city, updating $C^{\text{rev}}_2 = \{\text{city}\}$.

Turn 3: User revises the cuisine from “Italian” to “Sushi”, yielding $C_3 = \{\text{city}= \text{NY},\; \text{cuisine}= \text{Sushi}\}$ while $C^{\text{rev}}_3$ still contains only the city.

Turn 4: User switches to a booking task, $f_4 = \text{book\_restaurant}$, keeping the city argument shared across tasks.

This toy trace shows how the revealed subset $C^{\text{rev}}_t$ can lag behind the full argument set $C_t$, forcing the agent to continually revise its belief.

Across turns, user intent can evolve via three distinct transition types, each altering the hidden state in a different way.

The next section describes the simulation framework that operationalizes these intent dynamics, enabling quantitative measurement of how well agents cope with evolving user goals.

The Simulation Framework

We describe a pipeline that turns single‑turn data into verifiable multi‑turn conversations.

To evaluate LLMs on evolving user intent we build a simulation pipeline that turns single‑turn data into multi‑turn dialogues.

User intent can change during a conversation via three mechanisms: adding a new constraint (argument reveal), revising an existing constraint (argument revision), or switching to a different task (function switch).

How does this differ from a static intent representation?

Static intent fixes the function and arguments for the entire dialogue, while Intent Transition Dynamics records the incremental updates of function, arguments, and revealed constraints at each turn, enabling the agent to see exactly what new information appears.

The framework stitches together three stages—anchor extraction, retrospective expansion, and situated simulation—to generate verifiable multi‑turn conversations from a single‑turn example.

Why not simply concatenate generated turns without the backward chaining?

Without the predecessor chain the generated history would not be guaranteed to lead to the original verified intent; the backward chaining ensures each intermediate turn is a plausible sub‑task that logically culminates in the anchor, preserving verifiability.

Extract the anchor intent $(f^*, C^*)$ from a single‑turn example using a language model.

For each argument $c_i^*$ generate a counterfactual $c_i^{\text{ccf}}$ and a predecessor function $f_{\text{pre}}$ with overlapping arguments.

Recursively apply the predecessor generation to obtain a backward chain of desired depth.

Schedule intent‑transition events (reveals, revisions, switches) across $T$ turns while obeying the five consistency rules.

Render each user turn from the incremental update $\Delta I_t$ using discourse prefixes and optionally an LLM‑based re‑phraser.

**Figure 2.** Illustration of three types of intent state transitions: argument reveal, argument revision, and function switch. All three start from $I_{t-1} : f_{t-1} = \text{search\_restaurant}, c_1 = \text{New York}$, and at turn $t$ either disclose $c_2 = \text{vegan}$, update $c_1 arrow \text{Brooklyn}$, or switch $f_t$ to $\text{book\_table}$ with shared arguments carried over.

**Figure 3.** Evolving-intent conversations from single-turn data. Given a verifiable single turn data, we extract its intent and place it as the final turn anchor. We then synthesize a plausible history by generating counterfactual arguments and expand functions that can would led to this anchor. Finally, we perform a situated simulation so that the conversation ends with the anchor turn. Since the final turn coincides with the source intent by construction, the dataset's original verifier scores the agent's last action without any new annotation.

Experimental Setup

We evaluate how LLMs fare on evolving‑intent multi‑turn tasks and quantify the degradation.

We address four empirical questions: (1) does single‑turn performance transfer to evolving‑intent multi‑turn scenarios, (2) how do transition type, count, composition, and order affect results, (3) can mitigation strategies recover performance, and (4) how does the difficulty of the underlying single‑turn task influence outcomes.

LLMs lose substantial accuracy when moving from static single‑turn benchmarks to evolving‑intent multi‑turn dialogues.

Table 1 reports relative drops ranging from 4 % to 31 % across models and datasets.

We evaluate four verifiable single‑turn datasets—GSM8K (math), BIRD‑SQL (text‑to‑SQL), BrowseComp+ (agentic search), and SWE‑Bench Verified (software engineering)—using domain‑specific prompts to extract intents and generate counterfactual arguments with GPT 5.1.

For SWE‑Bench, each policy is wrapped in a mini‑SWE‑agent v2 scaffold; we intercept submissions and inject scripted user turns until exhaustion, using a per‑turn tool‑call budget of 100 (raised to 200 for Kimi K2.6 and DeepSeek V3.2). For BrowseComp+, we cap per‑turn search calls at 50.

The evolving‑intent scenario combines three transition types—reveal, revision, and switch—each occurring twice within a conversation, yielding up to seven turns (six intent transitions) for tractable experimentation.

**Table 6.** **RL with evolving intent.** Performance of Qwen3-4B before and after training on evolving-intent examples constructed using our framework. We train with GRPO on the transformed GSM8K examples using standard outcome reward. Training improves evolving-intent performance while preserving single-turn performance.

Main Results

Evolving intent causes substantial accuracy drops across models and datasets.

We evaluate nine LLMs on four benchmark suites, measuring how accuracy degrades as user intent evolves through a sequence of six simulated transitions.

LLMs lose up to ≈30 % accuracy when user intent evolves across turns.

Table 1 reports relative drops reaching roughly 30 % (e.g., DeepSeek V3.2 on BIRD‑SQL).

**Table 1.** LLMs get lost in evolving user intent. Accuracy (%) across four datasets, comparing fully specified single-turn interactions (Single) with evolving-intent interactions (Evolve) that compose argument reveal, argument revise, and function switch dynamics. Evolve applies each transition type twice, yielding six intent transitions per dialogue (seven turns including the initial turn). Parentheses report the relative change from Single, and background color indicates the degradation from each model's Single accuracy (saturating at a 50% relative drop).

Ablation Study

We dissect how different intent transitions and their compositions affect model accuracy.

We now probe the impact of individual intent‑transition types and their compositions on four LLMs evaluated on GSM8K.

**Figure 4.** Scaling intent transitions monotonically degrades accuracy. Accuracy of GPT 5.1, 5.2, 5.5, and Kimi K2.6 on GSM8K, where the number of intent transitions of each type (i.e., argument reveal, argument revision, and function switch) is varied independently per conversation.

Increasing the number of function‑switch transitions reduces accuracy markedly more than adding argument‑reveal or argument‑revision steps.

Figure 4 shows GPT 5.1 dropping ≈15 % (98 %→83 %) as function‑switch count rises from 0 to 3, whereas argument‑reveal causes only a ~5 % drop.

Next we examine how combining multiple transition types influences tracking performance.

Composing diverse intent transitions leads to severe accuracy loss, with function‑switch compositions driving performance to near‑zero on challenging benchmarks.

Table 2 reports GPT 5.1 achieving 0 % accuracy on SWE‑Bench verification when any function‑switch or mixed “revise + switch” scenario is present.

Formalizing Intent Tracking

Details of the POMDP‑based simulation, models, datasets, and sampling protocol.

We cast the user‑agent interaction as a Partially Observable Markov Decision Process (POMDP) that captures the hidden user intent and the observable dialogue.

The user’s hidden goal $I_t$ evolves turn by turn, while the LLM only sees the generated utterance $u_t$ and the prior conversation history $h_{t-1}$.

**Figure 8.** The user-LLM agent collaboration loop as a POMDP. At user turn $t$, the user text renderer $\mathcal{U}$ observes the latent intent state $\mathcal{I}_t$ and the interaction history $h_{t-1}$, and renders a user utterance $u_t$. In our implementation, this utterance is conditioned on the updated function and the newly revealed or revised conditions. The LLM agent $\mathcal{M}$ observes only $o_t = (u_t, h_{t-1})$ and must infer the user's latent intent from the conversation. The reward $R_t$ measures alignment between the agent action $a_t$ and the latent answer $y_t$, but our benchmark only evaluates the final-turn reward so that each trajectory remains automatically verifiable against the original single-turn answer.

Instantiate the POMDP with a chosen benchmark domain (e.g., GSM8K).

For each instance, extract the initial intent $I_0$ and its function/arguments.

Sample a turn‑level update $\Delta I_t$ (Argument Reveal, Argument Revision, or Function Switch).

Generate the user utterance $u_t$ using the rule‑based renderer $U$ conditioned on $\Delta I_t$ and the current history $h_{t-1}$.

Feed observation $o_t = (u_t, h_{t-1})$ to the LLM agent $M$ and obtain action $a_t$.

Update the interaction history $h_t = (h_{t-1}, u_t, a_t)$ and transition the intent to $I_{t+1}$.

Repeat steps 3–6 for the desired number of turns (typically 7).

At the final turn, compute reward $R_t$ by comparing the agent’s answer $a_t$ to the latent target $y_t$.

Turn 1: $u_1$ = “Compute $3 + 5 + 2$.” Agent observes $o_1 = (u_1, h_0)$ and replies $a_1 = 10$ (the sum of the three numbers).

Turn 2: $\Delta I_2$ revises the second addend from $5$ to $6$. $u_2$ = “Now compute $3 + 6 + 2$.” Agent observes $o_2 = (u_2, h_1)$ and replies $a_2 = 11$.

Turn 3: $\Delta I_3$ switches the function to product. $u_3$ = “Instead, multiply $3$, $6$, and $2$.” Agent observes $o_3 = (u_3, h_2)$ and replies $a_3 = 36$.

Final reward $R_3$ compares $a_3 = 36$ to the latent target $y_3 = 36$; the agent receives a perfect score.

This toy run shows how each $\Delta I_t$ reshapes the task: the LLM must reinterpret the same numbers under a new function, illustrating the challenge of tracking evolving intent.

Across all domains we apply the same pipeline, varying only the underlying benchmark and the sampled $\Delta I_t$ sequences, while keeping the renderer, history length, and final‑turn evaluation constant.

Additional Experiments

Additional experiments probe reasoning, response length, capacity, turn count, and RL fine‑tuning under evolving intent.

Reasoning mode yields negligible performance gains under evolving intent.

Table 4 shows at most a 2 % increase on BIRD‑SQL and a slight drop on GSM8K.

**Figure 9: Per-turn response length.** Average response length of open-source models across turns on BIRD-SQL. Later turns require models to spend more tokens reasoning over accumulated context.

Smaller model capacity exacerbates degradation under evolving intent.

GPT‑5.4 nano loses accuracy compared to larger variants (Figure 12).

**Figure 10: Effect of model capacity.** Accuracy of the GPT 5.4 model family on GSM8K under single-turn and evolving-intent settings. Smaller models degrade more under evolving intent.

Increasing turn count without intent change does not affect accuracy.

Turn‑matched controls keep performance comparable to the original four‑turn setting.

RL fine‑tuning improves evolving‑intent accuracy from 64 % to 76 %.

Training Qwen3‑4B with GRPO raises performance while preserving single‑turn results.

Additional Analysis and Discussion

We analyze context, memory, and data difficulty effects on LLMs under evolving intent.

The central premise—that real‑world tasks involve evolving user intent—remains our focus. Here we examine how context, memory, and problem difficulty influence LLM performance under such dynamics.

Performance drops substantially when additional intent transitions follow a function switch.

Figure 5 shows accuracy after immediate post‑switch evaluation versus after further reveal or revision transitions, revealing a ≈12 % decline.

The source accuracy of the two groups is comparable; the degradation stems from difficulty integrating pre‑switch context with later updates.

**Figure 5.** LLMs gradually shift away from earlier context after function switches. Analysis on GSM8K across four models under evolving-intent interactions, comparing cases evaluated immediately after a function switch with those evaluated after subsequent reveal or revision transitions. Performance drops after intervening transitions, suggesting that agents struggle to jointly integrate pre-switch context with post-switch updates.

Memory mechanisms improve accuracy but do not close the gap to single‑turn performance.

Figure 6 shows oracle recap raises accuracy from 65 % to 75 % under function switches, yet remains below the 80 % single‑turn baseline.

Both prompt recap and oracle recap yield gains, but even the strongest memory aid falls short of the single‑turn benchmark.

**Figure 6.** Memory mechanisms help but do not close the gap. Accuracy of GPT 5.5 on BIRD-SQL across the evolving-intent scenarios, comparing the default agent against two memory mechanisms: prompt recap, which prepends a reminder at every turn, and oracle recap, which re-states the full revealed intent at each turn. The dashed line indicates single-turn accuracy.

Source data difficulty amplifies multi‑turn degradation.

Figure 7 shows a 9 % accuracy drop in the multi‑turn setting versus a 3.6 % drop in single‑turn when moving from easy to hard hints.

The same models and tasks are used across conditions; the larger drop reflects heightened sensitivity to difficulty under evolving intent.

**Figure 7.** Multi-turn interaction compounds problem difficulty. Accuracy of GPT 5.5 on BIRD-SQL, where the same queries are simulated under single-turn and evolving intent scenario. Difficulty is controlled by providing or withholding the external-knowledge hint on otherwise identical questions. Values next to the arrow indicate the relative performance drop.

Table 3 reports turn‑wise intent‑tracking accuracy: near‑perfect for argument reveals and changes, but a notable drop for function switches, especially with multiple occurrences.

Conclusion and Limitations

We recap that static benchmarks miss evolving user intent, and our simulation framework quantifies the resulting performance drop.

We studied whether LLM agents can faithfully track evolving user intent and introduced a simulation framework that turns any verifiable single‑turn benchmark into a controllable multi‑turn environment using three transitions—argument reveal, revision, and function switch—while preserving the original verifier.

Across diverse domains, both frontier and open‑source models degrade substantially once intent becomes dynamic, exposing a gap that static, single‑turn evaluations miss and that is crucial for future collaborative agents.

Our framework, however, does not model finer‑grained user variation such as persona, communication style, typos, or grammatical errors, leading to rendered utterances that are stylistically uniform compared to real users.

A natural extension is to make the pipeline persona‑conditioned so that counterfactual arguments, predecessor functions, and utterances reflect diverse user backgrounds and communication patterns.

We also assume each turn contains a single intent transition to keep dynamics controllable and the final verifier well defined; in practice users may revise part of a request while switching tasks or ask multiple related questions in one turn.

Extending the framework to support multiple intents and multiple verifiable targets would broaden the range of conversational behaviors captured by the benchmark.

Simulation Implementation Details

How abstract intent trajectories become concrete multi‑turn user messages.

The simulation pipeline converts an abstract intent trajectory into concrete user messages across multiple turns.

Turn scheduling selects a chain of $g$ predecessor functions and $p$ counterfactual arguments, reserves slots for function‑switch and argument‑revision events, then fills remaining slots with argument reveals, guaranteeing that revisions never precede their perturbed values.

Two rendering backends produce the surface utterances: a deterministic rule‑based renderer that concatenates structured updates with sampled prefixes, and an LLM‑based naturalizer that rewrites those turns into more conversational messages while preserving critical tokens via validation.

**Rule-based turn:** Wait, I need to correct a few things. A regular hexagon can be divided into eight equilateral triangles. The perimeter of one of the equilateral triangles is 21 inches. **Naturalized turn:** Sorry, ignore the last bit. I had two numbers swapped: it is actually eight equilateral triangles, and the perimeter of each one is 21 inches.

**Naturalized turn:** Sorry, ignore the last bit. I had two numbers swapped: it is actually eight equilateral triangles, and the perimeter of each one is 21 inches.

We illustrate two memory‑enhancement techniques appended after the initial turn: a cheap Prompt recap that asks the model to re‑derive context, and an Oracle recap that explicitly retrieves the current intent and filters unrevealed arguments.

**User turn with PROMPT recap:** Hold on, instead of listing them, I want the count. Before answering, first list all the conditions from our conversation that apply to this question, then provide your answer.

**Oracle recap.** At each turn, the simulator identifies the active function, selects the arguments relevant to that function, removes arguments that have not yet been revealed, and substitutes the latest active value for any revised argument. **User turn without recap:** Hold on, instead of listing them, I want the count. **User turn with ORACLE recap:** Hold on, instead of listing them, I want the count. Before answering, recall that the goal is: "Among the account opened, how many customers are there?" The following conditions apply: the customers should be female; their year of birth should be before 1950; the district is Sokolov.

Verification of Simulation Components

Verification ensures each generated component preserves the final turn anchor before acceptance.

The simulator builds multi‑turn dialogues from three LLM‑generated components—Extract, Counterfact, and Predecessor—so we first verify each component before accepting a sample.

Extract must preserve the anchor intent; we reject any sample that omits required information (coverage) or yields a different answer than the original question (solvability).

Counterfact is accepted only when it replaces a single localized value, keeps the surrounding argument structure, and respects a bounded length‑ratio constraint, preventing additive or deletive rewrites.

Predecessor must define a plausibly related earlier task, avoid duplicating the final function, and introduce arguments that do not alter the answer to any downstream turn, including cross‑turn independence checks.

All verification steps employ GPT‑5.1 in reasoning‑medium mode, using an LLM judge to perform the rejection sampling described above.

Prompt Templates

Prompt templates for extraction and counterfactual argument synthesis illustrate the LLM pipeline stages.

We expose the concrete prompts that drive each stage of the LLM‑based pipeline. The first stage extracts a structured goal and constraints, the second synthesizes a predecessor function, and the third generates a perturbed counterfactual condition.

Extraction prompt (BrowseComp+ search)

Applying the prompt to a sample BrowseComp query yields a function name and three arguments that capture the goal and each constraint.

Example extraction output for a historic‑site query

The next stage asks the LLM to produce a perturbed version of a target condition, swapping exactly one factual value while preserving the sentence structure.

Counterfactual argument synthesis prompt

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers