Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimization

Sihan Ge, Yichen Lin, Chenyu Zhou, Jianghao Lin, Tao Yao, Dongdong Ge

InterOPT improves operations research modeling by explicitly tracking formulation-critical gaps before generating code.

How can LLMs systematically identify and resolve missing information in natural-language optimization requests before attempting to generate solver code?

Large language models often generate optimization models from incomplete business requests, leading to silent assumptions that fundamentally alter the problem's mathematical structure. InterOPT addresses this by separating the process into two stages: a Dynamic Gap Search that maintains a persistent ledger of unresolved business requirements, and a Gap-Guided Action Search that anchors clarification questions to those specific gaps. In choice-based settings, this framework significantly outperforms baselines in recovering missing requirements, though it remains competitive with strong prior methods in open-ended dialogue.

Paper Primer

The core challenge in operations research (OR) assistance is "premature formulation": an agent declares readiness while missing critical constraints or objectives, such as whether a vehicle route must return to its origin. InterOPT treats this as a monitoring-control problem, where the agent must distinguish between what is known, what is uncertain, and whether the current state is sufficient to build a valid model.

InterOPT functions as a two-stage loop: Dynamic Gap Search identifies unresolved formulation-critical gaps and records them in a ledger, while Gap-Guided Action Search uses this ledger to generate targeted questions. The method is a "ledger-to-action" pipeline: it forces the agent to bind every clarification question to a specific open gap, ensuring that interaction is driven by diagnostic necessity rather than generic conversational flow.

InterOPT achieves superior recovery of formulation-critical requirements in choice-based interaction protocols.

In choice-based experiments, InterOPT consistently outperforms baselines in exact slot recovery, converting unresolved business requirements into fully restored model parameters. InterOPT improves Core Exact recovery by 11.1 percentage points over the strongest choice-based baseline in ablation studies.

While InterOPT improves recovery, it does not eliminate silent assumptions or premature stopping. In open-ended settings, the agent still declares readiness prematurely in approximately 40% of runs, indicating that while the framework improves gap discovery, calibrating the final "ready to model" decision remains an open challenge.

Why is this a distinct problem from standard conversational search or intent elicitation?

In OR, a missing fact is not just a search failure; it is a structural error that changes the objective function or constraint set. Unlike general intent elicitation, a question here is only successful if it recovers a specific business fact that alters the mathematical formulation.

What is the "information boundary" enforced by the OR-Clarify benchmark?

The benchmark withholds specific "hidden slots" (formulation-critical facts) from the agent's initial brief. The agent is evaluated on its ability to recover these specific slots through interaction, with the judge using a frozen rubric that ignores vague questions or assumptions made without explicit confirmation.

This paper shifts the evaluation of OR agents from "can it solve this model?" to "does it know when it has enough information to build the model?"

For researchers building OR agents, this work demonstrates that explicit gap-tracking ledgers are more effective for requirement recovery than end-to-end conversational policies, particularly when the interaction space is constrained.

The Problem of Silent Assumptions

We expose how incomplete natural‑language requests cause silent assumptions and propose a gap‑focused solution.

LLMs are increasingly used to turn natural‑language problem descriptions into optimization models, but real‑world OR requests are frequently incomplete. Missing objectives, constraints, or business rules constitute formulation gaps that lead the model to make silent, unverified assumptions. Our benchmark OR‑Clarify isolates this pre‑formulation stage, and the InterOPT framework tackles it by explicitly identifying and clarifying such gaps before code generation.

A missing piece of the problem description that, if left unspecified, can change the very structure of the resulting optimization model.

Unverified defaults that an LLM silently inserts when a required piece of information is missing.

**Figure 1.** Motivation for pre-formulation clarification.

The essential gap lies between a user's natural‑language intent and the formal optimization requirements that a model must satisfy.

Related Work and Context

Survey prior LLM optimization work and define the pre‑formulation clarification task.

Prior studies have examined whether large language models can translate natural‑language problem descriptions into optimization models, evaluating model generation, solver integration, and domain adaptation.

Clarification and elicitation research focuses on agents asking questions to resolve underspecified user intent, but these works target search or QA scenarios rather than the business‑level formulation gaps relevant here.

The task is to decide whether a business brief contains enough detail to build a well‑defined optimization model and, if not, to ask targeted questions that recover the missing formulation‑critical facts.

The OR-Clarify Benchmark

Introducing a benchmark that quantifies formulation gaps via hidden slots.

The OR‑Clarify benchmark comprises 100 clarification cases covering 178 hidden slots.

Construction yields 100 cases with a mean of 1.78 hidden slots per case (75 P0, 83 P1, 20 P2).

A benchmark that turns formulation‑relevant uncertainty into explicit hidden‑slot targets, enabling controlled evaluation of an agent’s clarification ability.

**Figure 2:** Evaluation workflow and information flow. Hidden slots never enter the evaluated agent. The simulator supplies controlled answers, the passive monitor records protocol behavior, and the post-hoc judge scores the completed public transcript against frozen hidden-slot rubrics.

Compared to prior evaluation setups, OR‑Clarify isolates the information boundary, allowing precise measurement of exact recovery, readiness behavior, and interaction efficiency under both open and Choice interaction modes.

The InterOPT Framework

InterOPT clarifies missing formulation gaps through a two‑stage loop before optimization.

LLMs often skip hidden assumptions, so we need a systematic way to surface missing business conditions before code generation.

InterOPT turns the vague “is the model ready?” question into a concrete two‑stage loop that first records what is unknown and then asks a targeted question about one of those unknowns.

How is InterOPT different from a single‑step “ask‑once” clarification?

InterOPT separates diagnosis (Stage 1) from question generation (Stage 2), guaranteeing that every asked question is tied to a concrete, previously recorded gap rather than being guessed on the fly.

Stage 1 repeatedly asks “what do we still not know?” and adds any missing business condition to a persistent ledger.

$\Phi$ scans $b$ and $\tau_t$, finds no deadline, and creates a gap entry G001 with category “Time boundary” and status Open.

The ledger $M_t$ now holds $\{ \text{G001} \}$.

$O_t$ is computed as $\{ \text{G001} \}$ because the entry is Open.

Dynamic Gap Search turns an implicit missing deadline into an explicit, queryable entry, preventing the model from silently assuming a default.

Stage 2 takes the open gaps and asks the model to propose concrete clarification questions anchored to one of those gaps.

Why does the selector prioritize objective changes over pure notation questions?

Because an unresolved objective can completely reshape the optimization model, whereas a notation tweak only affects presentation; the former has higher downstream impact.

InterOPT Clarification Loop

InterOPT guarantees that every clarification question is tied to a concrete, previously identified gap.

Experimental Setup

Key experimental scale: 100 benchmark cases evaluated with five runs each.

The benchmark comprises 100 OR‑Clarify cases.

All main experiments evaluate on this fixed set of cases.

All experiments use DeepSeek V4 Pro as the tested model with $T_{\max}=20$, an agent temperature of $0.2$, and zero temperature for the simulator, judge, and selector. For InterOPT, Stage 1 adds at most three new gaps per turn, and Stage 2 instantiates three candidate states and questions on each asking turn.

Runs that fail the post‑hoc protocol audit remain in the denominator of headline metrics and are retained for diagnostic analysis, ensuring that failure cases are not discarded.

For methods with internal planning roles, all agent‑facing decisions—gap discovery, ask/ready decision, candidate generation, and candidate selection—are instantiated with the tested agent model, while the simulated user, protocol detector, and post‑hoc judge are fixed evaluation components that never expose hidden slots.

The Choice‑Based Setting evaluates four methods (MC, MC‑D, ReadyGate, InterOPT) with $K=5$ runs per case, using option‑based clarification where the simulated user selects among generated options A–C or provides a free‑form correction via option D.

The Open‑Ended Setting compares the methods reported in Table 2 using the FreeQA interface, where each action is a natural‑language question or `READY_TO_MODEL` and all transcripts are scored by the same judge.

Results and Ablation Analysis

We evaluate how each InterOPT component affects exact recovery and interaction cost.

InterOPT aims to close formulation gaps before code generation; this section measures how each component contributes to exact recovery and interaction cost.

ReadyGate is a readiness reviewer that checks for unresolved formulation gaps before the agent signals completion.

How does ReadyGate differ from a simple stop condition?

A simple stop condition halts as soon as the model signals confidence, even if hidden formulation gaps remain. ReadyGate, by contrast, inspects a persistent ledger of unanswered gaps and only permits stopping when that ledger is empty, explicitly eliminating silent assumptions.

We run ablations that remove each stage of InterOPT to see how performance changes across the Open and Choice protocols.

Removing Stage 1 drops Core Exact from 0.638 to 0.520, a 0.118 decrease in the Open/FreeQA setting.

Table 3, Open / FreeQA, “w/o Stage 1”.

Removing Stage 2 reduces Core Exact to 0.572 while nearly doubling the number of turns to 9.5 in the Open/FreeQA setting.

Table 3, Open / FreeQA, “w/o Stage 2”.

In the Choice (MC‑D) protocol, omitting Stage 1 lowers Core Exact from 0.638 to 0.612, a 0.026 drop.

Table 3, Choice, “w/o Stage 1”.

Omitting Stage 2 in Choice reduces Core Exact to 0.462 and inflates interaction to over 9.5 turns.

Table 3, Choice, “w/o Stage 2”.

ReadyGate alone attains Core Exact 0.474 but requires 6.476 average turns, illustrating a trade‑off between recovery and interaction cost.

Table 2, Open / FreeQA, “ReadyGate”.

Full InterOPT reaches Core Exact 0.638 with 5.490 average turns, balancing high recovery with moderate interaction.

Table 3, Open / FreeQA, “InterOPT”.

**Figure 4.** Exact restoration versus cumulative atomic questions under Open and Choice protocols. Top: Open; bottom: Choice. Left: All-Slot Exact; right: Core Exact. Curves average K = 5 runs, with 95% case-level bootstrap confidence intervals. Methods are compared only within the same response protocol.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers