Whale: A Simple Recipe for Joint Harness–Weight Optimization

Haechan Kim, Yoonho Lee, Gisang Lee, Chelsea Finn, Kangwook Lee

WHALE alternates model weight updates with executable harness search to solve the coupled bottleneck of agentic systems.

How can we jointly optimize both model weights and the executable harness code that controls agent behavior, rather than tuning them in isolation?

Agent performance is bottlenecked by the interplay between model weights and the harness code that manages tools, context, and control flow. Optimizing one component while the other remains fixed leads to over-optimization against a moving target. Weight-Harness Alternating LEarning (WHALE) decomposes this coupling by alternating between online rejection-sampling fine-tuning of model weights and iterative source-code search for the harness. Across search, math, and chess domains, WHALE outperforms weight-only, harness-only, and prompt-restricted baselines by up to 24 percentage points.

Paper Primer

WHALE treats the model and harness as a jointly trained system, using a modular, black-box interface to swap update methods. The core move is a cycle of small, interleaved updates: train the model under the current harness, then search for a better harness under the updated model.

This alternation acts like a relay race: the model learns to use the current harness's tools, and the harness search then adapts to the model's new reasoning patterns. By keeping these steps small, WHALE avoids the over-optimization that occurs when one component is fully trained against a static, unrefined counterpart.

WHALE consistently outperforms single-component optimization and prompt-restricted baselines.

Across SearchQA, Math, and Chess Puzzles, WHALE achieved accuracy gains of 7.67–24.38 percentage points over single-component baselines and 4.15–13.00 points over prompt-restricted Fast–Slow Training.

Small alternating updates are more efficient than stagewise optimization.

WHALE surpassed the final accuracy of stagewise optimization using only 29% and 49% of the rollouts in SearchQA and Math, respectively.

Why is joint optimization necessary if we can just optimize the model or the harness separately?

The bottleneck is domain-dependent: in some domains, the harness is the primary constraint, while in others, the model's weights are. Updating one component can unlock gains from the other, and fixing one while optimizing the other often leads to over-optimization against a brittle, unadapted counterpart.

What is the "harness" in this context, and how does it differ from a prompt?

The harness is the executable code governing the agent's environment, including tool schemas, context management, error handling, and termination logic. Unlike prompt-only adaptation, harness search can modify the underlying orchestration and middleware, providing a broader target for optimization.

WHALE demonstrates that agentic performance is not just a function of model scale, but of the co-adaptation between the model's internal weights and the external harness that defines its operational capabilities.

Researchers should stop treating model weights and agent harnesses as separate engineering artifacts. WHALE provides a template for treating them as a single, jointly-optimizable system.

The Case for Joint Optimization

We expose the joint weight‑harness bottleneck and propose WHALE, an alternating optimization framework.

The performance of an agent hinges on both its model weights and the executable harness that orchestrates context and control flow. Optimizing only one while freezing the other traps the system in a local optimum, because changes in weights alter which harness behaviours are effective and vice‑versa.

WHALE repeatedly alternates between updating the model under a fixed harness and searching for a better harness under the updated model, letting the two components co‑adapt.

**Figure 1.** Overview of WHALE. (a) Weight-only and harness-only adaptation move along a single axis of $J(\theta, h)$, whereas alternating updates allow both components to co-adapt. (b) Our simple joint harness–weight optimization recipe outperforms single-component updates across three diverse domains, as well as joint prompt–weight optimization.

The core bottleneck is a frozen component; alternating updates let weights and harnesses unlock each other’s potential.

Background and Baselines

We contextualize prior work and formalize the joint weight‑harness setting.

Related work spans three strands: multi‑turn reasoning agents, fixed‑model system optimization, and alternating joint optimization.

Agents interleave reasoning with external actions; recent methods train this behavior via search‑oriented or code‑execution environments.

Research expands beyond static model weights to search over natural‑language instructions, modular prompt programs, and full harness code.

Alternating optimization updates one component while holding the other fixed, preventing mutual drift that can trap either in a local optimum.

Only the model parameters $\theta$ are updated while the harness h stays fixed, letting the model learn to act within a static execution environment.

Only the harness h is altered while the model parameters $\theta$ remain frozen, so the system’s behavior changes purely through code‑level modifications.

The WHALE Algorithm

Alternating weight updates with harness search lets models and code co‑adapt.

Joint optimization of model weights and harness code gets stuck when one side is frozen; alternating updates let each side chase the other’s moving target.

MH treats harness design like an iterative recipe‑tasting loop: propose a few candidate scripts, evaluate them on a fixed model, keep the best, and repeat.

Iteration 0: archive $A_0=\{h_{\text{init}}\}$, artifact set $E_0$ contains the score of $h_{\text{init}}$.

Propose $C_0=\{h_{a},h_{b}\}$, evaluate both on $D_{\text{harness}}$, add their artifacts to $E_1$ and harnesses to $A_1$.

Select $h_{\text{accept}}$ as the higher‑scoring of $h_{\text{init}},h_{a},h_{b}$.

Iteration 1: archive $A_1$ now has three harnesses; propose $C_1=\{h_{c},h_{d}\}$, evaluate, extend archive to $A_2$.

Final accepted harness is the best‑scoring harness among the five total candidates.

MH’s growing archive lets later proposals refine any earlier candidate, avoiding premature convergence to a suboptimal harness.

How does MH differ from a simple hyper‑parameter grid search?

Grid search treats each harness as an independent point and discards it after evaluation. MH keeps all evaluated harnesses, allowing new proposals to reuse or remix code from any previous candidate, which creates a richer search space than a flat grid.

RSFT fine‑tunes the model on reasoning demonstrations while the current harness defines the prompt format, tools, and termination policy.

Is RSFT just ordinary supervised fine‑tuning on a reasoning dataset?

No. Ordinary fine‑tuning ignores the harness‑defined prompt and tool calls. RSFT explicitly conditions the model on the current harness, so the learned weights are tailored to the harness’s interface and can later benefit from a different harness after the next alternating step.

Start with an initial model–harness pair $(\theta_0, h_0)$.

Run RSFT for $E$ epochs on $D_{\text{weight}}$, producing updated weights $\theta_{k+1}= \text{ModelUpdate}(\theta_k; h_k, D_{\text{weight}})$.

Run MH for $I$ iterations (with $M$ candidates per iteration) on $D_{\text{harness}}$, producing a new harness $h_{k+1}= \text{HarnessSearch}(h_k; \theta_{k+1}, D_{\text{harness}})$.

Repeat the two phases until a stopping criterion is met (fixed number of cycles or adaptive early stopping).

Adaptive WHALE replaces the fixed budgets $(E,I)$ with per‑phase early‑stopping: the weight‑update phase stops when its training reward plateaus, and the harness‑search phase stops when the archive’s best score stops improving.

Experimental Results

WHALE outperforms single‑component baselines across all domains.

We evaluate whether alternating weight updates with full‑harness search beats optimizing either component alone, and whether full‑harness search adds value beyond prompt‑only adaptation.

WHALE achieves the highest mean@8 accuracy across all three domains.

Table 2 shows WHALE reaching 45% average while the best single‑component baseline caps at 36%.

WHALE improves over the stronger single‑component baseline by 7.67–10.05 percentage points, and it surpasses the prompt‑restricted FST control by 4.15–13.00 points.

**Figure 2.** Best-so-far test mean@8 accuracy on the test datasets as rollouts accumulate. Harness-only is held constant after 40, 60, and 40 iterations in SearchQA, Mathematical Reasoning, and Chess Puzzles, respectively.

**Figure 3.** Behavior metrics on test trajectories as rollouts accumulate. (a–c) SearchQA: the fraction of responses following the required answer format, the fraction of trajectories retrieving at least one document containing the reference answer, and correctness conditional on retrieving such a document. (d–e) Mathematical Reasoning: the fraction of responses reaching the token limit, and the fraction from which the verifier extracts a final boxed answer. Dotted vertical lines mark where each method achieves its best test mean@8 accuracy.

WHALE consistently outperforms single‑component baselines.

Adaptive Budgeting and Dynamics

We probe how weight updates and harness search jointly limit performance and test adaptive scheduling.

The central premise—model parameters and harness code must co‑adapt—reappears here: fixing one while updating the other stalls progress, whereas alternating joint updates let both improve together.

The patience rule lets each phase run until its training signal stops improving, then switches automatically, avoiding both noisy early switches and costly over‑optimization.

How does the patience rule differ from simply fixing a short schedule like (E, I) = (0.2, 6)?

A fixed schedule stops after a preset number of steps regardless of progress, risking premature termination (noisy selection) or unnecessary over‑optimization. The patience rule watches the training signal and stops exactly when improvement stalls, automatically landing in the interval that the ablation sweep identified as optimal.

Component‑wise limits reveal two domain‑dependent bottleneck regimes. In SearchQA, harness search is dominant: it reaches peak accuracy with only 5.79 % of the rollouts that weight updates need. In Math, the model dominates: weight updates consume 15.42 % of rollouts while harness search contributes merely 0.42 %.

We next examine scheduling choices. Alternating short phases versus a single stagewise pass is the core ablation, and we compare several (E, I) budgets across both domains.

**Figure 4.** Schedule comparison for (a) SearchQA and (b) Mathematical Reasoning. Each run—the five fixed $(E, I)$ schedules, stagewise $(E^*, I^*)$, the weight-only and harness-only baselines, and adaptive WHALE—is marked by a star at its best test mean@8 accuracy; faint lines trace each run’s best-so-far accuracy.

**Figure 5.** Running each phase to the maximum of its own line ends at a local optimum (the over-optimized extreme); too little evidence per phase stalls on noise (the noisy extreme); adaptive WHALE tracks the valley to the joint optimum.

**Figure 6.** Realized per-cycle budgets chosen by the patience rule of adaptive WHALE in (a) SearchQA and (b) Mathematical Reasoning: weight-update epochs (bars, left axis) and harness-search iterations (dots, right axis). Dotted horizontal lines mark the minimum phase lengths (0.2 epoch, I = 6).

Table 3 summarizes the best test mean@8 accuracies for each run, confirming that adaptive WHALE achieves the highest score in SearchQA (52.82 %) and the second‑best in Math (26.46 %).

SearchQA Implementation Details

We detail Adaptive WHALE for SearchQA, its experimental setup, and illustrative results.

Algorithm 2 implements Adaptive WHALE by replacing fixed budgets with per‑phase stopping rules for both weight‑update and harness‑search phases.

Adaptive WHALE (RSFT + MH) pseudocode

The experiments use three domains—SearchQA, Mathematical Reasoning, and Chess Puzzles—each with its own model size, dataset splits, and prompt limits as listed in Table 1.

Weight‑only runs a single SFT epoch over accepted trajectories, while harness‑only evaluates each candidate with one rollout per training example, incurring a cost of $|D_{\text{harness}}| \times M$ rollouts.

In SearchQA the harness may rewrite queries, set retrieval parameters, and limit assistant turns; in Mathematical Reasoning it can modify code extraction and output handling; in Chess Puzzles it controls board presentation and move validation.

All domains share a binary verifier $R(x,\tau)$ that returns 1 for a correct answer extracted from the final response and 0 otherwise, with domain‑specific parsing rules described in Section B.3.

The SearchQA verifier extracts the tag and uses a GPT‑5.4‑mini judge at temperature 0 to compare the answer against references, following the prompt in Section B.3.1.

Tables 2 and 3 show that Adaptive WHALE achieves the highest mean@8 accuracy across all domains, with the best test point reaching 52.82 % on SearchQA using only 295 k rollouts.

**Figure 7.** The Chess Puzzles example. (a) the puzzle position, with the black queen on e4 outlined. (b) after 1...Qb1+ 2.Bd1, where Qxd1 mates.

You are an impartial judge for a question-answering task. Mark CORRECT when predicted answer refers to the same entity/value/fact as any reference. Acceptable: (1) Surface variation (case, articles, punctuation) (2) Alias / abbreviation / full-vs-short form ('Pete' = 'Peter') (3) Added/removed qualifier ('actor X' = 'X') (4) Less-specific date/place CONTAINED in reference ('1962' if reference is 'September 10, 1962'; 'Wellington' if reference is 'Wellington, New Zealand') (5) Number/unit form ('4' = 'four', '188 acres' = '188 acre') (6) Different framing of the SAME geographic feature (e.g., 'Gulf of California' = 'between Baja California and Sonora' | both describe the same body of water) (7) Self-correcting answer | if the prediction explores alternatives but commits to a final answer that matches the reference, count CORRECT (e.g., 'X... actually, Y' where Y matches reference) Mark INCORRECT for: (a) Different specific person/place/song/team/company/year (b) Numeric/date mismatch where neither contains the other ('40 miles' vs '35 miles'; '1985' vs '1986') (c) Opposite or contradictory meaning (winner vs loser; 'no one' vs a named entity) (d) Different ENTITY TYPE even if related (city is NOT stadium; block is NOT channel; episode title is NOT episode number) (e) Failure to commit to an entity (only explanation/disclaimer) (f) Listing multiple distinct candidates joined by separators (slashes "/", pipes "|", commas, "or", "either...or") | even if one candidate matches the reference. A valid answer commits to a single entity; "Beijing / Tokyo / Seoul" listing the gold among 2+ alternatives is INCORRECT. (g) Verbatim passage or document quote | prediction is a multi-sentence passage, paragraph, or doc-formatted text (e.g., contains "Doc N (Title: ...)" patterns) that contains the reference answer as a substring but is not itself a concise answer. A valid answer is the entity/value alone, not the surrounding evidence prose. Question: {question} Reference answer(s): {gold} Predicted answer: {prediction} Respond with exactly one word: CORRECT or INCORRECT.

**TOOL CALL** ```xml <tool_call> <function=search> <parameter=query_list> ["Henri Coudreau birth place", "Henri Coudreau birth"] </parameter> <parameter=topk> 3 </parameter> <parameter=max_doc_tokens> 200 </parameter> </function> </tool_call> ``` **TOOL RESPONSE** ```json <tool_response> {"result": "Doc 1 (Title: \"Henri Coudreau\")\nHenri Coudreau Henri Anatole Coudreau (6 May 1859 Sonnac { 10 November 1899, State of Pará, Brazil) was a French professor of history and geography, explorer and geographer of French Guiana and the tributaries of the Amazon. At the time [... trimmed ...] ``` **MODEL RESPONSE** ```text and the search result explicitly states he was born in **Sonnac** (6 May 1859). <answer>Sonnac</answer> </think> The search results identify her husband as **Henri Coudreau** and the text explicitly states he was born in **Sonnac**. <answer>Sonnac</answer> ```

``` SYSTEM PROMPT # Tools You have access to the following functions: <tools> {"type": "function", "function": {"name": "code_interpreter", "description": "A tool for executing code.", "parameters": {"properties": {"code": {"type": "string", "description": "The code to execute."}}, "required": ["code"], "type": "object"}}} </tools> If you choose to call a function ONLY reply in the following format with NO suffix: <tool_call> <function=example_function_name> <parameter=example_parameter_1> value_1 </parameter> <parameter=example_parameter_2> This is the value for the second parameter that can span multiple lines </parameter> </function> </tool_call> <IMPORTANT> Reminder: - Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call><tool_call> XML tags - Required parameters MUST be specified - You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after - If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls </IMPORTANT> ```

Mathematical Reasoning Case Study

We count eight‑digit permutations divisible by 22 and compute their difference from 2025.

The AIME 2025 asks for the count of eight‑digit numbers that use each digit 1–8 exactly once and are divisible by 22. There are $8! = 40320$ such permutations in total. Divisibility by 22 requires the last digit to be even (condition 1) and the alternating sum of digits to be a multiple of 11 (condition 2).

A brute‑force scan of all $8!$ permutations shows that $2304$ satisfy both conditions, so $N = 2304$. The quantity we need is the difference $N - 2025$.

Chess Puzzle Case Study

Black solves the chess puzzle in two precise moves.

The puzzle presents a complex position with Black to move, described by the FEN string and a board diagram.

Black selects Qb1 (move e4b1), delivering a check.

White responds with Bd1 (move e2d1), interposing the bishop.

Black captures on d1 with the queen (Qxd1#), delivering checkmate.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers