From Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix

Olga Tsymboi, Dmitrii Stoianov, Ramil Latypov, Danil Taranets, Daniil Dryabin, Mikhail Gashkov, Viktor Zelenkovskiy, Aleksandr Fida, Gleb Alektorov, Nikita Gulyakov, Arthur Babkin, Aleksandr Medvedev, Pavel Gein, Anatolii Potapov

A modular post-training recipe for consolidating fragmented enterprise LLM traffic into a single, cost-efficient model.

How can enterprises consolidate a fragmented fleet of self-hosted LLMs into a single, high-performance model that matches the specific instruction-following and tool-calling requirements of their internal production traffic?

Enterprises often maintain a fragmented fleet of LLMs because newer models cannot easily replace older ones, leading to inefficient GPU utilization and rising serving costs. The authors consolidate this traffic by training three independent RL experts—one each for instruction following, function calling, and general dialogue—and merging them in weight space using two-stage SLERP. This recipe allows a 32B-parameter model to match or exceed the performance of a 7× larger baseline on internal benchmarks while serving 116 million requests per month at a fraction of the cost.

Paper Primer

The core challenge is that joint multi-objective training causes reward interference, where optimizing for one capability (like function calling) degrades another (like instruction following). The authors treat this as a modular engineering problem: they train separate experts for each weak axis and combine them, avoiding the "reward hacking" that occurs when a single model is forced to satisfy conflicting signals.

The merged 32B model achieves parity with a ~7× larger baseline on internal production benchmarks.

The model scored 69.6 on the in-house Arena (vs. 65.8 for the larger baseline), 0.85 on instruction following (vs. 0.83), and 0.79 on function calling (vs. 0.77). The model absorbs 50% of platform traffic (116M requests/month) while reducing per-token serving costs by 2.8× to 3.9×.

Weight-space merging outperforms joint multi-objective training.

Joint training configurations suffered from cross-domain interference, where gains in one area (e.g., instruction following) caused catastrophic regression in others (e.g., function calling), whereas the merge recipe maintained performance across all axes. The merge recipe avoids the fragile hyperparameter tuning required for joint training, allowing for independent expert updates.

Why not just use a single, larger model for everything?

Data-residency and regulatory constraints often mandate self-hosting, and a larger model increases serving latency and GPU memory requirements, making it difficult to scale across 200+ internal applications with varying needs.

Is this recipe specific to the Qwen3 model family?

The authors note that while their experiments were conducted on Qwen3-32B, nothing in the methodology—traffic-stratified benchmarking, one RL expert per axis, and weight-space merging—is inherently tied to that specific backbone.

Production-Driven LLM Consolidation

We expose why fragmented LLM fleets hurt enterprises and outline a consolidation strategy.

Enterprises under data‑residency constraints must self‑host large language models, but continuously adding newer models without retiring old ones expands the serving fleet and fragments a finite GPU pool.

In our organization this fragmentation spans over 200 internal applications, consuming a shared GPU pool and inflating the effective price per token.

We therefore consolidate traffic onto a single model, which now handles 50 % of platform requests—about 116 million queries per month—at a fraction of the previous serving cost.

Instead of maintaining many specialized models, we merge their capabilities into one high‑performance model that satisfies internal instruction‑following and function‑calling constraints.

Production error analysis reveals three dominant failure modes—semantic collapse in instruction following, over‑calling in tool use, and verbosity hacking in internal task distribution—each requiring a domain‑specific fix.

Instruction‑following and formatting errors account for 37.9 % of failures, while tool‑equipped requests represent roughly 12 % of traffic.

Fragmented model fleets dramatically increase serving cost; consolidating them yields substantial savings.

Stratified Internal Benchmarking

Construct a stratified benchmark that balances diversity and production representativeness.

Production traffic is dominated by templated requests, so naïve random sampling either repeats near‑duplicates or discards rare but important patterns, limiting the benchmark’s ability to surface real‑world failures.

We build the benchmark in two stages: first we sample a diverse, production‑representative slice of queries, then we judge each slice with a task‑specific evaluator that respects the nature of the task.

How does this two‑stage pipeline differ from simply evaluating all queries with a single uniform judge?

The uniform judge treats every query identically, which blurs the distinction between objective, reference‑based tasks and subjective, open‑ended tasks. By routing queries to task‑appropriate scorers, we preserve the signal needed for each task type and dramatically improve inter‑rater agreement.

Think of the sampler as first stripping away the variable “fill‑in” parts of a prompt, then clustering the remaining skeletons (like grouping cards by suit) before greedily picking the most different variable fragments within each cluster.

Masking yields the skeleton “Summarize the article about ___”.

LSH groups the four prompts into one bucket because their skeletons match.

Greedy max‑min selects the two most semantically distant topics (“climate change” and “quantum computing”) based on TF‑IDF cosine similarity.

Count budget limits the final sample to three prompts, preserving diversity while staying representative.

Masking isolates the truly variable content, preventing the sampler from being fooled by superficial token differences that do not affect the underlying task.

**Table 1.** Sampling methods compared by average pair-wise distance, Dist., higher is more diverse, and JS distance from the production pool for queried model $JS_{mdl}$, prompt length $JS_{len}$, service $JS_{svc}$, and task type $JS_{tax}$, lower is closer to production.

Instead of forcing a single judge to evaluate every query, we let a lightweight LLM classifier act like a triage nurse, directing each request to the specialist scorer that best matches its clinical (task) profile.

Why not simply apply the same rubric‑enhanced SBS judge to all tasks?

For reference‑based tasks the rubric adds unnecessary noise and discards the precise gold answer, which is why the uniform SBS judge only achieves $\kappa$ = 0.62. Tailoring the evaluator to the task preserves the signal needed for accurate measurement.

**Table 2.** Cohen’s $\kappa$ by task type for the initial uniform-SBS setup and the final task-specific pipeline. The average is weighted by the number of samples per task.

The Post-Training Recipe

We describe a three‑stage post‑training recipe that unifies supervised fine‑tuning and domain‑specific RL into a single deployable model.

Production error analysis highlighted three recurring pain points: instruction following, function calling, and misalignment to the internal task distribution. Our recipe tackles these axes in a unified pipeline that respects latency and cost constraints.

We first fine‑tune the base model on a mixed corpus, then split that checkpoint into three domain‑specific RL branches, and finally stitch the expert checkpoints together with a smooth interpolation.

Stage 1 SFT sees the 10 samples together, learning a single checkpoint that can handle both domains.

Stage 2 forks the checkpoint into three GRPO branches; each branch receives the same 10 samples but only the reward relevant to its domain.

After convergence, the three expert checkpoints are interpolated with SLERP, yielding a final model that retains the strengths of each branch.

The 80 % / 20 % split ensures the model retains broad coverage while still adapting to the specific quirks of production traffic.

Why not train separate SFT experts for each domain and then merge them?

Training separate SFT experts adds a full fine‑tuning pass per domain, which multiplies compute and storage costs. Empirically (Table 3) a single mixed SFT already preserves per‑domain quality, so the extra experts provide no measurable benefit.

GRPO refines each expert by optimizing a reward model that reflects the target domain, while a length penalty and a stronger KL term keep generations fluent and on‑distribution.

How does GRPO differ from standard PPO‑based RL for language models?

GRPO uses a group‑relative scoring scheme that compares the current policy against a set of reference policies, rather than a single baseline. This relative ranking yields smoother gradients and mitigates reward‑hacking, especially when combined with the length and KL regularizers.

Load the base Qwen3‑32B model and replace its tokenizer with the Cyrillic‑dense variant.

Run the combined SFT phase on the mixed corpus (general + in‑house + IF + FC data).

Fork the resulting checkpoint into three independent GRPO branches (general, IF, FC).

For each branch, train a domain‑specific reward model and apply GRPO with length penalty and KL regularization.

Merge the three expert checkpoints using two‑stage sequential SLERP interpolation.

Evaluate the merged model on internal benchmarks and deploy the single checkpoint.

High‑level pseudocode for the three‑stage recipe.

**Figure 1.** Evolution of GRPO mean reward (left) and mean response length in tokens (right) over training steps. Without length and KL regularization, the model exploits the RM verbosity bias.

**Table 4.** Ablation: effect of adapting the reward model to in-house data. The in-house-adapted RM does not improve over the general RM.

Performance and Ablations

Consolidated model matches larger specialists while slashing serving cost.

The merged 32B model attains quality on par with the 235B specialist while reducing per‑token serving cost by up to $9\times$.

Table 6 shows the merged model matching or exceeding the larger model on Arena, BFCL, SmartSearch, and ruWildChat benchmarks; deployment logs report a 2.8‑ to 3.9× input‑cost drop and 4‑ to 9× overall cost reduction.

Shared supervised fine‑tuning already preserves domain‑specific quality, but the same mixture fails to transfer when alignment (GRPO) is applied.

**Table 3.** SFT ablation results. Domain SFT denotes a model trained on a single domain: general for the arena columns, instruction following for IFEval, and tool calling for the BFCL columns.

**Table 5.** Cross-domain transfer performance of GRPO experts trained on different domains (General, Instruction Following, and Tool Calling) compared to the shared SFT baseline.

**Table 7.** Two-stage sequential SLERP merge orderings. Each entry is the mean ± standard deviation of combinations of near-convergence expert checkpoints merged with identical coefficients.

**Table 8.** Joint multi-domain GRPO vs. expert merging at the 32B scale. $^\dagger$ trained with a 1.7$\times$ larger budget than the other runs.

**Table 6.** Comparison of models on Dialogue, Instruction Following, and Function Calling benchmarks. Inh.: in-house benchmarks; SS: SmartSearch `F1_R`&G; ruMC/ruWC: Russian MultiChallenge / WildChat Hard Ru.

Consolidating domains into a single 32B model delivers specialist‑level performance with dramatically lower serving cost.

Summary and Caveats

We wrap up the recipe, its benefits, and its current limits.

We introduced a production‑driven recipe that consolidates a fragmented self‑hosted LLM fleet into a single deployment model.

Traffic analysis across more than two hundred internal applications revealed three gaps: instruction‑following, function‑calling, and alignment with the internal request distribution.

Joint GRPO caused these objectives to interfere, so we trained one expert per axis from a shared SFT checkpoint and merged them using a two‑stage SLERP procedure.

The main lesson is that modular objectives make enterprise post‑training easier to control, debug, audit, and extend.

Each axis produced its own failure mode, each of which required a targeted fix.

Combined with template‑aware traffic sampling and task‑specific judging calibrated against human annotators, the recipe provides a practical path from production error analysis to deployment.

The final Qwen3‑32B non‑reasoning model is competitive on target deployment metrics while serving 116 M monthly requests at lower cost.

A public checkpoint trained without internal data shows similar public‑benchmark gains, suggesting the recipe accounts for much of the improvement.

Limitations: evaluation covers only Russian and English, and the in‑house benchmarks are built from Russian‑language traffic of a single corporate deployment.

All quantitative claims should be read as validated for Russian and English only; verification on other languages and domains is left to future work.

We rely on LLM judges calibrated against human annotators to score open‑ended quality.

Deployment evidence comes from a single organization and platform, and all experiments use only the Qwen3 model family.

While the recipe does not depend on the backbone, we have not validated it on other model families or in other organizations.

Ethical statement: benchmarks are derived from logged internal traffic, processed on self‑hosted infrastructure, deduplicated, anonymized, and variable spans masked to protect privacy.

No public or external end‑user data is used; we control for contamination by holding out evaluation items and removing exact and template‑level duplicates.

Weak‑axis experts are trained on synthetic data only, so internal traffic does not directly supervise them.

Gold answers, benchmark translations, and task‑specific verification procedures were validated by professional annotators; each response pair was labeled by three annotators and the majority vote taken as consensus.

The model is intended for internal enterprise assistance; like any LLM it can produce factually incorrect outputs or violate constraints, and deterministic verifiers only reduce—not eliminate—this risk.

Tool calls emitted by the model should be gated and validated before execution in production.

Production Error Analysis

Internal benchmark ablations quantify the impact of each component.

This appendix details the internal benchmarks used to evaluate the impact of each design component on production‑driven LLM performance.

**Table 9.** Failure-type distribution over a human-reviewed sample of $n=2,500$ in-house traffic responses, one primary failure per response ($\kappa = 0.62$). Formatting denotes violations of structural, JSON-schema, or length constraints; the non-format row covers content-level instruction violations such as tone, role, enumeration, and lexical prohibitions.

**Table 10.** Primary task-type distribution on two independent slices of internal requests (LLM classifier labels) and the accuracy (Acc.) of each assigned label against human consensus (on items where annotators reached a consensus). Accuracy covers the four classifier task types only; Tool Calling is detected by a regular expression and Other is the residual category.

**Table 11.** Reconstructing the consensus overall SBS verdict from per-criterion annotator votes alone. The full criteria aggregator recovers almost perfect agreement; removing subjective criteria substantially degrades agreement, especially on content generation.

**Table 12.** Judge–human agreement for DeepSeek-V3-0324. The Avg. Open-ended row reports a task-weighted average over the summarization and content generation tasks.

Greedy max‑min sampling introduces a 1.9 % Jensen–Shannon distance shift relative to template‑based sampling, which yields near‑zero shift.

Described in the “Task‑type preservation under sampling” paragraph, the JS distance for Greedy max‑min is 1.9 % versus 0 % for template‑based.

In‑Tag sampling dramatically distorts the task‑type distribution, increasing the Jensen–Shannon distance by 14.5 %.

The “Task‑type preservation under sampling” paragraph reports a 14.5 % JS distance for In‑Tag.

Removing subjective criteria from the aggregation rule reduces Cohen’s Kappa for Summarization by 0.13.

Table 11 shows Kappa 0.85 for the full aggregator versus 0.72 for objective‑only.

Removing subjective criteria from the aggregation rule reduces Cohen’s Kappa for Content generation by 0.38.

Table 11 reports Kappa 0.87 versus 0.49 for the objective‑only variant.

Adding the full criteria checklist to the SBS judge improves $\kappa$ for Summarization by 0.07.

Table 12: baseline $\kappa$ = 0.61, Criteria‑Guided $\kappa$ = 0.68.

Adding the full criteria checklist to the SBS judge improves $\kappa$ for Content generation by 0.06.

Table 12: baseline $\kappa$ = 0.49, Criteria‑Guided $\kappa$ = 0.55.

Using the Per‑Criterion + Overall verdict raises $\kappa$ for Content generation by 0.30 over the baseline.

Table 12: baseline $\kappa$ = 0.49, Per‑Criterion + Overall $\kappa$ = 0.79.

Applying the Per‑Criterion (Obj→Subj) rule alone underperforms the baseline SBS judge for Summarization, decreasing $\kappa$ by 0.08.

Table 12: Per‑Criterion (Obj→Subj) $\kappa$ = 0.53 versus baseline 0.61.

Related Work

Enterprises keep separate LLMs because generic models miss Russian instruction and tool‑calling needs; we propose a post‑training recipe.

Related work falls into two strands: Russian‑language model training from scratch and multilingual backbone adaptation.

Early efforts built Russian models by training monolingual corpora from the ground up, improving fluency but not targeting instruction or tool use.

These works fine‑tune large multilingual models on Russian data, achieving better generation quality while retaining cross‑lingual capabilities.

Benchmarks and verification tools designed for English assume fixed word order and case, making them unsuitable for Russian where morphology and free order multiply valid forms.

Studies extending tool‑use capabilities to many languages report that most errors arise from mismatches in parameter‑value language rather than intent misunderstanding.

These systems extend large language models via RLHF and RLVR, employing GRPO and its variants to align models with reward models.

Research on RLHF, RLVR, and GRPO highlights that joint optimization across instruction following, function calling, and chat leads to fragile training dynamics.

Techniques that train separate experts for each skill and then merge their weights have become standard for scaling post‑training systems.

Russian IFEval Adaptation

We describe Russian adaptations of IFEval and BFCLv3 for instruction‑following evaluation.

To evaluate instruction‑following in Russian we built ruIFEval by translating the original IFEval samples with human annotators, preserving each prompt’s intent and the tested behavior. Because IFEval relies on exact, verifiable constraints rather than language‑specific morphology, the adaptation was straightforward and largely language‑agnostic. When constraints were language‑dependent we rewrote them—for example, character‑count constraints now use Cyrillic characters—so the verifier checks the same behavior in a linguistically appropriate way.

Existing multilingual benchmarks lack a Russian split, so we localized BFCLv3 into ruBFCLv3, ensuring each translated request, tool description, and target answer remains unambiguous and solvable. The translation pipeline jointly rewrites the JSON‑structured request and tools, then an ensemble of LLM judges verifies logical consistency and completeness across all fields. Examples with unusually high model disagreement were flagged for manual review, providing a human‑in‑the‑loop correction that is feasible for evaluation but not for training‑scale data.

Russian MultiChallenge Adaptation

This appendix details the Russian MultiChallenge adaptation and the ablation of alignment techniques on Qwen3‑8B.

We evaluate models on the MultiChallenge benchmark, a multi‑turn instruction‑following suite that probes long‑horizon failures, and we construct a Russian version by translating all dialogues while preserving roles, turn order, evaluation axes, and the binary pass criterion.

A naïve full‑dialogue translation caused models to follow embedded instructions instead of translating, leading to omissions and reference shifts; therefore we translate one turn at a time, supplying preceding turns as context, which is crucial for the RELIABLE VERSION EDITING subset whose conversations are on average 2.4× longer (≈2,300 words, 13 turns).

For all subsets except RELIABLE VERSION EDITING we apply an iterative translate‑verify‑revise pipeline: the gemini‑3.1‑pro‑preview model checks each turn against the English source for preserved instructions, facts, references, and rubric conditions, with flagged turns revised up to five times before expert annotators perform a final verification.

Most binary rubrics encode semantic conditions invariant to surface language and remain in English; only 44 of the 273 rubrics required localization—21 in INSTRUCTION RETENTION, 20 in RELIABLE VERSION EDITING, and 3 in SELF‑COHERENCE—while none in INFERENCE MEMORY needed changes.

**Table 13.** Ablation of alignment stages on Qwen3-8B. Score denotes win rate against the baseline; Avg. Len. is the mean response length in tokens. Len. Rebal. refers to rebalancing DPO training to favor shorter chosen responses. Len. Pen. and KL denote the multiplicative length penalty and increased KL divergence coefficient, respectively.

Synthetic Data Generation

Appendix D describes the Russian IF data pipeline and the RL reward design used for training.

We built a Russian‑focused instruction‑following (IF) dataset by adapting the AutoIF pipeline. All pipeline components—generation, validation, and filtering—operate on Russian text to respect language‑specific formatting, morphology, and stylistic constraints.

Constraint generation started from 54 hand‑written seed instruction types, which were translated into Russian and then expanded. By prompting an LLM with groups of five constraints, we generated roughly 100 k candidate constraints.

Exact‑match and semantic deduplication pruned the candidates to 72 k unique constraints.

For each constraint we created eight validation functions and three synthetic test cases per function. Consistency filtering removed test cases where fewer than half the validators produced the expected label, then discarded validators with accuracy below 0.5, finally keeping only constraints with at least three validators and five test cases (including ≥2 positive and ≥2 negative cases), leaving 50 k constraints.

Back‑translation validation compared a generated natural‑language instruction from each validator with the original constraint using cosine similarity. Validators below a similarity of 0.6 were removed, and constraints losing more than 60 % of their validators were discarded, resulting in 43 k constraints.

Each remaining constraint was randomly attached to three SFT samples, producing 131 k constraint‑augmented samples. After scoring candidate responses with the AutoIF prompt‑based procedure and keeping only those with a maximum score of 10, we retained 62 k samples.

We split the samples by constraint placement: 20 k samples received the constraint in the user instruction (via prepending, appending, or natural integration), while the remaining 42 k samples placed the constraint in the system instruction.

For each sample we generated eight candidate completions and discarded any sample whose completions all failed validation (accuracy < 1). The final dataset contains 10 k user‑level and 16 k system‑level constraint samples, each scored by a reward model to select the best completion.

During training we first added IF data in the supervised fine‑tuning (SFT) stage, which provides only positive demonstrations. An evaluation of DPO on the same IF data showed no IF metric improvement and increased average response length by ~1.5×, suggesting preference optimization encourages verbosity without guaranteeing constraint compliance.

We therefore switched to GRPO‑style reinforcement learning with verifiable rewards. While this yielded rapid gains in measured IF accuracy, the model learned to optimize validators directly, producing degenerate outputs that satisfy the constraint but ignore the underlying question.

For example, a system instruction asked for an acrostic spelling “EXAMPLE”. The model answered only with the required initials, satisfying the acrostic validator but providing no answer to the user’s statistical query.

**Table 14.** Alignment and reward ablation on ruIFEval for 8B model. P-S/P-L denote strict/loose prompt-level accuracy; I-S/I-L denote strict/loose instruction-level accuracy.

Function Calling Data Details

Appendix E details data creation, training stages, and ablation results for the FC recipe.

This appendix documents how the authors synthesize multilingual function‑calling data, train the model in two stages (SFT then GRPO), and evaluate each stage with a series of ablations.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers