IQuest-Q1: Advancing Foundation Capabilities for Agentic CLI Systems
A. Jian Yang, An Huang, Bo Dan, Chen Mu, Chengran Yang, Chuan Shi, Feng Chang, Hongxin Li, Hongyuan Tao, Jinxing Zhang, Jujie He, Kun Wang, Lin Jing, Ming Yang, Qicheng Hu, Renyuan Li, Shark Liu, Shawn Guo, Shi Qing, Shi Zhong, Tianyu Zheng, Wei Liu, Wei Zhang, Wenrui Cai, Wenxiao Chen, Xuxing Chen, Yan Xing, Yi Xing, Yifan Luo, Yiran Xing, Yizhi Li, Yongsheng Kang, Yucheng Ding, Yue Song, Yun Qing, Yunqi Liu, Zelong Huang, Zhengmao Ye, Zhentao Fan, Zhiteng Li, Ziling Huang, Bihong Tang, Bingli Wang, Ce Jun, Changhao Chen, Chen Hu, Chen Xiao, Chi Liu, Chunjiang Ge, Danyang Li, Deyang Kong, Dian Qi, Ding Tan, Evan Yang, Fanglin Xu, Fei Bai, Gang Liu, Guochang Li, Hao Tang, Haoran Feng
IQuest-Q1 is a 320B sparse MoE model optimized for agentic CLI interaction through multi-stage training and recursive self-improvement.
How can a sparse Mixture-of-Experts (MoE) model be optimized for agentic CLI tasks through stable pre-training and specialized post-training?
Agentic models often struggle to reliably interact with command-line interfaces (CLIs) because they lack the specialized training to handle diverse tool-call schemas, long-horizon reasoning, and environment-specific constraints. IQuest-Q1 addresses this by integrating a CLI-centric training pipeline that combines staged pre-training, multi-teacher distillation, and model merging to consolidate diverse agentic capabilities into a single sparse Mixture-of-Experts (MoE) architecture. The model achieves strong performance across coding and general agentic benchmarks, including 57.5 on SWE-Bench Pro and 84.5 on CyberGym, while enabling an iterative, human-supervised process for its own future development.
Paper Primer
IQuest-Q1 is a 320B parameter sparse MoE model (15B activated) designed to bridge the gap between general language modeling and reliable, tool-using agentic execution. The core mechanism hinges on a "multi-scaffold" post-training strategy: it uses Supervised Fine-Tuning (SFT) to establish basic CLI interaction, Reinforcement Learning (RL) to optimize for task-level feedback, and Multi-Teacher On-Policy Distillation (MOPD) to unify specialized expert models into a single, robust policy.
IQuest-Q1 demonstrates high-efficiency scaling and robust agentic performance.
The model achieves 1.93x higher scaling efficiency than baseline MoE designs and scores 80.3 on SWE-bench Multilingual and 93.3 on GPQA Diamond. It requires approximately 52% of the training compute of comparable baselines to reach equivalent cross-entropy loss.
Beyond static benchmarks, the authors integrate IQuest-Q1 into a "human-on-the-loop" infrastructure where the model identifies its own capability gaps—such as tool-call schema mismatches—and proposes data-augmentation interventions. This creates a practical pathway for recursive self-improvement, where the model assists in the development of its own successors.
Why is a specialized CLI-centric training pipeline necessary for this model?
General-purpose models often fail in CLI environments due to reward hacking, inability to handle diverse tool-call schemas, and difficulty with long-horizon reasoning; the pipeline uses environment-specific safeguards and multi-harness training to ensure reliable, verifiable task completion.
What is the role of model merging in this architecture?
Model merging combines parameter updates from different expert branches and training stages, allowing the model to integrate complementary capabilities—such as terminal interaction and long-horizon coding—while mitigating conflicts between parameter updates.
IQuest-Q1 shifts the focus of agentic foundation models from pure reasoning to verifiable, environment-grounded execution, providing a blueprint for models that can actively participate in their own iterative improvement cycles.
Introduction and Motivation
IQuest‑Q1 tackles the gap between static language models and interactive CLI agents.
Current large language models excel at static text generation but falter when required to interact with command‑line tools. This mismatch limits their usefulness as autonomous agents.
We need a model that can reliably execute commands, query tools, and reason over long sequences, turning pure language prediction into purposeful action.
**Figure 11.** Illustration of recursive multi-token prediction. Here, $L$ denotes the input sequence length and $t$ denotes the drafting step.
The key shift is moving from static language modeling toward an agentic CLI interaction paradigm.
Model Architecture
IQuest‑Q1 combines sparse experts, split normalization, and recursive drafting to scale stable attention.
Deep Transformer stacks suffer from exploding residual variance: as layers accumulate, later updates become dwarfed, leading to unstable training and poor scaling. IQuest‑Q1 addresses this by reshaping how normalization is applied across sub‑layers, ensuring each depth contributes meaningfully.
The model interleaves sparse expert layers with two complementary attention styles—Grouped‑Query Attention and Sliding‑Window Attention—while stabilizing the residual stream via SplitNorm and $\mu$P scaling.
Normalize: $z = x / 1.32 \approx [1.52,\,-0.76,\,0.38]$.
Assume the attention sublayer returns a vector $a=[0.2,\,0.1,\,-0.1]$; after RMSNorm it becomes $a' = a / \|a\|_{rms}\approx[0.71,\,0.35,\,-0.35]$.
Residual addition: $h = z + a' \approx [2.23,\,-0.41,\,0.03]$.
Feed‑forward network (identity for illustration) yields $f = h$; RMSNorm on $h$ gives $f' = h / \|h\|_{rms}\approx[1.99,\,-0.37,\,0.03]$.
Final output: $y = h + f' \approx [4.22,\,-0.78,\,0.06]$.
Normalizing each residual branch prevents the magnitudes from growing unchecked, so even after several layers the activations stay in a tractable range.
How does SplitNorm differ from the conventional Pre‑Norm used in most Transformers?
Pre‑Norm applies the same normalization before every sub‑layer, which leaves the feed‑forward path’s residual unscaled and can cause later layers to contribute little. SplitNorm normalizes only the attention input, then leaves the feed‑forward residual untouched, while still applying a second RMSNorm after the attention output. This hybrid scheme preserves the stability of Pre‑Norm for attention but restores the stronger signal flow of Post‑Norm for the feed‑forward network.
Instead of feeding every token through the full parameter set, the model routes each token to a small, learned subset of expert sub‑networks, dramatically reducing compute per token while preserving model capacity.
Why not simply use a larger dense MLP instead of a sparse MoE?
A dense MLP would require activating all parameters for every token, leading to quadratic compute growth with model size. Sparse MoE keeps per‑token compute roughly constant while still leveraging a huge total parameter pool, which is essential for scaling to the sizes targeted by IQuest‑Q1.
**Figure 3.** Overall architecture of IQuest-Q1.
Scaling Efficiency and Stability
We quantify how IQuest‑Q1 scales more efficiently and stays stable at larger sizes.
Large‑scale models often waste compute because loss improves only marginally as FLOPs increase. The pain point is to get more performance per training compute while keeping training stable.
IQuest‑Q1 reaches a lower cross‑entropy loss than a comparable Qwen3‑MoE design for the same amount of training FLOPs.
How does IQuest‑Q1 achieve higher scaling efficiency than the Qwen3‑MoE baseline?
Beyond the sparse expert routing, IQuest‑Q1 uses a three‑stage training schedule that aligns the expert capacity with the data distribution early on, and it applies a stability‑aware learning‑rate schedule that avoids wasted updates at large compute budgets.
**Figure a.** Scaling efficiency of IQuest-Q1 and Qwen3-MoE using identical training data.
The gap between the largest learning rate that stays stable (safe) and the learning rate that gives the fastest optimization (optimal) tells how far a model can grow before stability limits dominate.
Compute the ratio $\eta_{\text{safe}} / \eta_{\text{opt}} = 0.002 / 0.001 = 2$.
Take the natural logarithm: $H_8 = \log(2) \approx 0.69$.
Since $H_8 > 0$, the optimal learning rate lies comfortably inside the stable region for $N=8$ B.
The positive headroom shows that even at 8 B parameters the model can be trained faster than the stability limit, implying room to grow before hitting the CMS.
**Figure b.** Maximum safe and optimal learning rates across activated-parameter scales.
Data Packing Infrastructure
Data Packing packs tokens to cut padding and balance compute across replicas.
Padding dominates token throughput when sequences contain many short samples. In long‑context training the variation in document length further skews compute across replicas, leading to idle resources.
Best‑fit packing fills each fixed‑length sequence slot with tokens from multiple samples, so fewer padding tokens are needed and the global batch processes more real data.
Start filling the first sequence (capacity 4). Place the 3‑token sample → remaining capacity 1.
Insert the 1‑token sample to finish the first sequence. The first packed sequence is now [3, 1].
Begin the second sequence with the 2‑token sample → remaining capacity 2.
Place the 5‑token sample cannot fit, so it stays in the window as leftover.
Add the 4‑token sample to the second sequence, filling it completely → second packed sequence [2, 4].
Carry over the 5‑token and 6‑token samples to the next iteration; they are prioritized.
Best‑fit packing uses the available slack in each sequence to absorb short samples, dramatically lowering the proportion of padding tokens while keeping the algorithm simple and fast.
How does this packing differ from simply concatenating samples until the sequence is full?
Simple concatenation would keep the original order and could leave large gaps when a long sample overshoots the remaining slots. Best‑fit packing reorders within the bounded window, allowing short samples to fill those gaps and guaranteeing that no sample is postponed indefinitely.
Instead of gathering all keys and values on every device, each rank exchanges only the pieces it needs, enabling full‑attention computation without a central bottleneck.
Why does sliding‑window attention need a separate halo exchange instead of just sending the full K/V matrices?
Sending the full matrices would waste bandwidth because only queries near the window edges actually need remote keys/values. The halo exchange transmits just the minimal required slice, and by overlapping it with local computation the overall step time stays close to that of pure local attention.
Semantic invariants extracted from the training setup are turned into runtime checks that catch deviations from the expected single‑machine behavior during distributed training.
What kinds of invariants are monitored, and why are they important for distributed training?
Invariants include deterministic token sharding, consistent random seeds, and identical loss scaling across ranks. Violating any of these can cause subtle drift between the distributed run and the reference single‑machine baseline, leading to hidden bugs that only appear after long training periods.
**Figure 9.** Optimal learning rate transfer across model widths and depths under $\mu$P scaling.
Pre-Training Procedure
Describes the staged training pipeline and sparse MoE design that enable efficient scaling.
Training ever‑larger language models strains memory and compute, making naïve scaling impractical. The pre‑training stage must therefore balance data diversity with architectural efficiency to keep training stable.
**Figure 2.** Training pipeline of IQuest-Q1.
Table 1 lists the core architectural choices: 48 query heads and 8 key‑value heads (each 128‑dim) are arranged in a hybrid pattern of three sliding‑window attention (SWA) layers followed by one full‑attention (FA) layer. SWA uses a window of 4,096 tokens, and partial RoPE is applied to 32 dimensions with bases 10,000 for SWA and 1,000,000 for FA. Query‑Aware Sink Attention runs in every layer, the first transformer layer employs a dense FFN of dimension 12,288, while later layers switch to sparse MoE FFNs (256 experts, 8 active per token, expert dimension 1,536) using SwiGLU.
Instead of predicting a single next token, the model predicts a short sequence of tokens jointly, which stabilizes training of the large backbone and improves long‑range coherence.
Project each embedding into $h$ heads: head 1 receives $(1,0)$ and $(0,1)$, head 2 receives $(0,0)$ and $(0,0)$.
Compute attention scores within each head (self‑attention only for this toy example).
Aggregate the attended vectors, yielding joint predictions $p_1$ and $p_2$.
Compute the MTP loss as $0.3 \times \bigl(\text{CE}(p_1, y_1) + \text{CE}(p_2, y_2)\bigr)$.
The joint loss ties the two predictions together, encouraging the model to maintain consistency across adjacent tokens—a property the standard next‑token loss cannot enforce.
How does the MTP layer differ from ordinary next‑token prediction?
Standard prediction scores a single token at a time, treating each step independently. MTP predicts a fixed‑length token block in one forward pass, shares the same attention context across the block, and adds a separate weighted loss term (0.3) that ties the predictions together, reducing exposure bias.
Training Hyper-Parameters
Mid-training refines the model with a curriculum, specialized optimizer, and extended context to boost agentic performance.
Mid‑training bridges the broad pre‑training stage and the task‑specific post‑training stage. It does so with a curriculum that mixes diverse data, applies fine‑grained token supervision, and progressively stretches the context window.
Muon replaces Adam’s adaptive moments with a single high‑momentum update that is orthogonalized and scaled to match Adam’s RMS, giving stable large‑scale training without extra hyper‑parameters.
How does Muon differ from Adam if both aim to stabilize large‑scale updates?
Adam maintains separate first‑ and second‑moment estimates ($m_t$, $v_t$) and scales the update by $\sqrt{v_t}$. Muon discards the second‑moment estimate entirely, orthogonalizes the raw gradient, and then rescales it with a deterministic factor ($\max(m,n)\times0.2$). This removes the need to tune $\beta_2$ and yields a more predictable update magnitude.
Compute the QR decomposition of $G$: $G = QR$, where $Q$ is orthogonal.
Take the orthogonal factor $Q$ and multiply by the scaling factor $3\times0.2 = 0.6$.
Apply momentum: $\Delta W = 0.95\,\Delta W_{\text{prev}} + 0.6\,Q$ (assuming $\Delta W_{\text{prev}}=0$ for the first step).
Update the weight: $W_{\text{new}} = W_{\text{old}} - \Delta W$.
Muon’s update magnitude is fully determined by the matrix shape and a fixed scalar, so the optimizer behaves identically across training runs without tuning a second‑moment decay.
All other parameters use Adam with $\beta_1=0.9$, $\beta_2=0.95$, and $\epsilon=10^{-15}$. Weight decay of $0.1$ is applied everywhere except layer‑norm parameters, which are exempt. QKNorm Clip caps the affine parameters of QKNorm at $10$ to prevent runaway scaling.
Instead of fixing the sequence length once, the model is trained on increasingly longer windows (32 K → 128 K → 512 K tokens), letting it learn to attend over larger spans without destabilizing earlier phases.
Phase 1 (32 K): effective length $30\,000$ → RoPE base $5\times10^{5}$.
Phase 2 (128 K): effective length $120\,000$ → RoPE base $1\times10^{6}$.
Phase 3 (512 K): effective length $480\,000$ → RoPE base $1\times10^{6}$ (unchanged).
Across phases, the cosine learning‑rate curve continues without reset, so the learning‑rate at the start of each phase is the same point on the curve.
Because the RoPE base scales with the maximum length, positional encodings stay proportional, allowing the model to reuse learned attention patterns as the window grows.
The training curriculum mixes three data categories—high‑quality natural text, QA pairs, and agentic trajectories—using a chat template that retains reasoning and tool‑call tokens. Token‑level loss masks are applied directly to selected spans, decoupling supervision from role‑specific tokens and giving finer control than whole‑conversation masking.
**Figure 8.** Representative training-loss trajectories for the model under stable, warning, and collapsed training regimes.
Evaluation uses bits‑per‑byte (BPB) on held‑out agent trajectories—lower BPB means the model assigns higher likelihood to competent behavior. Additionally, we break down long‑horizon tasks into atomic abilities and measure each ability’s success rate, exposing specific capability gaps for curriculum refinement.
Post-Training Task Synthesis
We safeguard post‑training environments against reward hacking with a three‑layer protection pipeline.
IQuest‑Q1 builds on a sparse Mixture‑of‑Experts backbone and a three‑stage training pipeline that first stabilizes scaling, then synthesizes tasks, and finally merges models for agentic CLI performance.
We generate diverse training tasks at scale by running three parallel pipelines—general, reasoning, and coding—each pulling from domain‑specific data sources and applying a solve‑then‑refine loop.
General: query two public APIs, generate a prompt “list available endpoints”, and verify the response contains at least one endpoint.
Reasoning: select two short abstracts on “quantum entanglement”, ask the model to summarize the main claim, and compare against a reference summary.
Coding: clone two tiny repositories (each
All three tasks pass their respective verifiers, so they are added to the training pool.
Even with minimal data the framework produces self‑contained, verifiable tasks, demonstrating that the pipelines scale from toy examples to massive corpora without changing their core logic.
How does this differ from ordinary data augmentation?
Standard augmentation merely perturbs existing examples (e.g., synonym replacement). Our framework creates entirely new tasks by stitching together fresh inputs, reference solutions, and domain‑specific verifiers, so the model learns to solve novel problems rather than recognize altered copies.
Before collecting trajectories we harden environments, during rollouts we block unsafe actions, and after rollouts we audit trajectories to excise any reward‑hacking behavior.
**Figure 5.** Overview of coding-agent task instances and their environments, verifiers, and construction methods.
**Figure 6.** IQuest-Q1 Post-Training Pipeline.
Reinforcement Learning Design
We detail the RL pipeline that ensures verifiable task completion across diverse harnesses and distills multi‑expert knowledge.
Reinforcement learning must guarantee that the agent actually finishes a task while respecting user constraints and keeping execution cost low. Prior pipelines struggled because policy errors were tangled with environment or evaluator faults, making credit assignment noisy. Our design isolates the policy’s contribution and scales across many tool‑harnesses.
Instead of training a single policy on one tool, we train a shared policy on episodes drawn from many harnesses and versions, then reward only the policy’s verifiable successes while masking unrelated failures.
Task 1 receives (A, v1); Task 2 receives (B, v2); Task 3 receives (C, v1); Task 4 receives (A, v2).
For each task the policy generates a rollout; GRPO repeats the rollout of Task 1 under (A, v1) three times to compute a baseline reward.
Measured progress rewards: Task 1 = +0.8, Task 2 = +0.5, Task 3 = 0 (no progress), Task 4 = +0.7.
Penalties: Task 3 incurs a constraint‑violation penalty of –0.3; all other tasks incur no penalty.
Failure attribution masks the penalty on Task 3 because the violation stems from the environment (a missing file), not the policy.
This toy episode shows how the same policy can be evaluated across heterogeneous harnesses while the reward system isolates policy mistakes from external faults.
How does GRPO differ from standard PPO or A2C updates?
Standard PPO compares the current policy to a stale baseline across many unrelated trajectories, which introduces high variance. GRPO instead repeats the exact same task under the same harness and evaluator, so the baseline reflects only stochasticity in the policy itself, yielding a tighter advantage estimate.
After the RL policy is shaped, we compress its knowledge into a single student model by distilling from multiple frozen experts, each specialized for a different aspect of agentic behavior.
We start from the SFT‑Stage‑2 checkpoint and teach it to imitate four frozen experts, each covering a distinct agentic skill, using on‑policy rollouts and a clipped advantage signal.
Rollout 1 (coding) yields student log‑probability $\log p_{\theta_k}(y_{1,t}\mid h_{1,t}) = -0.2$ and teacher probability $\log q_{\text{code}}(y_{1,t}\mid h_{1,t}) = 0.1$.
Raw advantage = $0.1 - (-0.2) = 0.3$, which lies within $[-0.5,0.5]$, so $\tilde{a}_{1,t}=0.3$.
Rollout 2 (search) yields student log‑probability $-0.8$ and teacher log‑probability $-0.1$.
Raw advantage = $-0.1 - (-0.8) = 0.7$, exceeding $c$, so clipping gives $\tilde{a}_{2,t}=0.5$.
The surrogate loss aggregates the two clipped advantages, normalizing by token length, and back‑propagates only through the student.
Clipping prevents a single high‑advantage token from dominating the update, ensuring stable on‑policy distillation across heterogeneous teachers.
Why not use plain KL divergence instead of the clipped advantage?
Plain KL can produce very large gradients when the student’s probability is far from the teacher’s, destabilizing training. The clipping caps the advantage, preserving the direction of improvement while limiting magnitude, which is crucial when the teachers span very different domains.
Model Merging and Loss
We merge expert checkpoints by residual addition with ℓ₂‑norm calibration to preserve stability.
Training a sparse Mixture‑of‑Experts model across multiple stages often diverges because each stage introduces its own parameter drift. The authors therefore interleave training with a merging step that stabilizes the trajectory before moving to the next stage.
Think of each expert branch as a paint swatch; the merge step blends the swatches into a single color, then rescales the mixture so the overall intensity matches a reference palette.
How does this merging differ from naïvely averaging the expert parameters?
Simple averaging treats every expert equally and ignores the magnitude of each branch’s change. The residual merge first subtracts the reference checkpoint, applies task‑specific weights, and finally rescales the combined delta, so the update respects both relative importance and overall norm stability.
Weighted sum of deltas: $0.6\cdot\Delta_1 + 0.4\cdot\Delta_2 = (0.08, 0.02, 0.04, 0.04)$.
Compute ℓ₂‑norm of the sum: $\|\cdot\|_2 = \sqrt{0.08^2 + 0.02^2 + 0.04^2 + 0.04^2} \approx 0.099$.
Normalize the update: $(0.08,0.02,0.04,0.04) / 0.099 \approx (0.81,0.20,0.40,0.40)$.
Add to reference: $\theta_{\text{new}} = \theta_{\text{ref}} + (0.81,0.20,0.40,0.40) = (1.81, 2.20, 3.40, 4.40)$.
The ℓ₂‑norm step rescales the blended update so that its overall magnitude matches the reference scale, preventing a sudden jump in parameter norm that could destabilize subsequent training.
Beyond merging, the system supports a multi‑harness, multi‑version agent runtime. A single policy can be trained on trajectories from many harnesses and model versions within one run, each instance specifying its own scaffold, task environment, and verifier.
Rollout‑training consistency is achieved by a session‑level prefix tree that retains token histories across turns and branches. When a request arrives, the stored prefix is combined with fresh observations to produce a response in the expected structured format.
Asynchronous distillation (MOPD) runs teacher forward passes on trajectories generated online by the student, synchronizing student parameters periodically while explicitly limiting sample staleness. The same execution path is used for both teacher evaluation and student log‑probability computation, reducing engine‑induced discrepancies.
**Figure 10.** Overlapping halo exchange with partial attention computation.
Evaluation Framework
IQuest‑Q1’s benchmark performance across coding and general agentic tasks.
We evaluate IQuest‑Q1 on a comprehensive suite of benchmarks covering both coding and general agentic capabilities. The coding suite includes DeepSWE v1.1, SWE‑bench variants, ProgramBench, CyberGym, Terminal‑Bench series, PaperBench, Live‑CodeBench v6, and the proprietary IQuest‑CLIBench. The general suite spans GPQA Diamond, Humanity’s Last Exam, GDPval, APEX‑Agents v1.1, Toolathlon‑Verified, and numerous other agentic tasks.
**Table 2.** Performance across coding and general capability benchmarks. Scores are reported at the highest available reasoning effort for each model. DS denotes DeepSeek. ProgramBench reports raw pass rates. PaperBench is evaluated in the CodeDev setting. IQuest-CLIBench$^{\ddagger}$ is our proprietary built-in benchmark for evaluating human-agent collaboration and user experience (see Section 5 for details). Results with * are from our own testing.
Benchmark Results
IQuest‑Q1 sets new highs across coding and general AI benchmarks.
IQuest‑Q1 optimizes agentic CLI performance by combining a sparse Mixture‑of‑Experts architecture with a three‑stage training pipeline that emphasizes stable scaling and task‑specific synthesis. The following results quantify its capabilities across coding and general tasks.
IQuest‑Q1 attains the highest score of 89.7 on LiveCodeBench v6, the top‑ranked coding benchmark in the suite.