RecGPT-V3 Technical Report

Bowen Zheng, Chao Yi, Dian Chen, Gaoyang Guo, Han Zhu, Jiakai Tang, Jian Wu, Mao Zhang, Wen Chen, Yifan Lu, Yujie Luo, Yuning Jiang, Zhujin Gao, Bo Zheng, Dixuan Wang, Hao Fang, Jiancai Liu, Jing Yu, Ke Chen, Kewei Zhu, Mingke Xu, Wenjun Yang, Xunke Xi, Zile Zhou

RecGPT-V3 optimizes industrial-scale recommendation by replacing stateless history processing with persistent memory and latent reasoning.

How can we compress long-term user behavior into compact memory units and reason over them using latent tokens to improve recommendation accuracy while drastically reducing compute costs?

Large language model-based recommenders typically reprocess a user’s entire interaction history for every request, creating redundant computation and failing to carry forward long-term user understanding. RecGPT-V3 introduces a stateful Memory Hub that distills history into compact, evolving units, a hybrid-modal foundation model that reasons over both text and discrete item IDs, and latent intent reasoning that compresses verbose chain-of-thought into efficient latent tokens. In production A/B tests on Taobao, this architecture reduces serving compute by 52.4% while increasing GMV by 3.97%.

Paper Primer

The system replaces the "stateless" paradigm—where the model re-reads every past click—with a persistent Memory Hub. This hub acts as a structured, evolving cache that consolidates long-horizon behavior into succinct units, allowing the model to reason over a compact memory state plus only the most recent behavioral delta.

To bridge the gap between natural language intent and concrete item retrieval, the model uses a hybrid-modal foundation: it maps items to Semantic IDs (SIDs) that encode both semantic and collaborative signals. The model reasons jointly over these SIDs and text, effectively turning the LLM into a direct interface for the item space.

Finally, the system internalizes explicit chain-of-thought reasoning into compact latent tokens. This move is a compression-pivot: it replaces thousands of autoregressive reasoning tokens with a handful of learnable latent slots that remain decodable into human-readable rationales on demand.

RecGPT-V3 significantly improves both engagement and business conversion metrics while drastically reducing computational overhead.

Large-scale online A/B testing on Taobao's "Guess What You Like" feed. +3.97% GMV and +1.28% IPV, alongside a 52.4% reduction in end-to-end serving compute.

The Memory Hub effectively compresses user history without losing behavioral context.

Comparison of compute requirements between full-sequence processing and memory-augmented reasoning.

Why move from explicit chain-of-thought to latent intent reasoning?

Explicit reasoning is computationally expensive, often requiring thousands of tokens per request. Latent reasoning compresses these traces into a few tokens, reducing output token cost by 200× while maintaining the ability to decode the rationale for interpretability.

How does the hybrid-modal approach solve the "tag-to-item" bottleneck?

Previous models predicted natural-language tags, which are lossy and map to broad item sets. By reasoning in Semantic IDs (SIDs), the model generates identifiers that are natively compatible with the downstream retrieval system, creating a high-bandwidth channel between intent and item grounding.

RecGPT-V3 demonstrates that LLMs can function as efficient, stateful "brains" for industrial systems by internalizing reasoning and history, rather than treating every request as a fresh, stateless inference task.

Introduction

RecGPT‑V3 compresses behavior and replaces explicit reasoning with latent tokens for efficient, accurate recommendations.

Large language models are reshaping recommender systems from pure co‑occurrence matching toward reasoning about user intent. RecGPT‑V1 demonstrated this shift on Taobao, and RecGPT‑V2 scaled it with coordinated multi‑agent reasoning, both deployed in production with measurable user‑experience gains.

The series exposes three fundamental obstacles: (C1) stateless behavior modeling forces each request to reprocess the entire user history, wasting compute; (C2) a tag‑to‑item bottleneck loses ID‑level detail, widening the gap between intent and retrieved items; (C3) explicit chain‑of‑thought reasoning generates ~3 000 tokens per query, inflating latency beyond production tolerances.

RecGPT‑V3 addresses these issues with three innovations: a stateful Memory Hub that compresses long‑term behavior into structured memory units (cutting modeling cost by 55.8%); a Hybrid‑modal Foundation Model that lets the LLM reason jointly over natural‑language tags and Semantic IDs (SIDs); and Latent Intent Reasoning that collapses verbose rationales into compact latent tokens, lowering token output by 200× while remaining decodable.

**Figure 1.** Performance and efficiency of the RecGPT series on Taobao. Each generation brings further gains on CTR, IPV, TC, and GMV in both the Item Scene and Main Feed, while GPU cost drops steadily: RecGPT-V3 uses only 19% of RecGPT-V1's compute, a 52.4% reduction over RecGPT-V2.

RecGPT‑V3 delivers sizable accuracy lifts while slashing inference cost, proving that LLM‑based recommendation can be both high‑performing and production‑ready.

System Challenges and Goals

We expose inefficiencies of prior pipelines and motivate three core upgrades.

RecGPT‑V2 processes a user’s entire long‑term behavior at every inference step, forcing the Global Planner to re‑encode tens of thousands of tokens and consuming the bulk of the system’s compute. This stateless, one‑shot modeling also discards useful patterns that persist across sessions, such as repeat purchases or seasonal preferences.

To close this gap we target three fronts: a stateful Memory Hub that compresses and curates behavior into compact units, a hybrid‑modal foundation model that augments language with discrete SID tokens, and latent intent reasoning that folds explicit chain‑of‑thought into a few learnable latent CoT slots.

Together these upgrades promise dramatic gains in recommendation quality while slashing inference compute, enabling production‑scale deployment on a major e‑commerce platform.

**Figure 2.** Overview of RecGPT-V3. A Memory Hub distills a user's long-term behavior into incrementally curated memory units; conditioned on these units and recent behaviors, a planner and its multi-expert modules reason with the Hybrid-modal Foundation Model (natural language and Semantic IDs) and Latent Intent Reasoning (compact latent tokens) to predict the next item to interact with.

Structured Behavior Compression

Compress user behavior into a few structured units and keep them fresh with incremental updates.

User behavior histories can contain millions of events, making full‑sequence encoding prohibitively expensive for real‑time recommendation.

Compress a user’s long behavior sequence into a handful of memory units that each capture a distinct pattern, so the model can reason over a tiny, structured representation instead of millions of raw events.

How does Structured Behavior Compression differ from a naïve clustering of raw events?

Plain clustering would only output group centroids; Structured Behavior Compression also generates a natural‑language summary $s_k$, attaches rich metadata, and guarantees $K\ll N$ by design, enabling downstream models to consume a token‑level representation directly.

Cluster the six actions into two patterns: {“baby‑care”} and {“K‑pop fandom”}.

Assign pattern identifiers $p_1=$“Infant Care”, $p_2=$“K‑pop Fandom”.

Generate natural‑language summaries: $s_1=$“Frequent purchases of infant supplies for ages 0–12 months.”, $s_2=$“Dedicated fan of a K‑pop group, buying merch and streaming videos.”

Attach metadata: for $m_1$ store indices $\{1,2,6\}$, preferred brands $\{$“BabyCo”, “TinyToys”$\}$; for $m_2$ store indices $\{3,4,5\}$, preferred brands $\{$“SingBA”, “珍琢”$\}$.

Output $M=\{m_1,m_2\}$, reducing the token count from 6 to 2.

This toy example shows that a handful of units can capture both categorical patterns and rich textual preferences, dramatically shrinking the representation while preserving interpretability.

Even a compact memory becomes stale as user interests drift; without refresh the recommendation quality degrades.

Periodically refresh the compact memory by merging new behaviors into existing units and spawning fresh units for novel patterns, keeping the representation current without re‑encoding the entire history.

Why not simply rerun Structured Behavior Compression after every batch of new actions?

Re‑compressing the full history would re‑introduce the $O(N)$ cost that the whole system is designed to avoid; $G$ updates only the affected units, yielding $O(K+\Delta N)$ work, which is orders of magnitude cheaper because $K\ll N$.

Compare each new action to existing pattern embeddings.

“buy toddler walker” matches $m_1$ (Infant Care) → update $m_1$’s summary to include toddler toys.

“stream new K‑pop album” matches $m_2$ (K‑pop Fandom) → refresh $m_2$’s summary with the latest music interest.

“purchase gaming console” matches no existing pattern → create a new unit $m_3$ with pattern “Gaming” and summary “Enthusiastic gamer buying consoles and accessories”.

Retain $m_1$ and $m_2$ (they were updated) and add $m_3$; the curated memory becomes $M(t+\delta)=\{m_1(t+\delta), m_2(t+\delta), m_3\}$.

The example illustrates the three operation types (Update, Retain, New) that keep the memory both current and parsimonious.

**Figure 3.** Overview of the Memory Hub. Structured Behavior Compression distills a user’s long-term behavioral history into a compact set of structured memory units, reducing token usage by 94.5%. Evolving Memory Curation then keeps this memory current by selectively updating existing units and extracting new patterns from unmatched behaviors, yielding a condensed, traceable, and evolvable user representation for downstream recommendation.

Memory Hub Mechanics

Memory Hub incrementally curates user behavior into a compact, up‑to‑date memory.

The hub maintains a small set of latent memory units that summarise a user’s long‑term interests and are refreshed only when new evidence arrives.

Identify related subset: $\Delta B^{\text{rel}}_1=\{$“marathon nutrition supplements”$\}$, $\Delta B^{\text{rel}}_2=\emptyset$.

Apply Update to $m_1$: its summary becomes “outdoor running + marathon nutrition”.

Retain $m_2$ unchanged because $\Delta B^{\text{rel}}_2$ is empty.

Form novel set $\Delta B_{\text{novel}}=\{$“air‑fryer recipes”$\}$ and Extract a new unit $m_3$ with summary “kitchen gadgets”.

Combine: $M(t+\delta)=\{m_1', m_2, m_3\}$.

The hub updates only the affected unit, creates a fresh unit for the truly new interest, and leaves unrelated units untouched, keeping the memory both current and compact.

Compute the behavioral delta $\Delta B(t,t+\delta)$ from the latest user interactions.

For each existing unit $m_k$, check whether $\Delta B^{\text{rel}}_k$ is non‑empty.

If non‑empty, invoke $\text{Update}$ to refresh $m_k$; otherwise retain $m_k$ as‑is.

Collect all behaviors not assigned to any unit into $\Delta B_{\text{novel}}$.

Run $\text{Extract}$ on $\Delta B_{\text{novel}}$ to create new memory units.

Form the updated memory $M(t+\delta)=M_{\text{update}}\cup M_{\text{new}}$ for downstream recommendation.

How does the Memory Hub differ from recomputing the user state from the full interaction history at every request?

Recomputing requires feeding the entire raw sequence through the model each time, which is $O(T)$ in sequence length. The Memory Hub instead keeps a fixed‑size set of $K$ latent units and only updates those that receive relevant new evidence, turning the per‑request cost into $O(K)$ plus a cheap delta extraction.

Hybrid-Modal Foundation Model

Hybrid‑Modal Foundation Model replaces coarse tag bottleneck with SID‑augmented tokenization for precise, high‑bandwidth retrieval.

Standard LLMs emit free‑form tags that must later be grounded by a retriever. Those tags are coarse, so many distinct items collapse onto the same phrase, weakening the link between the model’s intent and the items actually returned.

The model augments the LLM’s text vocabulary with discrete SID tokens, letting it speak directly in item identifiers instead of vague natural‑language tags.

How does this differ from simply adding raw item IDs to the LLM’s vocabulary?

Raw IDs are arbitrary and carry no semantic similarity; the hierarchical SID codes are learned from multimodal embeddings, so items that are semantically close share the same coarse code. This structure lets the model generalize to unseen items that belong to the same semantic cluster.

Encode Item A’s text, image, and side info → embedding $H_A$.

Quantize $H_A$ at level 1 → coarse code 1 (shared prefix).

Quantize $H_A$ at level 2 → fine code 2.

Form SID for A: <$C_1$><$C_2$>.

Repeat for Item B: same level‑1 code 1, different level‑2 code 3 → SID <$C_1$><$C_3$>.

The shared coarse token signals that A and B belong to the same semantic cluster, while the differing fine token lets the model distinguish them when needed.

**Figure.** The architecture of the Hybrid-modal Recommendation Foundation Model, illustrating the process from item content and collaborative pairs through a multimodal encoder and residual quantization to generate SID tokens, followed by a two-stage pre-training process involving SID grounding and instruction tuning.

By coupling high‑bandwidth SID tokens with the LLM’s expressive language, the Hybrid‑Modal Foundation Model delivers precise item grounding without sacrificing the model’s ability to reason over open‑ended user intent.

SID Token Integration

We ground Semantic ID tokens in item semantics through a two-stage continual pre-training and instruction-tuning pipeline.

SID tokens are discrete identifiers that bridge the gap between natural language and collaborative filtering by representing items as compact, learnable tokens within the model's vocabulary.

To integrate SID tokens effectively, we employ a two-stage pipeline: continual pre-training to ground tokens in item semantics, followed by instruction tuning to enable cross-task application. This approach balances domain-specific learning with the preservation of the backbone's general-purpose capabilities.

Continual Pre-training: Ground SID tokens by associating them with textual item attributes (titles, categories) while blending general-domain data to prevent catastrophic forgetting.

Instruction Tuning: Train the model on diverse alignment tasks, including SID-text translation and sequential recommendation, to apply SID tokens in practical scenarios.

General-Domain Preservation: Maintain 20% open-domain tasks during instruction tuning to ensure the model retains its ability to follow complex prompts and produce structured outputs.

**Table 3.** SID-related alignment tasks and their input-output mappings. The upper block covers SID-text bidirectional translation; the bottom row is SID-based sequential recommendation.

Latent Intent Reasoning

Compressing explicit reasoning into a few latent tokens keeps accuracy while slashing cost.

Explicit chain‑of‑thought (CoT) reasoning boosts recommendation accuracy, yet generating thousands of rationale tokens autoregressively creates prohibitive latency at billion‑user scale. The core question is whether we can keep those accuracy gains without paying the token cost.

Think of a long essay being distilled into a handful of bullet‑point summaries that still capture the argument – the latent tokens are those bullet points, learned to stand in for the full chain‑of‑thought.

How does Latent Intent Reasoning differ from ordinary knowledge distillation?

Distillation typically transfers a teacher’s logits into a smaller student network, preserving only output probabilities. Latent Intent Reasoning instead compresses an entire multi‑step reasoning trace into a set of learnable tokens that are later used as explicit conditioning for the downstream prediction, and those tokens can be decoded back into a human‑readable rationale.

Initialize two latent embeddings z₁, z₂ randomly.

Apply the single‑segment reconstruction loss: map z₁ → first three steps, z₂ → last three steps.

Compute reconstruction error; back‑propagate to adjust z₁, z₂.

After training, feeding x produces z ≈ [⟨reasoning‑summary‑1⟩, ⟨reasoning‑summary‑2⟩] that together encode the whole six‑step trace.

During inference, the model reads only the two latent tokens instead of generating six autoregressive steps.

Even with K = 2, the latent tokens retain enough information to reconstruct the full six‑step rationale, demonstrating that reasoning can be compressed without losing fidelity.

Stage 1 – Explicit‑to‑Implicit CoT Alignment: run the teacher CoT model to produce R, then train the latent tokens z to reconstruct R via three tasks (single‑segment, multi‑segment, full‑trace).

Stage 2 – Reinforcement Learning from Ranking Feedback: keep the latent tokens fixed, fine‑tune the downstream policy p(y|x,z) using online ranking signals (e.g., CTR, GRPO) to maximize business metrics.

**Figure 5.** Overview of Latent Intent Reasoning. Three reconstruction-based alignment tasks compress an explicit chain-of-thought trace into a short sequence of learnable latent tokens $z$ that faithfully encode it. A two-stage post-training pipeline then internalizes this reasoning into the model: *Explicit-to-Implicit CoT Alignment* first distills teacher traces into the latent tokens, and *Reinforcement Learning from Ranking Feedback* then refines the policy against online business rewards.

Training Latent Reasoning

Compress explicit reasoning traces into a few latent tokens and train them to reconstruct the full chain of thought.

Explicit chain‑of‑thought (CoT) traces are costly: a typical reasoning episode spans ~2,300 tokens, inflating inference latency and GPU memory.

We slice a long reasoning trace into a handful of fixed‑size windows and give each window a dedicated latent token, then teach those tokens to behave like ordinary language embeddings.

Compute $N/C = 45/10 = 4.5$, round down to 4 segments.

Apply the ceiling $K_{\text{max}}=5$, so $K = \min(4,5) = 4$ latent tokens are allocated.

Each token will cover up to 10 steps; the last token covers the remaining $45 - 3\cdot10 = 15$ steps, which exceeds $C$ but is allowed because the budget stops at $K_{\text{max}}$.

Insert the four latent tokens into the trace, yielding a mixed sequence of length $|x| + 4 + |y|$.

Train with $L_{\text{warm}}$; the gradients adjust the four latent vectors so that, when the model predicts the surrounding text, they behave like ordinary word embeddings.

Warm‑up guarantees that latent tokens are *compatible* with the pretrained vocabulary, preventing a representation mismatch that would otherwise cripple downstream alignment.

After warm‑up we teach each latent token to *store* the exact reasoning segment it replaces by asking the model to reconstruct masked text at varying granularities.

Single‑segment mask: choose $J=\{2\}$, replace $R_2$ by $z_2$, keep $R_1$ and $R_3$ as text.

Model receives $c(\{2\}) = (x, R_1, z_2, R_3, y)$ and a prompt “recover the hidden segment”.

Loss $L(\{2\})$ pushes $z_2$ to contain enough information to regenerate the 7‑step $R_2$ given its neighbors.

Multi‑segment mask: $J=\{1,3\}$, context becomes $(x, z_1, R_2, z_3, y)$; the model must reconstruct both $R_1$ and $R_3$ simultaneously.

Full‑trace mask: $J=\{1,2,3\}$, context reduces to $(x, z_1, z_2, z_3, y)$. The model must generate the entire original trace $R$ from latent tokens alone.

Success on the full‑trace task proves that the latent sequence is a *sufficient* encoding of the original reasoning, preserving interpretability while cutting token count dramatically.

With the alignment curriculum in place, we move to the post‑training pipeline, which consists of two stages: (1) explicit‑to‑implicit CoT alignment that compresses teacher‑generated reasoning, and (2) reinforcement learning from ranking feedback that tunes the latent model toward business metrics.

Distill reasoning traces from a strong teacher model into explicit CoT outputs (≈ 2,300 tokens each).

Apply the latent‑token partitioning (Section 4.1) with $C=20$ and $K_{\text{max}}=10$, yielding at most ten latent tokens per trace.

Warm‑up the latent embeddings using $L_{\text{warm}}$ so they sit in the pretrained embedding space.

Train on the three reconstruction tasks (single‑segment, multi‑segment, full‑trace) using the multi‑granularity loss $L(J)$.

Mix in general‑reasoning data (≈ 70 %) and tag‑prediction data (≈ 8 %) to prevent catastrophic forgetting (see Table 5).

**Table 5.** Training data mixture in Stage 1.

Generate a tag sequence $y=\{t_1,\dots,t_m\}$ for each user query.

Retrieve candidate items using $y$, score each with the production ranking model, and compute the dense CTRScore reward $r_{\text{ctr}}(y)$ (Eq. 16).

Apply Constrained Reward Shaping: the CTRScore is accepted only if alignment, diversity, and length thresholds are satisfied.

Optimize the policy with GRPO, using group‑relative advantages to reduce variance.

The table illustrates three tasks: Single-Segment, Multi-Segment, and Full-Trace, showing the corresponding input context $c(\mathcal{J})$ and target masked segments $R_{\mathcal{J}}$.

How does this latent‑token alignment differ from simply truncating a long CoT trace?

Truncation discards information arbitrarily, whereas the alignment curriculum forces each latent token to *reconstruct* its assigned segment (and, jointly, the whole trace). The model therefore learns a *structured* compression that remains decodable, not a lossy cut.

Why not use the standard HitRate reward directly in reinforcement learning?

HitRate is binary and identical for all rollouts within a group, yielding zero gradient (reward sparsity). By averaging the CTRScore from the production ranker we obtain a dense signal that reflects the actual downstream metric users experience.

Experimental Setup

RecGPT‑V3 beats RecGPT‑V2 on every online metric, with a 7.51 % GMV lift.

RecGPT‑V3 outperforms RecGPT‑V2 on all online A/B‑test metrics, achieving the largest gain of +7.51 % in gross merchandise value (GMV) for the item scenario.

Table 6 shows consistent positive deltas for IPV, CTR, PV, DAU, TC, and GMV across both item and feed evaluations.

We allocated 1 % of traffic to each group, evaluated under two scopes: a grid‑based item recommendation and a mixed‑content feed stream.

**Table 6.** Results and Analysis. Table 6 summarizes the online A/B test results. RecGPT-V3 consistently outperforms RecGPT-V2 across all metrics in both evaluation scenarios.

**Table 7.** Human evaluation of memory unit quality. “Behavior Pattern” measures whether the assigned pattern identifier correctly categorizes the user’s behavioral cluster; “Behavior Index” measures whether the representative indices correctly point to interactions belonging to the identified pattern.

Memory Hub Evaluation

Memory hub slashes compute cost while preserving high‑fidelity behavior representations.

The memory hub cuts overall compute cost by 55.80 % versus the RecGPT‑V2 baseline.

Table 8 shows the Global Planner’s cost reduction when the hub replaces the full‑sequence input.

The table compares the computational cost of RecGPT-V2 and RecGPT-V3 systems. RecGPT-V2 is listed with a "Full sequence" cost of 100%. RecGPT-V3 is broken down into "Per-inference" (33.43%) and "Memory curation" (10.77%) components, resulting in a "Total" cost of 44.20%.

Human annotation finds the memory units 82.89 % accurate in representing user behavior patterns.

Table 7 reports accuracy over 2,514 annotated patterns.

High index accuracy ensures downstream reasoning can retrieve the exact source events the hub references, making the compressed representation trustworthy for recommendation.

**Figure 7.** SID–text semantic alignment across four bidirectional translation tasks.

Removing general‑domain data causes catastrophic collapse, dropping GSM8K performance to 4.70 %.

Benchmarks in §5.3.1 report near‑zero scores without the mixed data.

General‑domain data lifts category‑level HR@30 by 26.2 % (0.2250 → 0.3050).

Table 9 compares recommendation quality with and without the data.

**Table 9 | Downstream recommendation quality comparison.**

Latent Reasoning Performance

Latent reasoning matches quality while cutting inference cost dramatically.

Latent reasoning speeds inference 3.46× while delivering recommendation quality comparable to explicit chain‑of‑thought.

Table 11 shows total time 295 s vs 1,020 s (3.46× faster) and HR@30 0.3462 vs 0.3508 for explicit CoT (Table 10).

Post‑training ablations (Table 10) reveal that adding the latent‑reasoning module to the hybrid‑modal foundation model raises HR@30 from 0.3050 (explicit CoT) to 0.3462, and RL further pushes it to 0.3693, surpassing explicit CoT on both HR@30 and CTR.

**Figure 6.** General capability evaluation across four benchmarks. The hybrid-modal foundation model (w/ General-Domain Data) preserves most of the backbone's capabilities, while removing general-domain data (w/o General-Domain Data) leads to catastrophic collapse.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers