Stealing Reasoning Traces from Proprietary LLM APIs

Alexander Panfilov, David Schmotz, Ilia Shumailov, Luca Beurer-Kellner, Joachim Schaeffer, Ameya Prabhu, Jonas Geiping, Maksym Andriushchenko

Encrypted reasoning traces in LLM APIs are globally compatible, allowing weaker models to act as decryption oracles.

How can adversaries extract and exploit the hidden "reasoning traces" (chains of thought) generated by proprietary LLM APIs?

Modern LLM providers hide internal "chain-of-thought" reasoning in encrypted blobs to prevent intellectual property theft and data leakage. These encrypted blobs are globally compatible across different models and users within a provider's ecosystem, allowing an attacker to inject a trace from a secure frontier model into a less-guarded, weaker model. The weaker model acts as a decryption oracle, transcribing the hidden reasoning into plaintext upon request, which enables large-scale distillation, credential theft, and invisible prompt injection.

Paper Primer

The core vulnerability is a security asymmetry: while frontier models are heavily hardened against direct extraction, their "sibling" models—optimized for speed and cost—lack these defenses. The authors exploit this by porting an encrypted reasoning trace into a weaker model and using a simple jailbreak prompt to force the model to output the trace verbatim.

The attack enables scalable, verbatim extraction of proprietary reasoning traces across major API providers.

The authors successfully decoded 315,320 reasoning blocks scraped from public repositories, recovering hundreds of sensitive artifacts. The extraction recovered 367 PII artifacts and 182 credentials, including 62 API keys and 33 passwords, from genuine user sessions.

This vulnerability is not limited to distillation; it creates a persistent privacy risk. Because users cannot read the encrypted reasoning blocks, they often inadvertently publish sensitive data (like API keys or PII) that the model silently internalized during its hidden reasoning process.

Why is this approach more effective than directly jailbreaking the frontier model?

Directly attacking a frontier model requires bypassing both model-level refusal training and system-level output filters. By using a weaker, more compliant model as a "fuzzy" decoder, the attacker bypasses these safeguards entirely using a single, fixed extraction prompt.

What is the scope of this vulnerability?

The vulnerability stems from the stateless API design, which relies on client-side storage of reasoning blobs. As of July 2026, this allowed traces to be replayed across different sessions, users, and model scales within the same provider's ecosystem.

Abstract

We expose a cross‑model API flaw that lets attackers extract hidden reasoning from LLMs.

Large language model providers now return step‑by‑step reasoning as encrypted blocks to the client, which the client forwards with each request. We discover that these encrypted blocks are fully compatible across sessions, users, and models, enabling a scalable decryption jailbreak by injecting a trace from a strong model into a weaker one, forcing the latter to output the trace in plaintext.

Exploiting this flaw yields four attack vectors: it bypasses anti‑distillation safeguards to extract proprietary reasoning from models such as Anthropic, OpenAI, and Google. It also allows large‑scale private data extraction, as we decoded 315,320 reasoning blocks from public logs, recovering 367 PII artifacts and 182 credentials. Finally, it reveals hidden hazardous content and enables invisible prompt injections that poison public agentic rollouts, prompting us to propose concrete cryptographic and system‑level mitigations.

The Rise of Reasoning Models

We expose why hidden reasoning traces create a new, opaque attack surface for LLMs.

Frontier large language models have evolved into “reasoning models” that emit extensive internal chains of thought before producing a visible response. These hidden Reasoning Traces often contain far more dense and sensitive information than the final output, including intermediate hypotheses, tool outputs, user data, and contextual secrets. Exposing them in plaintext makes proprietary systems vulnerable to model distillation and can unmask safety or refusal mechanisms.

A Reasoning Trace is the model’s internal monologue generated before the final answer, recording step‑by‑step thoughts.

The model generates a thinking string: “Factoring 8139881 by testing divisibility against small primes: 3, 7, 11, 13, 17 …”.

It then produces the final text field with the answer “The largest prime divisor is 5003.”.

This example shows that the trace leaks the full factoring process, exposing intermediate calculations that are not visible to the user.

Modern Application Programming Interface (API) providers—including Anthropic, OpenAI, and Google—have deprecated plaintext reasoning to close this attack surface. A security asymmetry exists: heavily safeguarded frontier models (e.g., Claude Opus 4.8, GPT‑5.6 Sol) enforce strict refusal training, while their weaker siblings (e.g., Claude Haiku 4.5, GPT‑5.6 Luna) lack such defenses. By capturing an encrypted reasoning blob from a strong model and injecting it into a weaker model, an attacker can force the weaker model to decode and transcribe the hidden trace verbatim.

The shift from standard LLMs to reasoning‑heavy architectures creates a new, opaque attack surface that attackers can exploit at scale.

Decoding Reasoning at Scale

We exploit cross-model reasoning compatibility to extract hidden reasoning traces using weaker, more compliant decoder models.

Modern LLMs generate hidden reasoning traces that are cryptographically protected to prevent unauthorized distillation. We demonstrate that these traces are vulnerable to extraction by exploiting the broad compatibility of reasoning blocks across different models within the same provider ecosystem.

A Thinking Signature is the opaque, base64-encoded envelope that carries a model's hidden reasoning trace between API calls without requiring server-side storage.

The security of these traces relies on the assumption that reasoning blocks are tied to specific contexts. However, our analysis reveals that providers often use a single global key for encryption, enabling three levels of compatibility that attackers can exploit.

**Figure 1.** Decoding reasoning traces in Anthropic, OpenAI and Google APIs. Top: Reasoning-trace extraction in two API calls. An Opus 4.8 request (top left) returns a signed thinking block along with a thinking summary. Sending just the thinking signature from Opus 4.8 to a Haiku model and requesting it to output its own reasoning in <thinking-copy> tokens makes Haiku transcribe the Opus 4.8 hidden reasoning (top right). Bottom: Extracted traces closely track the number of generated thinking tokens. We evaluate each model on 120 Codeforces programming problems and record the number of thinking tokens generated by the source model, as reported by the API (x-axis). We then reconstruct the reasoning trace from its signature, pass it as an input message to the same model that generated encrypted reasoning, and measure its API-reported token count (y-axis).

Generate an encrypted reasoning trace by querying a capable target model.

Identify a weaker, compatible decoder model that supports assistant-turn prefilling or injection.

Inject the target's Thinking Signature into the decoder's context window.

Apply an ad-hoc jailbreak prompt to coerce the decoder into transcribing the injected reasoning token-by-token.

Verify faithfulness by comparing the extracted token count against the API-reported thinking token count.

**Figure 2.** Injection schemes exploiting in- and cross-session compatibility. We find that available form of thought injection depends on the provider and exact model. **Current-turn injection** places the thought in the current assistant turn, so the model continues its visible answer straight from it; as of July 2026 it is accepted by every GPT and Gemini model we tested and by the 4.5 generation of Claude. In addition, Sonnet 4.5, Haiku 4.5, and all Gemini models accept prefilling of the assistant turn's visible output. **Past-turn injection** is available only against models that do not omit previous reasoning blocks (e.g., Sonnet 5, Opus 4.8, Fable 5, and the GPT-5.6 series).

**Table 1.** Cross-model compatibility of encrypted reasoning. As per July 2026. Row: the source model that produced the encrypted reasoning block; column: the target model receiving the injected reasoning. A $\checkmark$ indicates that, for this combination, the target model interacts with the injected thought. **Claude:** the thinking traces of any model can be replayed by any other, except Fable 5’s thoughts. **GPT:** the GPT-5.6 series can replay the traces of all earlier model generations. **Gemini:** the thinking traces of any model can be replayed into any other.

First-Party Attack Vectors

First‑party attacks steal encrypted reasoning via API calls without querying the frontier model.

First‑party attack vectors leverage the model’s own API to retrieve encrypted reasoning traces, enabling downstream attacks without direct queries to the frontier model. We detail two concrete instantiations: distillation attacks and jailbreaking attacks.

An adversary uses their own API calls to capture the encrypted reasoning block that the model emits as part of its response, then replays it into a cheaper decoder model.

How is a First‑Party Attack different from a traditional black‑box model‑extraction attack?

Traditional black‑box attacks query the frontier model directly and learn from its final outputs, whereas a First‑Party Attack harvests the model’s own encrypted reasoning via its API and replays it into a separate decoder, avoiding any direct interaction with the protected model.

The attacker replays captured encrypted reasoning into a small decoder model, then uses the decoder’s outputs to train a student model that inherits the original model’s reasoning capabilities.

Why does extracting full reasoning traces give a stronger student model than answer‑only distillation?

Full traces expose the intermediate deductions and decomposition steps, allowing the student to imitate the teacher’s problem‑solving process directly, whereas answer‑only distillation forces the student to infer those hidden steps from the final answer alone.

The attacker extracts reasoning about a harmful topic while the visible answer is benign, then decodes the reasoning to obtain the prohibited information.

Why does reasoning extraction bypass output‑filtering safeguards?

Filters are applied to the final visible answer, but the model’s internal chain‑of‑thought is generated without those constraints; extracting the encrypted reasoning therefore reveals content that the filter never saw.

Third-Party Attack Vectors

We reveal hidden third‑party attacks that extract secrets and inject malicious behavior via encrypted reasoning traces.

The paper’s premise is that LLMs emit hidden reasoning traces which, if accessed by a third party, can be weaponized to steal secrets or embed malicious instructions.

A third‑party attacker either reads another user’s hidden reasoning or plants a malicious hidden step, then replays it in a new session.

Prefilling a model’s hidden reasoning with tokens from another model steers the subsequent visible response toward the prefilling model’s style.

**Figure 3.** Prefilling Kimi K3’s reasoning changes the style of its visible responses. In this example, we observe that prefilling a small number of Claude-generated reasoning tokens into Kimi K3’s reasoning trace shifts its final output to closely match Claude’s. In all cases, the visible response is free-form generation and is not itself prefilled. We quantify this phenomenon in Appendix B.

0.3 % (1,028) of the 315,320 decoded thinking blocks contain at least one privacy leakage.

Decoded from 6,708 publicly available agent trajectories collected from GitHub and Hugging Face.

4.9 % (328) of the 6,708 sessions leak at least one real sensitive item in their hidden reasoning.

Session‑level analysis of the same public trace corpus.

From genuine user sessions we recovered 62 API keys, 8 passwords, 24 access tokens, 7 private keys, 30 personal emails, and 6 non‑localhost IP addresses.

Manual inspection of the 328 leaking sessions identified the listed credential types.

**Figure 6:** Distinct artifacts recovered from reasoning blocks scraped from publicly available user-posted traces, grouped into three headline categories (all sources; see Appendix D).

64 of the 704 artifacts recovered from reasoning are completely absent from the visible chat history (Table 4).

Cross‑referencing decoded hidden traces with the public conversation logs.

Prompt‑injection attacks demonstrate that an adversary can embed a malicious instruction in an encrypted reasoning block, which later replays in a different session and causes the model to exfiltrate data or perform unwanted actions.

Mitigations and Limitations

Scope, ethics, and mitigation strategies for encrypted reasoning extraction.

Frontier LLMs emit hidden reasoning traces that can be extracted via API‑level injection, exposing private data and model behavior.

Our evaluation is limited to the API versions and reasoning models from Anthropic, OpenAI, and Google available in early July 2026; proprietary cryptographic changes could invalidate the attacks, and we lack ground‑truth traces to verify every extracted token.

We disclosed the vulnerabilities to the affected providers, Microsoft, and Hugging Face before publication; all acknowledged receipt, and subsequent attempts to repeat the attacks failed.

Extraction of 367 PII items and 182 credentials was performed in an isolated environment, with all secrets deleted after automated classification to prevent further leakage.

We advise researchers and developers to strip reasoning blocks from any released agentic logs and to avoid committing raw API transcripts containing signatures to shared repositories.

**Table 2.** Leakage vectors mapped to protections and processes. Each tick states what the mitigation supplies; the final column records the operational steps that deliver it.

Adopt server‑side storage of reasoning traces or bind encrypted traces to session identifiers to block extraction while balancing added overhead.

Conclusion

Reasoning models raise privacy‑security trade‑offs as encrypted traces hide user data while exposing new attack surfaces.

Current API designs place encrypted reasoning blocks on the client side to cut server storage costs, but the broad cross‑compatibility of these blocks unintentionally opens decryption channels that enable model distillation and other attacks.

As reasoning models become embedded in increasingly complex workflows, they will routinely handle ever larger volumes of private and sensitive user data, creating a clash between pervasive data collection and encrypted, illegible reasoning.

When such hidden chains of thought consume personal information or API keys, users cannot see what data is stored, where it is retained, or how it influences model actions; because the blocks are opaque, conventional scrubbing of private data from logs no longer works, leaving privacy violations undetectable and raising severe compliance and security risks.

To address this dilemma, providers should at minimum disclose whenever personally identifiable information is absorbed into hidden reasoning, and clearly describe the cryptographic guarantees that prevent its leakage; they must also treat the security of legacy or less‑capable models as a critical weak point, since exploits there can bypass safeguards of more advanced systems.

In short, an architecture that hides a user’s own data from the user while leaving it vulnerable to extraction offers neither privacy nor security.

Acknowledgments

The authors acknowledge contributors, support staff, and funding sources.

The authors thank, in alphabetical order, Albert Catalán‑Tatjer, Andy Zou, Cheng Zhang, Derck Prinzhorn, Edoardo Debenedetti, Hanna Foerster, Jeanne Salle, Joschka Braun, Mikhail Terekhov, Roland S. Zimmermann, Sail Wang, Shashwat Goel, and Yiren Zhao for valuable feedback and discussions. AP and JS thank Perusha Moodley, Ning Yang, and the MATS team for their support and administrative assistance, while AP and DS acknowledge the International Max Planck Research School for Intelligent Systems (IMPRS‑IS). AmP acknowledges funding from the Federal Ministry of Research, Technology and Space (BMFTR) (FKZ: 16IS24085B) and from the Good Ventures Foundation.

Reproducibility Statement

We disclose the cost and current reproducibility status of our experiments.

As of August 2026, the results presented in Figure 1 are no longer reproducible with attacks described in Section 2.4 and Appendix C because of mitigations implemented by providers following our disclosure.

All experiments were conducted using open- and closed-source models accessed via Application Programming Interface (API). In total, we spent approximately \$30,000 on API credits.

Read the original paper

Open the simplified reader on Paperglide

Browse all simplified papers