Researchers Find a Way to Expose AI Models’ Hidden Reasoning — and Their Secrets

Computer scientists have uncovered a technique that can pull back the curtain on frontier AI systems’ hidden reasoning, showing that some models may leak private data and possibly even reveal clues about how rival models were trained. The finding matters because it affects major API-based systems from OpenAI, Anthropic and Google, and because it suggests that today’s AI products may be more transparent — and more vulnerable — than their makers intended.

The research, published this week by a team from the University of Tübingen, the Max Planck Institute, MATS Research and the security firm Snyk, shows that an attacker can sometimes recover a model’s internal chain-of-thought style output by interacting with a smaller version of the same system. In one demonstration, the technique also exposed secrets such as passwords and API keys, although the companies involved have since patched the issue.

Beyond the security implications, the work has also intensified an already heated debate over AI “distillation” — the process of transferring capabilities from one model to another. The researchers say their method may make it possible to infer whether certain Chinese open-weight models were trained on reasoning traces from US-made proprietary systems, though they stress that the evidence is suggestive rather than conclusive.

What did researchers discover?

The team found a way to infer hidden reasoning traces from some frontier models by exploiting how AI companies distribute model outputs through application programming interfaces, or APIs. In plain terms, they showed that a model’s private working notes are not always as private as developers assumed.

Large language models often solve difficult prompts by breaking them into intermediate steps. Those steps — sometimes called chain of thought — are usually kept out of public output because companies want to limit misuse and prevent competitors from copying the reasoning patterns behind their products. Even so, those traces can still travel through the system in encrypted or otherwise protected form for technical reasons.

According to the researchers, feeding those protected reasoning traces into a smaller related model can sometimes force the model family to expose the hidden text. The smaller sibling, which has typically undergone less alignment training, appears more willing to reproduce what the larger model tried to conceal.

Why does this matter for AI security?

It matters because reasoning traces are not just a curiosity; they can become a source of leakage. In the team’s testing, the method was able to reveal sensitive material, including API keys and passwords, if those items had been embedded in captured reasoning output from a user’s machine.

That creates two distinct risks. First, there is the immediate privacy and security problem of secret data spilling out of a model’s internal process. Second, there is a strategic risk: if hidden reasoning can be recovered at scale, it becomes easier to imitate high-end models or to train competing systems on information their creators meant to keep hidden.

“All major frontier model providers we tested share this vulnerability,” said Alexander Panfilov, a computer scientist at the University of Tübingen who worked on the project. He said the flaw can lead to personal information exposure and enable large-scale reasoning-distillation attacks.

How does the attack work?

The method takes advantage of the fact that many AI companies offer multiple versions of the same model family. A powerful version may be wrapped in stricter controls, while a smaller or cheaper version of the same architecture may have weaker safety tuning and fewer guardrails. That difference is what the researchers exploited.

In their experiments, the team sent reasoning traces captured from proprietary systems to a smaller model variant with a similar internal structure. Because the weaker model was less resistant to revealing its own inner process, it could be coaxed into reproducing or approximating the concealed chain-of-thought content.

Computer-security researchers say the approach is clever because it uses the model family itself as the weak point.

“The idea of swapping out messages to a weaker model variant which has the same decryption key but weaker alignment is very cool,” said Florian Tramer, a computer scientist at ETH Zürich who studies security. “It is definitely becoming an issue.”

The researchers notified OpenAI, Anthropic and Google last month. Each company has since modified its API in an effort to reduce the risk of replay-style attacks. The team says the specific trick for retrieving private information no longer works as it did before, although some reasoning traces may still be recoverable by related means.

Which AI models were affected?

The vulnerability was identified in frontier models from OpenAI, Anthropic and Google that were accessed through APIs. The research did not focus on consumer chat interfaces; it examined the underlying service behavior exposed to developers and other API users.

On the open-weight side — meaning models that can be downloaded and run locally — the findings were more nuanced. The team says the Chinese model Kimi K3 from Moonshot AI produced notably similar outputs to hidden reasoning traces associated with Claude Opus 4.8 and GPT 5.6 Sol for some prompts.

That similarity, the authors argue, may be consistent with distillation, but they are careful not to overstate the conclusion. Their paper says the results do not prove causation, only that the resemblance is striking enough to raise questions.

By contrast, two other open-weight systems the researchers checked — DeepSeek and Inkling from the US firm Thinking Machines — did not show the same level of reasoning similarity with Claude Opus.

Comparison of models and findings

What is AI distillation, and why is it controversial?

Distillation is a standard machine-learning technique used to transfer knowledge from one model into another. In practice, developers use outputs from a powerful teacher model to help train a smaller, cheaper student model. That approach can make advanced capabilities more efficient and easier to deploy.

The controversy comes from the growing fear that one company’s research investment can be used to bootstrap another company’s competing system. In the current geopolitical climate, that issue is especially sensitive when the companies are in the US and China, where each side worries about the other gaining an edge by quietly copying hard-won progress.

Over the past year, that concern has surfaced repeatedly in Washington. OpenAI told US lawmakers in February that DeepSeek appeared to have copied one of its models to build the reasoning system known as R1. In June, Anthropic said Alibaba had systematically distilled its models to help build Qwen, its own family of AI systems.

There is still no direct evidence in this new study that any Chinese lab used the specific technique to distill from US models. Still, the researchers say the ability to uncover hidden reasoning could make that kind of reverse engineering more practical than many observers assumed.

How strong is the evidence that Chinese models were distilled from US models?

The evidence is suggestive, not decisive. The authors found a resemblance between Kimi K3’s outputs and the concealed reasoning traces of certain closed models, but they explicitly say that similarity alone does not establish a causal distillation relationship.

In other words, matching behavior can point to imitation, but it can also arise from other causes: shared training data, similar prompt structures, or convergent optimization choices. Because of that, the research opens a line of inquiry rather than closing the case.

One important detail is that the study’s comparison involved only 90 questions per model, which is enough to spot patterns but not enough to settle a broad strategic claim about national AI ecosystems. Even so, researchers say the result is notable because it suggests hidden reasoning may be a richer source of training data than companies anticipated.

Kyle Miller, a researcher at the Center for Security and Emerging Technologies, said it remains unclear how much distillation truly changes the balance between US and Chinese labs. He argued that Chinese companies likely have enough technical capability to build leading models from scratch if needed, and that cutting off distillation would not necessarily reshape the competitive landscape in a dramatic way.

Why are some experts defending distillation?

Because distillation is not only common, it is also one of the reasons AI systems improved so quickly in recent years. It helps developers compress large-scale intelligence into a more deployable package, making models faster, cheaper and easier to distribute.

That is why some AI leaders warn against treating the practice as inherently suspicious. Meta CEO Mark Zuckerberg argued in a blog post this week that distillation is a core part of how the open-source ecosystem functions, and that over-restricting it could leave the US at a disadvantage.

Academic researchers also say blanket crackdowns could slow the field in ways that are not easy to reverse.

Yarin Gal, a computer scientist at Oxford University, said distillation is widely used and has helped speed up AI progress. He warned that if everyone starts blocking everyone else from distilling, the overall pace of advancement could suffer.

What did the researchers do to test the idea?

They ran a structured set of prompts across several models and compared the results to hidden reasoning traces captured from proprietary systems. The core test was simple: if a model was nudged with the opening words of a concealed chain-of-thought, would it generate similar reasoning or answer patterns?

For some models, the answer was yes. In particular, Kimi K3 showed a pronounced similarity in some of the comparisons, making it stand out from the field. That pattern is what prompted the team to discuss the possibility of distillation from hidden reasoning traces.

They also explored the privacy angle by looking at whether the same method could reveal secrets from reasoning content. It could, which is why the companies moved quickly to patch the weakness once the issue was reported.

Timeline of the key events

What did the companies say?

Anthropic said it welcomed independent research and had already begun putting in place short-term mitigations for the behavior described. The company also said the researchers did not recover encryption keys, access its infrastructure or retrieve personal data from Anthropic systems.

Google and OpenAI did not comment for the report. Moonshot AI and Z.ai also did not respond before publication.

The response pattern is consistent with a common frontier-AI dynamic: researchers disclose a problem privately, companies patch quickly, and the public learns about the underlying weakness only later, once the paper is published.

What happens next?

The most immediate consequence is likely to be a security review across API-based AI products. If the technique can expose internal reasoning or sensitive tokens, providers will face pressure to redesign how they handle encrypted traces and replay behavior.

The broader consequence may be political. Distillation has already become part of the US-China AI rivalry, and a method for recovering hidden reasoning could sharpen those accusations on both sides. It may also encourage companies to be more conservative about showing internal model deliberation at all.

That could have a side effect: as AI systems become more capable, they may also become less interpretable. Companies may decide that the safest way to protect model internals is to hide even more of the reasoning process from users and developers.

For now, the main lesson is that frontier models are not sealed black boxes. Even when companies try to hide how they think, researchers continue to find ways to make those hidden steps visible — with consequences for privacy, competition and the future of AI development.

• The technique targets hidden chain-of-thought style reasoning inside frontier AI models.
• It exposed private data such as API keys and passwords before vendors patched the issue.
• The study raises, but does not prove, the possibility of distillation from US models into some Chinese open-weight systems.
• OpenAI, Anthropic and Google have already implemented mitigations.
• Experts disagree on whether limiting distillation would meaningfully change the AI race.