OpenAI Disrupts an AI Model Distillation Campaign

OpenAI says it has dismantled a coordinated campaign to extract the protected reasoning from its models and use it to train or improve other systems. The activity began on July 1 and peaked at 16,000 requests from more than 4,000 users on July 24 and 25.

The company says it later identified related patterns across a network of more than 15,000 users, which was completely disrupted on July 28. OpenAI attributes the core of the activity to people linked to Moonshot AI, the company behind Kimi, although it also acknowledges that it is unclear whether all the operators belonged to the same actor.

It Was Not a Data Breach

According to OpenAI, the attackers did not break its encryption, access a database, or enter stored conversations directly. They manipulated interactions with the models to make parts of their internal reasoning appear in forms visible to whoever requested them.

That protected reasoning is the internal record a model uses to solve a task. It does not always match the final answer and may contain information the system is designed not to show. If someone manages to collect it at scale, they can use it as training material to reproduce another model's capabilities.

This practice is known as adversarial distillation: systematically and without authorization extracting a model's responses or reasoning to train, copy, or improve another one.

OpenAI detected particularly elaborate methods. In one of them, the operators copied encrypted reasoning from one conversation and asked another model, in a separate conversation, to decrypt and transcribe it.

Independent security researchers also reported vulnerabilities related to attacks between models and conversation compaction. OpenAI confirmed that these attack paths were real and says the external research helped accelerate its protective measures.

What Risk Does It Pose?

The problem goes beyond one company copying another's work. Extracted reasoning could be used to train a model without preserving the safety controls applied to the responses users receive.

That could make it easier for a system to reproduce advanced capabilities without taking on the same research, infrastructure, and security costs. The risk increases in dual-use areas, where a capability can have legitimate applications as well as harmful uses.

The technique would not be exclusive to OpenAI either. The company shared information with the Frontier Model Forum and government information-sharing channels so that other developers of advanced models can look for similar attacks.

The Measures Taken

OpenAI says it combined technical controls, account restrictions, and coordination with partners to stop the campaign. The announced actions include:

• Banning or restricting fraudulent accounts.
• Strengthening registration and infrastructure controls.
• Expanding monitoring of networks of related accounts.
• Protecting hidden reasoning across users, workspaces, organizations, and model families.
• Closing a pathway that allowed encrypted reasoning from another user to be reused to recover its content.
• Detecting and retaining transmitted responses that could expose protected reasoning.

When some of the activity moved through third-party services, OpenAI worked with those providers to locate and deactivate the accounts involved.

The company warns that distillation attempts will likely become more sophisticated as models improve and interest grows in imitating them at a lower cost. It also acknowledges that open fronts remain, including implementations hosted by partners and attacks that use tool outputs, which require reviewing more than the visible text of a response.

For you, this means AI systems must protect more than your data and their servers. They also have to prevent their internal capabilities from being extracted at scale and reused without the same safety limits. The next battle will be detecting these coordinated campaigns before they turn thousands of seemingly normal interactions into a manual for copying a model.