AI News

Anthropic Cuts AI Distillation Attacks by Up to 99%

Anthropic-Cuts-AI-Distillation-Attacks-by-Up-to-99

Key highlights:

  • Anthropic says its new safeguards have cut the reasoning data collected by distillers by 90% to 99%, making it far harder for outsiders to copy Claude’s abilities without permission.
  • The company introduced preserved thinking, stronger detection tools, and tougher account limits to stop people from extracting hidden reasoning.
  • Its two stage classifier now catches all the distillation methods described in a September report, the company says.
  • Anthropic has banned thousands of accounts linked to these activities, though it admits these protections can affect regular users too.

Anthropic has stepped up efforts to block unauthorized copying of its AI models, saying its latest safeguards have sharply reduced the amount of reasoning data extracted through distillation attacks. The company reports that distillers’ access to reasoning tokens has dropped by a factor of 10 to 100, or even more, since launching these new protections.

This announcement highlights a tough problem for AI developers: stopping other companies from using their models to build cheaper rival systems. Anthropic says these activities aren’t just about business competition; they raise concerns about cybersecurity, biological dangers, and national security. The company responded with technical limits, automated detection, and action against suspicious accounts, though it admits these steps come with price tags and sometimes disrupt normal customers.

Why AI Model Distillation Has Become a Concern

Distillation means using answers from a strong AI model to train another model to act the same way. By collecting lots of generated data, developers can clone advanced abilities without spending the time and money to build a top model from scratch. Anthropic believes that the most valuable material isn’t just the final answer; it’s the reasoning that comes before it. These transcripts show how the model thinks, which can help a smaller system learn faster.

At first, Anthropic tried to make this information less useful by summarizing Claude’s reasoning before giving it to users. In one test with the SWE-bench Verified coding benchmark, a pretrained Claude model trained on full transcripts scored 80.4%, but with summarized transcripts, that dropped to 71.8%. But attackers adapted. Some tried thousands of different prompts to get Claude to give its full reasoning. Others used large networks of fake accounts to keep extracting information.

Three Methods Strengthen Anthropic’s Defenses

Anthropic expanded its defense plan to limit reasoning access, spot extraction attempts, and make repeated abuse harder.

First, preserved thinking blocks new API accounts created from August 31, 2026, onward from editing Claude’s earlier context in supported models. This stops people from messing with old chats to grab protected reasoning. The company also limited reasoning access in certain models and accounts. With Claude Sonnet 5.5, reasoning transcripts go only to the organization that started the request.

Second, the two-stage classifier is designed to catch attempts to extract hidden reasoning. Anthropic says it tested this through internal and external red teaming, then patched up weak spots found along the way. The classifier’s false positive rate is under 0.1%, about as solid as safeguards meant to prevent help with bioweapon development. This system covers Claude Sonnet 5.5, Claude Opus 5 and 5.5, and Claude Fable 5 and 5.1. Anthropic rolls out new protections as it spots new extraction techniques, like requests to translate earlier reasoning or reproduce it in tags next to an answer.

Third, repeat offenders get targeted. Accounts suspected of distilling face tighter limits or bans. Since September 24, Anthropic has also started charging for requests blocked before the model answers in specific areas where false positives are below 0.1%, including distillation.

Security Risks and the Cost of Enforcement

Anthropic links unauthorized distillation to potential risks from powerful models released without equally strong safeguards. It points to Zhipu’s GLM-5.3, saying the company tried to distill cyber capabilities from top US models before switching to an older Claude model and another lab’s model. Anthropic says the US Center for Advancing Innovation and Standards for Super Intelligence (CAISSI) rated GLM-5.3 as the most cyber-capable open-weights model yet. Anthropic also says its own tests found the model’s safeguards could be bypassed.

National security is also a concern. Anthropic says it quietly forwarded requests to Moonshot’s Kimi models to Claude during distillation attempts. One request was about CCTV surveillance of a specific person, coming from someone Anthropic believes is linked to China’s People’s Liberation Army.

Anthropic reports huge drops in stolen reasoning data, but says the long-term success of its defenses isn’t clear yet. The company gave up significant revenue and trillions of tokens’ worth of paid traffic from distillers, while research and product teams spent considerable effort strengthening protections.

Ordinary users also face consequences. Some developers had to update integrations, innocent requests may get blocked from time to time, and customers might be charged for some rejected requests. Anthropic argues these costs are necessary and stresses that model builders and third-party platforms must work together to spot abuse and keep fake access in check.

Devanshi Kashyap
Devanshi is a curious learner who enjoys exploring new ideas and expressing creativity through art.
You may also like
More in:AI News