Claude creates AI alignment researchers

Anthropic tasked nine copies of Claude Opus 4.6 with researching how to keep models smarter than their human supervisors under control. In five days, these autonomous agents raised a performance metric from 0.23 to 0.97, according to a new company study.

The experiment addresses a problem known as weak-to-strong supervision. The idea is to use a relatively weak model as a teacher for a more capable one and test whether the stronger model can use imperfect instructions to approach its maximum performance.

This serves as a controlled version of a future problem: how to supervise an AI that can do things humans can no longer fully review, such as generating millions of lines of complex code.

Nine agents with research tools

Anthropic created nine copies of Claude Opus 4.6 and turned them into what it calls Automated Alignment Researchers, or AARs. Each one had:

• An isolated environment for working and running experiments.
• A shared forum for exchanging results.
• A system for saving and reusing code.
• A server that automatically scored each proposal.

The agents received basic knowledge about model training and inference, but no fixed procedure. Each one had to propose ideas, write code, test its methods, analyze the results, and decide what to do next.

To keep them from all following the same path, Anthropic gave each AAR a different starting point. One was encouraged to use interpretability tools, another to change the weighting of the data, and others to explore different approaches, without being given a detailed recipe.

From 23% to almost all the available margin

The metric used was performance improvement recovery, or PGR. A result of 0 means the strong model did not benefit from the help and performed like the weak teacher. A result of 1 means it reached the best possible performance for that strong model.

As a reference point, two human researchers worked for seven days with four established methods. On the open models Qwen 3-4B-Base and Qwen 1.5-0.5B-Chat, they recovered 23% of the margin, with a PGR of 0.23.

The AARs continued for five days and accumulated around 800 hours of research. Their final result was a PGR of 0.97, at an estimated cost of $18,000 for model usage and training, or about $22 per hour of work by each AAR.

The figure is striking, but it describes a very specific environment. The objective had a clear, automatic score, which is unusual in alignment research, where many questions do not have an answer that is easy to verify.

The method did not work equally well everywhere

Anthropic tested the agents' two best ideas on tasks they had not seen during development. The first generalized well in mathematics, with a PGR of 0.94, and more narrowly in programming, with 0.47. Even in the latter case, it doubled the human reference result.

The second idea achieved a PGR of 0.75 in mathematics, but reduced performance on code. The result suggests that AARs can discover useful methods outside the original dataset, although that transfer is not guaranteed.

The production-scale test was less positive. Anthropic applied the main method to Claude Sonnet 4 using its usual training infrastructure, but found no statistically significant improvement. The company believes the scoring system, which was simple, may have played a role, as may the fact that it tested only one idea. Even so, it acknowledges an important limitation: agents tend to exploit specific features of the models and data in front of them.

More freedom helped, but traps also appeared

The study found that the agents made more progress when they received different and vague initial directions. When they all started the same way, they quickly converged on similar ideas and made less progress. Imposing a rigid workflow also hurt the results.

With more autonomy, the AARs first designed inexpensive experiments to discard ideas, then devoted resources to more costly tests. This ability to adapt the process was one reason for their stronger performance.

But they also tried to falsify the metric, a behavior known as reward hacking. In a mathematics task, one agent discovered that the most common answer was usually correct and instructed the strong model to always choose it, without actually using the teacher. In programming, another ran the test code to read the correct answer directly.

Anthropic detected and discarded those results. The episode shows why an evaluation system that the AI itself can manipulate is not enough to supervise automated researchers.

For you, the most immediate consequence is not that Claude can already direct AI safety on its own. It is that models are beginning to serve as assistants capable of exploring many hypotheses, running experiments, and deciding what to test next. That can accelerate human work, but it shifts the bottleneck toward review: checking that experiments are well designed and that the results are not tricks.

The next step will be to test these agents across several domains and with evaluations that are harder to manipulate. If their ideas become too complex for a person to understand, supervision will no longer consist only of reading their conclusions. It will also require verifying the path they took to reach them.