METR, a nonprofit in Berkeley, California, that evaluates artificial intelligence (AI) models, published an audit of research at Anthropic on November 5, 2026. It found that Anthropic, the company behind the Claude assistant, was likely improving its models about twice as fast as before. The estimate compares the pace as of October 2026 with the pace from March 2024 to October 2025, on well-specified tasks. METR says most of that acceleration probably comes from Anthropic’s heavy use of its own AI in its research.
That matters because Anthropic’s own safety policy treats a doubling of progress driven by automated research as an early-warning threshold. In a second report released the same day, METR says the risk report Anthropic published in August understated the acceleration already underway. METR also flags a limitation of its own work: Anthropic redacted most of the evidence the evaluators found most informative.
How METR ran the audit
METR’s team, the equivalent of 2.5 full-time staff, worked on Anthropic’s data from August 12 to October 29, 2026. It received internal models’ scores on benchmarks, the standardized tests used to compare AI models. It also got training metrics (training is the phase where a model learns), staff surveys and about 15 interviews. METR says it did not query the unreleased models itself, and did not ask to.
METR says it was not paid. Anthropic provided laptops for security reasons, plus free credits to use its models, which METR mostly spent on data analysis. The audit predates a separate arrangement that Anthropic announced on October 15: “employee-like” access for a METR team.
Two years of progress in one
METR measures the pace of progress with AECI, Anthropic’s version of the Epoch Capabilities Index, a scoring system. The index boils a model’s results on many benchmarks down to a single score. Since October 2025, that score has been rising 1.9 times faster than its earlier trend. METR’s 95% confidence interval, its likely range, is 1.5 to 2.8.
The 1.9 figure means Anthropic made as much benchmark progress in the past year as it did between October 2023 and October 2025. It roughly equals the gap between the older Claude 2.1 and the more recent Claude Sonnet 4.5.
METR’s 1.9 is higher than the 1.7 Anthropic published in the system card for Claude Haiku 5.5, its small model. A system card is the technical document a company releases with a model. The difference comes mainly from the newer internal models METR included. Other ways of modeling the trend give anywhere from 1.1 to 2.9.
Adding rival labs’ models to the comparison lowers the estimate to 1.4. METR offers two interpretations. If Anthropic was behind its rivals in 2025, 1.4 is closer to the real figure. If its models were less tuned to score well on benchmarks, 1.9 holds.
METR says it is confident on one point: the acceleration did not come from faster growth in the computing power used for training.
Running out of hard benchmarks
The report names three recent internal models that Anthropic never released, using code names given to METR. Model 2 finished training in July 2026, Model 3 in August and September, and Model 4 in September and October. Model 4 scores above 50% on all 15 benchmarks it was tested on, for an AECI score of 178.6. Nine months earlier, Claude Opus 4.6, one of Anthropic’s top-tier models, was still below 50% on many of them.
With so few hard benchmarks left, METR says it is “somewhat nervous” about its ability to measure AECI now and in the future. The full report included a forecast for a future model, but Anthropic redacted it.
Staff who say they are 12 times more productive
Anthropic shared polls from Slack, the messaging app it uses internally. Employees answer with an emoji reaction to estimate how much the latest model speeds them up. In October 2026, the average self-reported speedup for the best internal model reached about 12 times.
METR calls this “only very loose evidence.” The answers take little thought, the respondents are probably not representative, and self-reports can overstate real productivity gains.
The interviews point the same way, though less strongly. METR used Claude to classify the statements of 11 researchers. The question was whether AI had at least doubled the pace of progress in their area of work. Claude rated two researchers yes, three leaning yes, four unclear and two no.
One researcher said, “It is hard to see how I am not being 3x more productive.” Another put AI’s share of the acceleration at 15 to 20% and dismissed self-reported productivity gains.
METR attributes most of the current acceleration to AI, perhaps 80%. It calls that estimate highly uncertain. For now, AI cannot by itself make major contributions to the most important parts of training. Even so, researchers describe significant “signs of life” in that direction.
METR saw no bottleneck in computing power, data or algorithms that would slow this pace over the next 12 months. It adds that it is not confident on that point.
METR says Anthropic’s August report understated the speedup
Under Anthropic’s Responsible Scaling Policy, or RSP, the company has to strengthen its safeguards when its models reach certain capability thresholds. The doubling tied to automated research is an early-warning threshold in that policy.
In its public August 2026 risk report, Anthropic wrote that its leading indicators showed acceleration “by less than a factor of 2.” METR had access to the private version of that report. Combining the figures in it, METR arrives at 1.97. It says “around 2” would have been a better description.
METR finds it “plausible” that the threshold had already been crossed by July 15, the date the August report covers. Most of the relevant researchers at METR and Anthropic lean the other way. METR agrees with Anthropic on the immediate danger: the risk of catastrophe from the July models was low.
A heavily redacted audit
Anthropic had the right to redact any non-public information. It said its redactions were “aimed at removing confidential information that could accelerate other AI developers.” METR says the redactions force the public to rely on its judgment much more than would be ideal.
The full report, about 100 pages, was delivered on October 22 and remains private. From October 27 to 29, Anthropic tightened its guidance on what to redact. That pushed the public version back by about five days.
Anthropic also dropped part of the contract midway. That part would have checked how much of its computing power goes to making models more capable, and how much to safety. The company says it will revisit it later.
Sam McCandlish, an Anthropic co-founder who leads much of its safety work, responded the same day on X, formerly Twitter. “AI is speeding up our research, but it’s hard to say exactly how much,” he wrote.
He acknowledged that the redactions make METR’s work harder to check. As reasons for them, he cited commercial sensitivity and the risk of “handing out a roadmap for others to go faster.” Anthropic plans to share its own analysis “in the coming weeks,” including where it stands against its RSP.
The Associated Press (AP) notes that the audit took place while Anthropic CEO Dario Amodei was calling on the industry to slow down. In a September 12 essay, he urged the AI world to “slow the pace at which we improve the capabilities of AI models.” According to the AP, Anthropic invited METR in as it prepares for its stock market debut.
The limits METR acknowledges
METR lists what could be inflating its numbers. The acceleration might be limited to well-specified tasks that are easy to grade automatically, without carrying over to messier work. Before late 2025, Anthropic was improving more slowly than the rest of the industry, which can make the acceleration look larger.
Factors unrelated to AI, such as hiring, likely played a role in late 2025. They could explain most of the recent acceleration.
The numbers could also be too low. Progress may still be speeding up, and full automation of some research tasks could come soon. Benchmark scores also sometimes understate a model’s real-world impact.
The report adds that Claude itself helped with most of the data analysis, at both Anthropic and METR. The evaluators say they found no sign of systematic bias.
What it means
An outside evaluator has measured how fast a major AI company is improving its models, using that company’s internal data. METR calls that level of access unprecedented. Its central figure, about twice the earlier pace, reaches Anthropic’s early-warning threshold if automation accounts for a large share of it.
The public sees only a curve of benchmark scores, internal polls and interview quotes. Everything else is asserted without public evidence: AI’s exact share, the details of training and the forecasts for upcoming models were all redacted.
For businesses that use Claude, the practical effect is still unknown. The report covers internal models and gives no public release dates. The “12 times” figure reported by Anthropic staff says nothing about the gains any other organization would see.
Anthropic’s promised analysis is due to say where the company stands against its RSP and to set out new safety measures. Another open question is whether other AI companies will open their data the same way. According to the AP, Sam Altman, CEO of ChatGPT maker OpenAI, said in September that he also supported independent evaluators with employee-like access.
Sources
- Independent Assessment of AI R&D Acceleration at Anthropic in October 2026, METR, November 5, 2026
- Review of the “Risks from automated R&D” section in Anthropic’s Risk Report (August 2026), METR, November 5, 2026
- An update on independent evaluators, Anthropic, October 15, 2026
- Responsible Scaling Policy, Anthropic, accessed November 6, 2026
- Sam McCandlish’s post on METR’s report, X, November 5, 2026
- Anthropic is rapidly accelerating AI development amid slowdown calls, outside research group says, Associated Press (via KSAT), November 5, 2026
Header image: MACCUS chart, data from METR’s assessment of AI R&D acceleration at Anthropic, November 5, 2026.
