METR Anthropic Report: AI Is Now Speeding Up AI Research 1.9x — What It Really Means

On 5 November 2026, an independent AI evaluation group called METR published a report that asked the most consequential question in artificial intelligence right now: is AI already speeding up the creation of better AI? After eleven weeks inside Anthropic, with access to internal data no outsider had ever seen, the METR Anthropic assessment gave an answer that is both striking and carefully hedged. Anthropic’s models are improving about 1.9 times faster than they were before October 2025 — roughly two years of historical progress packed into a single year.

The headline travelled fast, and the METR Anthropic report deserves a slower read than the headlines gave it. The 1.9x figure is an estimate built on a single index, a specific statistical choice, and evidence that Anthropic itself heavily redacted before publication. METR’s own best guess is that about 80% of the acceleration comes from Anthropic’s use of AI in its own research — but the group is the first to say that figure is a judgement, not a measurement.

This article explains the METR Anthropic assessment in plain English: what was measured, how the 1.9x was calculated, why the same data can also produce 1.1x or 2.9x, what Anthropic blacked out, and what the finding means for anyone who builds with, buys, or regulates AI. The result is a genuine milestone in how we understand AI progress. It is also a number with wide error bars and a story the public still cannot fully check.

METR Anthropic assessment — researchers reviewing AI model data on computer screens in a laboratory

Table of Contents

The METR Anthropic Headline: AI Research Is Running 1.9x Faster

The central claim of the METR Anthropic report is simple enough to state in one sentence. Since October 2025, benchmark scores for Anthropic’s models have been growing at an estimated 1.9 times the rate they grew at before that date. The 95% confidence interval runs from 1.5x to 2.8x, which means the statisticians behind the work are fairly confident the true figure sits somewhere in that band — but the band itself is wide.

To make that concrete, METR translates the rate into time. Anthropic appears to have made as much progress on benchmarks in the past year as it made between October 2023 and October 2025 — roughly the distance from Claude 2.1 to Claude Sonnet 4.5, compressed into twelve months. That is the sense in which the METR Anthropic finding is described as “two years of progress in one”.

Three further judgements sit alongside the headline number, and each matters as much as the number itself. First, METR says it is confident the acceleration did not come from faster growth in training compute — the brute-force scaling of bigger training runs that powered most AI progress for a decade. Second, it guesses that a majority, perhaps around 80%, of the current acceleration comes from Anthropic’s own use of AI in research. Third, it found no clear bottleneck — in algorithms, compute, or data — that would stop the trend over the next twelve months, although it stresses low confidence in that judgement.

Put together, the METR Anthropic picture is of a leading lab whose research engine has shifted gear, with AI itself doing much of the pushing, and with no obvious wall ahead. It is also a picture painted with deliberately broad brushstrokes: METR says its overall estimate carries “high uncertainty” and that it would not be surprised if the true acceleration were closer to 1.3x or 3x. The honest summary of the report is not a single number. It is a range, a direction, and a set of caveats that the rest of this article works through one by one.

How the METR Anthropic Assessment Was Done

METR — short for Model Evaluation and Threat Research — is an independent AI safety and evaluation organisation. Anthropic invited it in, and from 12 August to 29 October 2026, roughly two and a half full-time METR researchers worked inside the company with what METR describes as unprecedented access to internal data on the AI research process.

The access covered pretraining and reinforcement-learning metrics, training-compute figures broken down by domain, benchmark scores for internal models, Anthropic’s own analyses of AI speedup, forecasts for a future model’s metrics, and interviews with about fifteen staff members. METR says it neither requested nor received payment for the work; Anthropic supplied laptops and token credits for the assessment.

Three questions guided the METR Anthropic review. Has the growth of Anthropic’s internal capabilities actually sped up? How much of any speedup is due to AI itself? And are there bottlenecks that could slow it down? The team examined internal-only models Anthropic calls Model 2, Model 3 and Model 4 — training finished in July 2026, August to September 2026, and September to October 2026 respectively — alongside public models such as Mythos Preview for comparison. One important limit: METR did not get query access to the unreleased models. It worked from scores and data Anthropic chose to share, not from hands-on testing.

The METR Anthropic assessment also had an unusual feature that shapes everything else in this article. Anthropic held contractual rights to redact any non-public information from the published report, and it used them. METR states plainly that Anthropic “redacted a majority of the evidence we found most informative for these conclusions”. The full write-up runs to about a hundred pages and has not been published. What the public got on 5 November is a summary post, with the most informative evidence described only in general terms.

METR Anthropic compute — engineers inspecting server racks in a data centre

That access arrangement is both the report’s strength and its weakness. No outside group has ever been given this depth of access to a frontier lab’s internal research data, which is why the METR Anthropic findings carry weight that a press-release claim never would. But the redactions mean outsiders cannot retrace the reasoning in full. As METR itself puts it, the public has to rely on the group’s judgement “much more than would be ideal, given the weight of the questions answered here”. Keep that tension in mind: it returns in every section below.

The Index Behind the METR Anthropic Number

The 1.9x figure does not come from a stopwatch or a single test. It comes from an index called AECI — Anthropic’s version of the Epoch Capabilities Index. Understanding what AECI is, and what it is not, is the key to understanding the whole METR Anthropic report.

Think of AECI the way you would think of a standardised test score. Each model gets one hidden “ability” number, and each benchmark gets a difficulty rating and a measure of how sharply scores change as ability passes that difficulty. Feed in many models and many benchmarks, fit the curves, and you get a single ability number per model over time. Plot those numbers on a timeline and you can see whether the line is getting steeper. METR used Epoch AI’s public code to build the index, then fitted straight-line trends before and after a break fixed at 7 October 2025.

Before the break, the index rose by about 15 points a year. After it, by about 29 points a year. Divide 29 by 15 and you get 1.9 — the headline acceleration. A second statistical check supported the idea that the slope itself changed, rather than the line simply jumping once when a strong new model arrived: the slope-change model’s error was 1.9 points against 3.9 for the jump model, and it won in 81% of bootstrap draws. In other words, the METR Anthropic analysis says progress did not just step up once; it is climbing faster, month after month.

The METR Anthropic Benchmark Set and the Saturation Problem

The index rests on fifteen benchmarks, grouped into maths, software engineering, science, multimodal tasks, and language or memory. Their names are redacted, so the public cannot see exactly which tests drive the number. What is public is that Anthropic’s newest internal model, Model 4, scores above 50% on every one of the fifteen, with scores approaching or exceeding 90% on several of them, according to reporting on the assessment.

That creates a measurement problem METR flags itself. As benchmarks fill up — as models approach the ceiling of what a test can distinguish — an index like AECI gets harder to read. A model that scores 95% cannot show much further improvement on that test even if it is genuinely getting smarter, so the index starts to understate real progress. Conversely, the last few percentage points near the top of a benchmark can be the hardest to win, so small score gains can hide large capability gains. Either way, a saturating benchmark set is a foggy instrument, and the METR Anthropic team says so openly.

There is also a scope limit built into AECI itself. The index tracks tasks with checkable answers — maths problems, coding challenges, science questions — because those are the tasks you can score reliably. It says less about messier, open-ended research work where the right answer is not obvious in advance. The METR Anthropic conclusion is therefore best read as a claim about well-specified tasks, not about all of AI research. METR says exactly that in its own summary: Anthropic is likely improving capabilities “on well-specified tasks” at about twice the earlier rate.

Why the METR Anthropic 1.9x Could Really Be 1.1x or 2.9x

Here is the part of the METR Anthropic report that most coverage skipped, and it is the most important part for anyone who wants to quote the number honestly. The 1.9x depends on a modelling choice called the link function — the curve that turns a model’s hidden ability into an expected benchmark score. Change that curve, and the same data gives a different answer.

The link function encodes an assumption about how the tasks inside a benchmark are spread by difficulty: how many are trivially easy or broken, and how many are nearly impossible. METR’s default is the logistic curve, which assumes a symmetric spread, and that gives 1.9x. But the other standard choices give very different results from the identical data:

  • Complementary log-log — 1.1x. This curve assumes many easy tasks, so the last points of improvement near the top of a benchmark are cheap, and recent gains count for less.
  • Probit — 1.8x. Close to the default, and the most commonly used alternative in this kind of analysis.
  • Logistic (default) — 1.9x. METR’s chosen curve, assuming a symmetric spread of task difficulty.
  • Learned Stukel — 2.6x. A flexible curve fitted to the data itself rather than assumed in advance.
  • Cauchit — 2.8x. Heavy tails on both sides of the difficulty spread.
  • Log-log — 2.9x. Assumes many hard tasks, so the same late gains imply much more underlying ability.

The intuition is straightforward. If a benchmark is assumed to hold many easy tasks, then squeezing out the last few points near the top is not very informative, so the speedup shrinks. If it is assumed to hold many hard tasks, then those same late gains mean the model must have become much more capable, so the speedup grows. The METR Anthropic report is unusually honest about this: most of these curves fit the data about equally well out of sample, at around 86 to 87% R-squared, so the data itself cannot pick a winner.

The METR Anthropic Range, Stated Fairly

The fair one-line summary of the METR Anthropic finding is therefore not “1.9x”. It is “somewhere between about 1.1x and 2.9x, with METR’s best guess near 2x”. METR itself frames it that way: it would be surprised by no acceleration at all, and surprised by acceleration above 3x on an improved version of the index. The confidence interval of 1.5x to 2.8x around the 1.9x point estimate is a different kind of uncertainty — the statistical noise around one specific modelling choice — and it should not be confused with the wider 1.1x to 2.9x range that comes from the modelling choice itself.

Two comparisons help put the number in context. Run the same method across all frontier developers, not just Anthropic, and the estimate drops to about 1.4x — suggesting Anthropic is accelerating faster than the field, or that its internal models are simply further along the curve. And Anthropic’s own Haiku 5.5 system card reported 1.7x, with the gap mostly explained by METR including data on newer, unreleased models. The METR Anthropic number is therefore not an outlier among related estimates; it sits at the upper end of a cluster that mostly points in the same direction.

None of this makes the finding uninteresting. A range of 1.1x to 2.9x still rules out “nothing has changed”, and the direction of travel is consistent across every variant of the analysis. But it does mean that anyone repeating “AI is improving twice as fast” as a settled fact is overstating what the evidence supports. The METR Anthropic report’s own authors would be the first to correct them.

The METR Anthropic 80% Question: How Much of the Speedup Is AI Itself?

The most quoted claim after the 1.9x is the attribution figure: about 80% of the current acceleration comes from Anthropic’s use of AI in its own research. This is the claim that turns a measurement into a story — AI building better AI, a loop that could compound. It is also, by METR’s own account, the least solid number in the report.

The 80% is a judgement, not a measurement. METR says most staff on the project shared an estimate of about 80% for October 2026, “with large uncertainty”. The evidence behind it is a bundle of different signals, none decisive on its own.

Training compute is confidently ruled out as the cause: METR says it received convincing evidence that the acceleration did not come from faster growth in training compute, although the details of that evidence were redacted. Interviews with researchers centred on about a 2x speedup from AI use. Self-reported uplift keeps rising. There are early signs of AI making autonomous contributions to research. And the non-AI factors METR investigated — hiring, data improvements, algorithmic discoveries, resolving other bottlenecks — do not, in its judgement, explain a lasting speedup.

One detail deserves care: METR thinks non-AI factors played a larger role in the late-2025 acceleration than they do now, and that AI’s contribution has grown relatively more important over time. The specifics of those late-2025 non-AI factors were redacted, so the public cannot judge that split for itself. The METR Anthropic report is asking readers to trust its synthesis of evidence they cannot see — and it says so, rather than pretending otherwise.

METR Anthropic AI-assisted research — developer working late with AI coding tools

The 12x Slack Polls: Felt Speed Versus Measured Speed

The most eye-catching evidence in the METR Anthropic report is also the loosest. Anthropic staff have voted in Slack, with emoji reactions, on how much the latest model speeds up their work. Those self-reports rose from about 1.2x for Claude 3.7 Sonnet in January 2025 to about 12x for Model 4 in October 2026. METR calls this “very loose evidence”: the people who answered may not be typical, and the answers took little effort.

The 12x and the 1.9x measure different things, and confusing them is the easiest mistake a reader can make. The 12x is how much faster an individual researcher feels on their own tasks. The 1.9x is how fast the lab’s models are improving on measured benchmarks. A person can feel many times faster while the lab’s overall progress speeds up far less, because research also waits on compute, experiments, decisions, and review. A twelvefold personal speedup does not mean the lab ships twelve times as much science.

The interviews sat between the two numbers. Coded by METR with Claude’s help, they came out as 2 yes, 3 leaning yes, 4 borderline, 0 leaning no and 2 no on the question of a large speedup — which METR reads as mildly favouring a 2x or larger speedup. That is supportive, but it is a long way from a proof, and the METR Anthropic team presents it as what it is: one thread among several, pointing in the same direction.

Assistance Today, Autonomy Tomorrow?

One of the most consequential findings in the METR Anthropic assessment is about the kind of AI contribution involved. As of October 2026, METR says, AI was accelerating Anthropic’s research primarily as an assistant — helping researchers write code, run experiments, and analyse results — rather than through fully autonomous contributions. Anthropic itself has said Claude authored more than 80% of the code merged into its codebase as of May 2026, though the company warns that lines of code do not measure quality, and METR repeats that warning.

But the report also notes early signs that fully autonomous contributions will play a larger role soon. That distinction matters enormously. An assistant that makes each researcher faster is a productivity story. A system that can run whole research programmes on its own is a different kind of event — the point at which the acceleration loop stops needing humans in the middle. The METR Anthropic assessment does not say that point has arrived. It says the ingredients are gathering, and that the next twelve months are worth watching closely.

What the METR Anthropic Report Could Not Show You

Any honest account of the METR Anthropic assessment has to spend as much time on what is missing as on what is present. The redactions are not a footnote; they are the report’s central limitation, and METR treats them that way.

Start with the redaction summary itself, which opens the report. Anthropic had the right to remove any non-public information, and it removed most of the evidence METR found most informative, “aimed at removing confidential information that could accelerate other AI developers”. Anthropic co-founder Sam McCandlish replied publicly that METR had sensitive internal data and independence, that many conclusions matched his own, and that the heavy redactions were necessary because sharing “exactly what is driving the speedup” could hand rival labs a roadmap. That is a reasonable commercial argument. It is also, unavoidably, an argument that the public cannot check.

What exactly is hidden? The names of the fifteen benchmarks are redacted — METR lists only the groups they belong to. A planned check of how compute is split between capabilities work and safety work was cancelled midway at Anthropic’s request. The detailed estimates for the cumulative and instantaneous effects of AI on Anthropic’s rate of progress, at various stages of the model-training pipeline, were redacted. The non-AI factors behind the late-2025 speedup are redacted. Forecasts for models after Model 4, and estimates of algorithmic gains, were removed. Much of the redacted evidence, METR says, had multiple plausible interpretations, and both METR and Anthropic researchers have significant uncertainty about how to synthesise it into a single picture.

METR Anthropic human-AI collaboration — human hand reaching toward a robotic hand

The independence picture has its own blemishes, which the METR Anthropic report discloses rather than burying. METR marked four items of the AI Evaluator Forum’s AEF-1 checklist as not met: query access to the models was not requested, method details cannot be shared, redaction rights were not limited to concerning findings, and METR has no responsible-disclosure policy. The team also leaned heavily on Claude itself for data collection and analysis, and notes that this could distort results — although it judged deliberate misleading to be extremely unlikely. One team member had discussed a role at Anthropic and held equity in a competitor; an unconflicted member checked their work.

None of these points means the report is wrong. They mean its conclusions rest on trust in METR’s judgement to a degree the group itself finds uncomfortable. When you read the 1.9x and the 80%, you are reading the verdict of a small team that saw evidence you cannot see, about a company that chose what to hide. The METR Anthropic report is best treated as a credible signal from a trusted evaluator, not as a reproducible scientific result. That distinction is the difference between a useful guide and a false certainty.

What the METR Anthropic Finding Means for the Rest of Us

So why should anyone outside a frontier lab care about the METR Anthropic report? Because if AI research is genuinely accelerating — even at the low end of METR’s range — the effects ripple outward to everyone who builds on, buys, or competes with AI.

Faster model releases are the most immediate consequence. If a lab can make two years of progress in one, the gap between model generations widens, and the upgrade cycle shortens. Businesses that plan model upgrades on a yearly cycle should expect each new release to be a bigger jump than the last one was, and should budget for re-testing prompts, tools and integrations more often. The METR Anthropic finding is, in practical terms, a warning that the ground under AI products will keep moving faster than product roadmaps assume.

Planning horizons shrink. A company choosing between building on today’s model and waiting for the next one faces a different calculation when progress is accelerating. The model you wait six months for may be much further ahead than the last six-month jump suggested. That cuts both ways: waiting gets more attractive, but so does building now with the expectation that the next model will handle the hard parts you are struggling with today.

Safety work faces the same acceleration. Anthropic’s own Responsible Scaling Policy lists AI research acceleration as a key risk indicator, and METR published a separate review of the AI research section of Anthropic’s August 2026 risk report alongside this assessment.

The concern is straightforward: if the systems that need oversight are improving faster than the oversight methods, the gap between capability and control widens. METR’s stated worries are a significant risk of further speedup over the next year, autonomous research contributions arriving soon, and benchmarks running out of room to measure progress at all. None of these is a prediction of disaster. All of them are reasons the METR Anthropic report argues for faster, better evaluation — including evaluations with the kind of access METR itself was given.

Competition intensifies. If one lab is accelerating at 1.9x while the field averages 1.4x, the lead compounds. Rivals that cannot match the pace face a widening gap, and the pressure to cut corners on safety testing grows. That is precisely why the redactions matter beyond academic curiosity: the details of how Anthropic is accelerating are exactly the details competitors would most want, and exactly the details the public most needs to judge whether the acceleration is safe.

METR Anthropic research team — team discussing findings around a whiteboard

For individual users, the practical takeaway is simpler. The AI tools you use today will get noticeably better, noticeably faster, than the upgrade pattern of the last two years suggested. Skills that depend on a model’s current limits — careful prompt engineering, working around a model’s blind spots — will age faster. Skills that depend on judgement, taste, and knowing what to ask for will age slower. The METR Anthropic report does not change what you should do tomorrow, but it does change how quickly you should expect the tools to change around you.

The Sceptical Case Against the METR Anthropic Number

A fair article must give the sceptics their hearing, and the METR Anthropic report itself supplies most of their arguments. Here they are, stated plainly.

It is one lab. Everything in the report is about Anthropic. The same method applied across all frontier developers gives about 1.4x, not 1.9x. It is possible that Anthropic is genuinely ahead of the field — its internal models, including the unreleased Model 2, 3 and 4, are part of the story. It is also possible that something specific to Anthropic’s history is flattering the number. The report cannot tell us which.

The baseline may have been unusually slow. The 1.9x compares progress since October 2025 with progress between March 2024 and October 2025. If Anthropic’s growth in that earlier window was unusually slow — because of a reorganisation, a strategy shift, or simply a quiet period — then the acceleration looks bigger than it is. METR lists this as one of the reasons the true figure could be lower, and the METR Anthropic team is explicit that it could be overestimating for exactly this reason.

Well-specified tasks are not all of research. AECI measures tasks with checkable answers. Real research also involves choosing the right problem, designing experiments that might fail, interpreting ambiguous results, and deciding what to try next. AI is strongest exactly where answers are checkable, so an index built on checkable tasks may overstate progress on the messier parts of research. METR acknowledges this, and it is the main reason the METR Anthropic conclusion is framed around well-specified tasks rather than research as a whole.

The modelling choice does heavy lifting. As shown above, the same data yields anything from 1.1x to 2.9x depending on the link function. The data cannot choose between the curves, so the headline number rests on a judgement call about which curve is most appropriate. A critic could fairly say the report’s central number is as much a modelling decision as a measurement.

The evidence is redacted, and the analysis leaned on Claude. The public cannot audit the most informative evidence, and METR used Claude — Anthropic’s own product — for parts of the data collection and analysis. METR judged deliberate misleading to be extremely unlikely, but the structural point stands: an assessment of Anthropic’s progress that relies partly on Anthropic’s own tools, with Anthropic controlling what can be shown, is not the same as an independent replication.

None of these objections kills the report. The direction of the finding survives every robustness check METR ran, the interview evidence points the same way, and the compute evidence rules out the most obvious alternative explanation. But together they set the right level of confidence: the METR Anthropic assessment is strong evidence that something real is happening, and weak evidence about exactly how big it is. Treat the direction as the news, and the number as an estimate with honest error bars.

METR Anthropic: Key Facts at a Glance

FactDetail
ReportIndependent Assessment of AI R&D Acceleration at Anthropic in October 2026
Published5 November 2026, by METR (Model Evaluation and Threat Research)
Assessment window12 August to 29 October 2026 — eleven weeks, about 2.5 full-time staff
Headline findingBenchmark scores growing 1.9x faster since October 2025 (95% CI: 1.5x to 2.8x)
Honest range1.1x to 2.9x depending on the statistical link function; METR’s best guess near 2x
Index usedAECI — Anthropic’s version of the Epoch Capabilities Index; 15 benchmarks
Trend changeAbout 15 index points a year before October 2025; about 29 points a year after
AI attributionAbout 80% of current acceleration guessed to come from AI use in R&D — a judgement, not a measurement
Staff self-reportsSlack polls rose from about 1.2x (Claude 3.7 Sonnet, Jan 2025) to about 12x (Model 4, Oct 2026)
Compute ruled outMETR says it is confident the speedup did not come from faster training-compute growth
BottlenecksNone clearly identified for the next 12 months — but with low confidence
Internal modelsModel 2 (trained July 2026), Model 3 (Aug–Sep 2026), Model 4 (Sep–Oct 2026); Model 4 above 50% on all 15 benchmarks
RedactionsAnthropic redacted a majority of the evidence METR found most informative; benchmark names hidden; full ~100-page report unpublished
Field comparisonSame method across all frontier developers gives about 1.4x; Anthropic’s Haiku 5.5 system card reported 1.7x
Cost to readersFree to read — the public report is published on METR’s website

Frequently Asked Questions

What is the METR Anthropic report?

The METR Anthropic report is an independent assessment published on 5 November 2026 by METR, an AI evaluation organisation, after eleven weeks working inside Anthropic with access to internal research data. It estimates that Anthropic’s models have been improving about 1.9 times faster since October 2025 than before, and that AI itself is behind most of the acceleration. Anthropic redacted much of the underlying evidence before publication.

Does 1.9x mean AI is now twice as good every year?

No. The METR Anthropic 1.9x is a ratio of growth rates, not of capability levels. It means the pace at which benchmark scores are rising has roughly doubled — about 29 index points a year instead of about 15. It does not mean models are twice as capable as they were a year ago, and the estimate depends on modelling choices that could put the true figure anywhere from about 1.1x to 2.9x.

What does the METR Anthropic 80% figure actually mean?

It is METR’s subjective estimate that about 80% of Anthropic’s current research acceleration comes from using AI in research itself. It is a judgement based on interviews, self-reports and ruling out other causes — not a direct measurement — and METR says it carries substantial uncertainty. The METR Anthropic team believes AI’s share has grown over time and that non-AI factors mattered more in late 2025.

Why was so much of the METR Anthropic report redacted?

Anthropic held contractual rights to redact any non-public information, and said the cuts were aimed at removing confidential details that could accelerate rival AI developers. METR states that a majority of the evidence it found most informative was removed, including benchmark names, detailed AI-contribution estimates, and forecasts for future models. The public must therefore rely heavily on METR’s judgement rather than checking the evidence directly.

Is AI now doing research on its own?

Not yet, according to the METR Anthropic assessment. As of October 2026, AI was accelerating Anthropic’s research mainly as an assistant — helping researchers code, experiment and analyse — rather than making fully autonomous contributions to the most important model-training work. METR reports early signs that autonomous contributions will play a larger role soon, but the fully self-directed research loop has not arrived.

Does the METR Anthropic finding apply to other AI labs?

Only partly. The report covers Anthropic alone. Running the same method across all frontier developers gives an acceleration of about 1.4x rather than 1.9x. Other labs may be accelerating too, but there is no equivalent high-access assessment of OpenAI, Google or Meta, so the METR Anthropic number should not be quoted as an industry-wide fact.

Where can I read the METR Anthropic report?

The public report is free to read on METR’s website. It is a summary post; the full report of about a hundred pages has not been published, and a separate METR review of the AI research section of Anthropic’s August 2026 risk report was published alongside it.

Read the full METR report here.

The METR Anthropic assessment is the first serious measurement of a question that will define the next decade of AI: how fast can AI improve itself? The answer, as far as one lab’s internal data can tell us, is “noticeably faster than it used to, and probably because of AI itself”. That is a big claim, and the evidence for it is real — an eleven-week, high-access review, a consistent direction across every statistical variant, and a confident ruling-out of the most obvious alternative explanation.

It is also a claim with visible seams. The headline number moves from 1.1x to 2.9x when a single statistical assumption changes. The attribution figure is a judgement. The most informative evidence is blacked out. The benchmark set is saturating, the scope is one lab, and the analysis itself leaned on the company’s own product. None of that makes the report dismissible. All of it makes the report quotable only with its caveats attached.

The right response to the METR Anthropic finding is the one METR itself models: take the direction seriously, treat the number as an estimate, and ask for more. More access for independent evaluators, more published evidence, better benchmarks that do not saturate, and assessments of other labs so one company’s story can be compared with the field’s. Until then, the honest summary stands: AI research is very probably accelerating, AI is very probably a large part of why, and the exact speed is something we are still learning to measure.

For more on how AI systems are doing real scientific work — and how to tell a genuine result from a demo — see our explainers on the AI system that proved the Aaronson-Ambainis conjecture in three days and Anthropic’s FlashProve verification system, and our look at how long AI research findings stay true.

Leave a Comment

Your email address will not be published. Required fields are marked *