RESEARCH

METR: AI Models Are Improving Faster Than Safety Testing Can Keep Up

J James Whitfield Nov 22, 2026 2 min read
Engine Score 8/10 — Important

METR warning that AI safety testing can't keep pace with capability growth, high-impact safety research

Editorial illustration for: METR: AI Models Are Improving Faster Than Safety Testing Can Keep Up
  • METR’s Q4 2026 Capability Update, published November 20, warns that leading models like Opus 5.5 and GPT-6 Astra are improving faster than researchers can reliably measure.
  • METR says model capability on long, open-ended tasks has been doubling roughly every 3.6 months.
  • Extrapolating that trend, METR estimates models could reach a 50% success rate on 200-hour tasks by July 2027.
  • The report’s core warning is that humans may lose the ability to assess catastrophic risks before new systems are deployed.

What Happened

METR, the AI safety evaluation nonprofit, published its Q4 2026 Capability Update on November 20, warning that “AI models are rapidly improving at AI research tasks, and they’re getting harder to measure,” according to The Decoder (Nov 22, 2026). The report names Opus 5.5 and GPT-6 Astra as examples.

Why It Matters

This is a separate, newer finding from METR’s November 8 report that clocked Anthropic’s internal R&D pace at roughly 1.9x. Where that report measured one lab’s acceleration, this one addresses a broader structural problem: evaluation methods are falling behind the systems they’re meant to assess.

Technical Details

METR found that on coding and computational tasks with direct feedback, current models already outperform individual experienced professionals, though not large groups of top experts combined. On longer, open-ended tasks, measured capability has been doubling about every 3.6 months; extrapolating that curve, METR projects a 50% success rate on 200-hour tasks by July 2027. The organization notes models still struggle with open-ended questions that lack immediate feedback.

Who’s Affected

AI labs, including Anthropic and OpenAI, face pressure to build evaluation methods that scale alongside capability rather than lag behind it. The finding also bears on regulators evaluating AI agent monitoring gaps, since weaker measurement tools make it harder to catch risky behavior before deployment.

What’s Next

METR’s own stated limitation — that current tests may both overestimate and underestimate real-world risk depending on task type — leaves the next capability update as the key marker for whether the 3.6-month doubling trend holds. The organization is urging labs to treat measurement capacity as its own scaling problem.

Share

Enjoyed this story?

Get articles like this delivered daily. The Engine Room — free AI intelligence newsletter.

Free · No spam · Unsubscribe anytime