# Opinion: AI benchmarks are broken, and we keep using them anyway

URL: https://ai-pulse.aglaganova.com/blog/opinion-ai-benchmarks-are-broken-and-we-keep-using-them-anyway
Published: 2026-10-09
Author: Roman Aglaganov, Full-stack developer
Category: Opinion
Region: Global
Tags: benchmarks

> Every new model arrives with a chart showing it beats everyone else on tests most people have never heard of. Here is why those numbers mean less than they seem, and what to do instead.

![Diagram of the benchmark life cycle: new test, models improve, saturated, harder test, repeat](https://ai-pulse.aglaganova.com/api/media/file/benchmarks-1.png)

Every AI launch now comes with the same slide: a table of benchmark scores, the new model’s numbers in bold, every rival a few points behind. It is the tech industry’s version of a car advert where the vehicle drives along an empty mountain road. Technically accurate, not much like your commute.

## Problem one: the tests get used up

Benchmarks have a short life. MMLU, a test of knowledge across 57 subjects released in 2020, was once a serious challenge; top models now score around 90%, leaving little room to tell them apart. Researchers keep building harder ones. Humanity’s Last Exam, published in early 2025, collected about 2,500 expert-level questions precisely because the old tests had run out of headroom, and the best models at launch scored under 10%. Those scores have been climbing ever since.

## Problem two: the answers leak

Models learn from enormous scrapes of the internet, and benchmark questions live on the internet. If test questions end up in the training data, a high score may measure memory rather than ability. It is the AI equivalent of finding the exam paper in the bin the night before.

## Problem three: Goodhart’s law

> “When a measure becomes a target, it ceases to be a good measure.”

The economist Charles Goodhart’s observation fits AI uncomfortably well. Once a benchmark becomes the number everyone competes on, it is tempting to tune models for that test specifically, and the score drifts away from the thing it was meant to capture.

## What to do instead

Benchmarks are still useful as rough signals, especially new and carefully guarded ones such as the ARC-AGI puzzles, which are easy for people and stubbornly hard for AI. But the best test of a model is the job you actually need it for. Try your own tasks, with your own documents, and keep the questions somewhere a web crawler cannot find them.

In short: read the chart, then run your own exam. And be suspicious of any score that ends in “state of the art” and starts with a marketing team.

## Sources

- [Humanity’s Last Exam (arXiv:2501.14249)](https://arxiv.org/abs/2501.14249)
- [Humanity’s Last Exam website](https://lastexam.ai)
- [ARC Prize](https://arcprize.org)