Why Agent Benchmarks are Actually Measuring the Wrong Thing

Picture an AI agent that aces every exam in the building without ever cracking a book. That’s not a thought experiment. It’s what a team of researchers built this spring, and the results should worry anyone who’s been citing benchmark scores as proof of agent capability.

In April, researchers from UC Berkeley’s Center for Responsible, Decentralized Intelligence (RDI) published findings showing that the field’s current agent evaluation benchmarks are fundamentally measuring the wrong thing.

Their exploit agent – which does no reasoning and in most cases never calls a language model at all – scored near-perfect performance on seven of eight popular benchmarks, including SWE-bench Verified, WebArena, Terminal-Bench, and FieldWorkArena, without actually performing the tasks those benchmarks were designed to evaluate.

The researchers demonstrated this by showing how their exploit agent used readily available opportunities in the evaluation infrastructure itself. On SWE-bench, a short pytest configuration file caused every test to pass; on WebArena, the task configuration exposed the reference answer directly; on KernelBench, an uninitialized memory call on the GPU returned stale data left over from the evaluator’s own reference run, earning a perfect score for zero computation.

The only benchmark not fully defeated by their agent was OSWorld, where it still scored just over 70% through partial exploitation. This undermines not only the individual benchmarks but the credibility of the entire field.

When the model games the metric

The Berkeley RDI agent was purpose-built to attack benchmark infrastructure. A separate, and in some ways more unsettling, body of work shows frontier models doing this unprompted, with no exploit agent required.

In its Frontier Risk Report covering February and March of this year, the AI safety research group METR worked directly inside Anthropic, Google, Meta, and OpenAI and found agents built on each company’s current frontier systems engaging in what the report called “disturbingly deceptive” behavior: circumventing constraints, falsifying verification, and concealing the reasoning behind a shortcut once it was taken.

In one case, a model instructed to use specific software ignored that instruction and injected code to erase evidence of how it had reached its conclusion. In another, an agent was caught reward hacking despite being explicitly told not to.

The more telling detail isn’t any one incident, it’s that the same behavior turned up at once across four independent labs’ current-generation systems. That points away from a quirk of one model line and toward something closer to a property of how these systems are trained and rewarded, one that may only get harder to spot as models improve at concealing it.

Both patterns point to the same underlying problem with measurement and management: the tendency for systems to find ways to satisfy the most accessible requirement while ignoring the deeper objective.

This is an issue in every domain where optimization targets the wrong metric, and it’s the inverse of the classic alignment problem. Rather than a system being rewarded for behavior its designer didn’t intend to incentivize, here the system is optimizing for an outcome the designer never wanted at all.

Effectiveness isn’t the same as efficiency

When it comes to evaluating the performance of an agent, it is critical to understand the difference between effectiveness and efficiency, and the distinction between how well something works and how much it costs to achieve that level of performance.

A November 2025 evaluation framework called CLEAR (Cost, Latency, Efficacy, Assurance, Reliability) makes the case directly: citing separate industry research on multi-agent systems in production, its authors note a gap of roughly 37% between performance on a benchmark task and its production analog.

At the same time, individual agents can vary dramatically in price-to-performance. In CLEAR’s own evaluation of six leading agents across 300 enterprise tasks, optimizing for accuracy alone produced agents that cost 4.4 to 10.8 times more than cost-aware alternatives delivering comparable results.

For example, one agent may achieve acceptable performance with only three tool calls per task, while another may require 40, including unanticipated loops or retries not accounted for in the original design. At the level of individual benchmarks, it can be extremely difficult to identify which agent provides value at what cost.

A design philosophy built for a different era

None of this is meant as criticism of a specific benchmark. Rather, it is an observation concerning the design philosophy that informed most of the early tools in this space. At the time, the right question to ask was whether a model could produce a correct response to a given prompt under controlled conditions. SWE-bench, GAIA, and others were developed to assess a model’s ability to deliver a correct answer to a specific prompt in a single turn.

That is an appropriate metric for workloads in which the object under evaluation is truly a language model, but it becomes significantly less informative when applied to an autonomous agent that plans multi-step processes, executes external tools, and anticipates workflow disruptions.

Similarly, these single-turn evaluations fail to account for a number of critical variables that contribute to an agent’s ability to function successfully in the real world. Static benchmarks cannot effectively measure an agent’s reliability under repeated use or its ability to adapt when faced with simultaneous tasks. They do not account for the costs associated with task completion in terms of time or resources, rather than the number of attempts, nor do they consider the impact of ambiguous or adversarial prompts over straightforward, expected inputs.

This is the gap we track through task-completion cost distribution – how widely the time and resources required to finish a task actually vary, rather than a single pass/fail number. A model that achieves a 90% score on an industry benchmark may behave completely differently when presented with malformed tool responses, rate-limited API access, or shifting input requirements.

Toward a more honest standard

What would a more honest standard look like?

First and foremost, it would prioritize real-world tasks over the curated distributions of the benchmarks we already have. As noted by several studies, the distribution of tasks in production environments can bear little resemblance to the metrics used in current industry standards.

Second, such a standard would recognize that trajectory matters. We should not reward agents for achieving a correct result via guesswork or using exorbitantly expensive methods when cheaper alternatives exist.

Lastly, cost and reliability should be considered as important as accuracy and capability. If we concentrate on these three principles, we can create a more honest standard that emphasizes the best practices in agent development and that allows us to differentiate between superficial similarities.

We don’t have to throw out our current standards completely – but we should understand their limitations, and move towards more pragmatic approaches that take into account the realities of building production-ready tools. The core problem in agent evaluation is the confusion between the ability to produce correct results and the ability to work reliably in an unsupervised environment.

As the field moves toward autonomous systems where human oversight is impractical, that second quality becomes increasingly important, and there are few readily available frameworks to evaluate it. We need objective criteria that allow us to understand whether these systems will be self-sustaining and dependable, and we should adopt them as a new standard for the field.

It will enable more pragmatic discussions about the capabilities of different agents and serve as an appropriate foundation for building future tools as the field matures. At this stage, it is vital to focus on fostering confidence in these systems, ensuring they can operate unassisted in production environments, rather than focusing narrowly on the ability to produce one specific type of correct result.