UK’s AI Security Institute finds standard benchmarks systematically underestimate what AI agents can actually do

In a study covering seven benchmarks, the UK’s AI Security Institute shows that standard AI evaluations systematically underestimate agent capabilities by capping the compute budget. On software engineering tasks, success rates jumped about 25 percent when the token budget was increased tenfold.

An AI agent’s performance is a curve that rises with test-time compute, the amount of processing power an agent is allowed to burn while working on a task. Cut the budget while the curve is still climbing, and the measured score tells you the minimum, not the maximum. That’s what the AISI researchers set out to prove in their latest work. The big question: how much do capabilities scale with compute, and what does that mean for cybersecurity?Ad The effect shows up across domains. In cybersecurity, about 8 percent of tasks were only solved when the budget exceeded 10 million tokens; some even required 50 million. The newest models hit even higher scores at budgets above 100 million tokens.AdDEC_D_Incontent-1 On software engineering tasks (TerminalBench 2.0, SWE-Bench Pro), success rates jumped about 25 percent when the token budget went from one million to ten million. For math and academic tasks (Humanity’s Last Exam), the gain was around 22 percent up to a budget of five million tokens. Extra compute doesn’t help everywhere equally. On HealthBench, a medical task benchmark, all models hit their plateau within the standard budget. According to AISI, more compute helps most where agents can verify their own work, like running code or testing an exploit. But it barely moves the needle where feedback is missing or delayed.Ad Human task time predicts how many tokens agents need Another finding ties the time a human expert needs for a task to the agent’s token consumption. Across 211 software engineering tasks from the research institute METR and 78 cyber tasks from AISI, this relationship follows a power law. A one-minute task costs the agent thousands of tokens. A one-hour task costs millions. A one-week task costs billions.