- Vals AI found multi-agent “teams” of AI models cost 1.8x to 5.1x more than solo agents while barely improving output quality.
- Only one of four GPT-6 Sol and Claude Opus 5.5 team-versus-solo comparisons showed a statistically significant gain.
- Anthropic’s own internal tests show quality gains flatten well before reaching 100-agent swarms.
- OpenAI researcher Noam Brown says multi-agent systems mainly buy speed, not better answers.
What Happened
Evaluation company Vals AI tested GPT-6 Sol and Claude Opus 5.5 solo and in multi-agent teams on its Vibe Code Bench coding benchmark, at both medium and maximum reasoning effort, The Decoder reported on October 11, 2026. Teams cost between 1.8 and 5.1 times more than single agents to run. Of four team-versus-solo comparisons, only one — GPT-6 Sol at medium reasoning — showed a statistically significant improvement, scoring 7.3 points higher than its solo counterpart.
Why It Matters
The result cuts against the industry’s enthusiasm for agent “swarms” as a path to better output. Anthropic’s own internal testing of Opus 5.5 backs up the finding: quality gains shrink as more agents are added, and scaling from 10 to 100 agents barely moved scores after 24 hours of work. On a knowledge-base task, Opus 5.5 scored 0.53 solo, 0.70 with 10 agents, and just 0.74 with 100 — most of the gain came from the first 10, a pattern that echoes Nvidia’s own push to cut coding-agent token usage rather than add more agents.
Technical Details
On a Lean theorem-proving task, Opus 5.5 improved from 0.39 solo to 0.66 at 10 agents and 0.68 at 100. Anthropic’s Fable 5.1 showed stronger gains on that same task above 10 agents but trailed Opus 5.5 across every test, and its knowledge-base score actually dipped slightly scaling from 30 to 100 agents. OpenAI researcher Noam Brown told the Dwarkesh Podcast that multi-agent systems mainly buy speed rather than quality — four agents solved tasks twice as fast, but also cost twice as much.
Who’s Affected
Enterprises paying per-token for agent orchestration face a direct cost question, as do vendors marketing multi-agent “swarm” products as a quality upgrade rather than a speed upgrade — the same framing problem behind OpenAI’s attempt to rein in its own swarming agents with a Decisions API. Teams run at maximum reasoning effort saw no measurable benefit at all from adding agents, per Vals AI’s tests.
What’s Next
Watch whether agent-platform vendors reposition swarms explicitly as a latency tool rather than a quality tool, and whether benchmarks beyond Vibe Code Bench and Lean theorem-proving confirm the plateau holds across task types.
