Agent teams cost more without consistent coding gains
Illustration, not documentary evidence of the event.
Adding agents did not consistently improve coding results in Vals AI’s tests of GPT-6 Sol and Claude Opus 5.5. Teams cost between 1.8 and 5.1 times as much as single agents, according to the-decoder.com, which reported the evaluation company’s findings.
Vals AI compared solo and team configurations on Vibe Code Bench at medium and maximum reasoning effort. Only one of four comparisons produced a statistically significant improvement: the GPT-6 Sol team at medium reasoning scored 7.3 points higher. At maximum reasoning, neither model’s team configuration showed a meaningful advantage.
For developers choosing an agent setup, this suggests separating two purchasing decisions: paying for better output and paying for faster completion. Start with a single-agent baseline on tasks representative of your application. Compare it with a team using the same model and reasoning effort, and record output quality, elapsed time, and total cost. Define the improvement that would justify the added expense before reviewing the results. The reported benchmark supports that comparison, but does not establish how teams will perform on your workload.
Speed may still justify a team even when quality barely changes. The report describes Anthropic tests in which larger Opus 5.5 teams reached a given performance level faster, while expanding from ten to 100 agents produced only small score gains after 24 hours. Separate ProgramBench tests paired speed gains with higher token usage. That makes turnaround time a distinct benefit to measure, rather than evidence of a better final result.
Task structure also matters. In the report’s account of his Dwarkesh Podcast appearance, OpenAI researcher Noam Brown said web research and math lend themselves to parallel work, while novel writing does not. A practical criterion is whether a job contains independent pieces that can be checked and combined. If you can identify those pieces, test delegation there first. If every agent needs the same evolving context, require a measured benefit before expanding the team.