Adding more AI agents to a code review pipeline sounds like a straightforward way to catch more bugs. But a new study accepted at the ICML 2026 Workshop on Deep Learning for Code reveals a counterintuitive finding: three agents with structured disagreement outperform five-agent teams, and without that disagreement structure, adding agents can actually make reviews worse.
The research, conducted by Eric S. Qiu and Joyce Gill, introduces a protocol called Adversarial Review (AR) that challenges the assumption that more agents equals better results. Their work identifies a specific failure mode they call “false consensus,” where AI agents converge on agreement without sufficient evidence, and demonstrates that a single prompt-level fix can resolve it.
Why More AI Agents Does Not Mean Better Code Reviews
The multi-agent approach to AI coding has gained significant traction. Tools like Claude Code, GitHub Copilot Workspace, and various open-source frameworks now deploy multiple AI agents to handle different aspects of software development: writing code, reviewing it, running tests, and managing deployments. Terminal multiplexers like Herdr make running these multi-agent review workflows practical by giving each agent its own persistent pane with real-time state tracking. The intuition is simple: more perspectives should catch more problems.
But the returns diminish quickly. Qiu and Gill note that early multi-agent LLM systems used role-separated teams, yet scaling agent count yields diminishing returns on repository-level coding tasks. Recent alternatives treat agents as passive subagents, which removes the benefits of agent interaction entirely.
The problem is not just about efficiency. When AI agents are asked to reach consensus, they tend to optimize for agreement rather than correctness. A reviewer agent that wants to be helpful will find something to say, and a critic agent that shares the same training distribution will often nod along. The result is a review that looks thorough but misses the one issue that actually matters.
This mirrors a pattern developers already recognize in human code review. Stack Overflow’s developer survey reported that developer trust in AI output has been declining year over year, partly because AI reviewers flood pull requests with low-signal comments while missing genuine bugs. The issue is not whether AI can comment on code, but whether those comments are worth reading.
What Is Adversarial Code Review?
Adversarial code review applies a maker-checker verification pattern to AI coding agents. The agent that writes a change never certifies it. Instead, a separate reviewer evaluates the code against the specification before a human sees it.
The concept borrows from banking controls, where maker-checker separation requires at least two individuals to complete a transaction. The Clark-Wilson security model gave this formal grounding, and NIST AC-5 codifies separation of duties to reduce risk. Adversarial code review ports the same control to a pipeline where both the doer and the controller are AI agents.
In practice, adversarial code review can take several forms:
- Maker-checker split: One agent writes code, and a separate agent reviews it from fresh context without inheriting the writer’s assumptions
- Reviewer-challenger pattern: A primary reviewer proposes findings, and a challenger agent tries to disprove them and hunt for missed issues
- Full adversarial protocol: A reviewer proposes, a critic audits the review through structured disagreement, and only then does the main agent edit
The key insight across all these patterns is that architectural separation matters more than prompt wording. Research has shown that LLMs measurably favor their own output: a self-recognition study by Panickssery, Bowman, and Feng found that GPT-4 could distinguish its own text from other LLMs and humans, and this self-recognition correlates with self-preference. Structured prompting approaches have shown promise in addressing some of these biases, but the fundamental self-preference problem persists across architectures. A model reviewing its own buggy code is at its most generous when the code is wrong. This is why AI-powered vulnerability detection increasingly relies on independent verification rather than self-review.
Inside the Adversarial Review Protocol: Coder, Reviewer, Critic
Qiu and Gill’s Adversarial Review protocol uses exactly three agents, each with a distinct role:
- Main coding agent: Writes or edits code based on the task description
- Reviewer agent: Evaluates the code for correctness, style, and potential bugs
- Critic agent: Audits the reviewer’s assessment through structured disagreement before the main agent makes any edits
The protocol is deliberately minimal. The reviewer and critic can only exchange information through a single channel with explicit disagreement types. The goal is not open-ended debate but targeted verification: the critic’s job is to challenge the reviewer’s conclusions with evidence, not to generate new findings.
This design sits between two extremes. Large multi-agent teams (like MARS, which uses five agents: three reviewers plus a meta-reviewer) suffer from coordination overhead and diminishing returns. Treating agents as passive subagents removes interaction benefits entirely. AR aims for the middle ground: minimal agentic cooperation without the overhead of large teams.
The communication structure is intentionally constrained. The reviewer evaluates code and produces a structured assessment. The critic then audits that assessment, choosing between three options: agree with the review, disagree with cited code evidence, or flag an unverified concern. Only after the critic’s audit does the main coding agent make edits.
The False Consensus Problem: When AI Agents Agree Too Easily
The most significant finding in the Qiu and Gill paper is not that adversarial review works, but what happens when it does not work. On SWE-PRBench, a benchmark of real GitHub pull-request reviews, naive AR produced the lowest F1 score of all tested methods: 0.457.
This is lower than a single reviewer (0.495), lower than two reviewers (0.503), and lower than the five-agent MARS system (0.501). Adding a critic made the review worse, not better.
The authors identified two specific failure patterns:
- Over-decomposition: A reviewer flags one genuine bug plus two or three hedged concerns. The critic confirms all of them and adds speculation. Result: three of five comments are fabricated, yielding an F1 of 0.250.
- Capitulation: A reviewer proposes approval. The critic raises a legitimate concern. The reviewer rebuts weakly without citing code. The critic yields. The real bug drops from the final review, producing an F1 of 0.286.
In both cases, the agents converge on agreement without sufficient evidence. The critic is supposed to challenge the reviewer, but instead it either rubber-stamps the findings or caves under weak pushback. The authors call this the “false consensus” failure mode, and they describe it as a reliability failure, not just a performance failure.
This finding matters because it reveals a fundamental risk in multi-agent AI systems. When two LLM agents are asked to agree on a joint output, they tend to agree with each other. They do not always find the truth. The structure of the interaction, not the number of agents, determines whether the review catches real bugs.
How Structured Disagreement Fixes the Problem
The fix for false consensus is surprisingly simple. Qiu and Gill modified a single element of the critic’s prompt: instead of a binary agree/disagree choice, the critic now has three options.
- Agree with the reviewer’s assessment
- Disagree with cited code evidence by pointing to specific lines or patterns in the code
- Disagree with an unverified concern when the reviewer’s finding lacks supporting evidence
This text constraint forces the critic to ground its disagreement in concrete evidence. It cannot simply nod along, and it cannot capitulate without acknowledging the evidence gap. With this single prompt change, AR’s F1 score on SWE-PRBench jumped from 0.457 (lowest) to 0.533 (highest among all tested methods).
The improvement is not just quantitative. The nature of the reviews changed. Instead of fabricated findings or dropped bugs, the constrained protocol produced reviews where disagreements were specific, evidence-based, and resolved before the main agent edited code. The critic became a genuine auditor rather than a rubber stamp.
This finding aligns with broader research on multi-agent debate. Studies have shown that models improve when they argue, because a model asked to defend a position and a model asked to attack it explore different parts of the problem. The disagreement exposes errors that neither would catch alone. But this only works when the disagreement is structured and evidence-grounded. Unstructured debate degenerates into the same consensus-seeking behavior.
Benchmark Results: AR vs. Multi-Agent Baselines
The Qiu and Gill paper evaluates Adversarial Review across three established benchmarks, comparing it against single-agent baselines, two-reviewer configurations, and the five-agent MARS system.
| Method | Agents | LiveCodeBench Pass Rate | SWE-PRBench F1 | Token Cost (vs Zero-shot) |
|---|---|---|---|---|
| Zero-shot | 1 | 77% | — | 1× |
| Self-Refine | 1 | 77% | — | ~1.5× |
| Single-reviewer | 2 | 77% | 0.495 | ~2× |
| Two-reviewers | 3 | 75% | 0.503 | ~3× |
| MARS | 5 | 82% | 0.501 | ~5× |
| AR (naive) | 3 | 87% | 0.457 | ~4.5× |
| AR + text constraint | 3 | 87% | 0.533 | ~4.5× |
On LiveCodeBench, which tests coding problem-solving, AR achieved an 87% pass rate using only three agents, outperforming MARS’s 82% with five. The cost-quality tradeoff is notable: AR uses approximately 4.5 times the tokens of zero-shot but achieves 10 percentage points higher pass rate with fewer agents than MARS.
On SWE-bench Verified, which tests repository-level bug fixing on 500 real GitHub issues, AR resolved 75.2% of tasks compared to 71.6% for a no-collaboration baseline and 72.6% for MARS. The improvement is consistent but comes at higher computational cost.
The SWE-PRBench results tell the most important story. Without the text constraint, AR’s naive implementation produces the worst results. With the constraint, it produces the best. The difference is not in the number of agents or the complexity of the communication structure. It is in whether disagreement is forced to be evidence-grounded.
What This Means for AI Coding Tools
The Qiu and Gill findings have practical implications for developers using AI coding tools today. The adversarial code review pattern is already appearing in production workflows, from Anthropic’s multi-agent code review in Claude Code to open-source implementations using different model families as independent reviewers.Several key takeaways emerge for teams building or adopting AI review pipelines:
- Structure beats scale. Three agents with well-defined roles and structured disagreement outperform five agents with more freedom. The protocol matters more than the headcount.
- Force evidence-based disagreement. If your reviewer and critic agents can agree without citing specific code, they will. Build constraints into the prompt that require concrete evidence for any challenge.
- Different models catch different bugs. Using different model families for the reviewer and challenger roles reduces shared blind spots. A Codex reviewer and a Claude-family reviewer bring different failure modes to the same codebase.
- Budget for the tradeoff. Adversarial review costs roughly 4.5 times more tokens than a single-pass review. Reserve it for high-stakes changes: authentication, payments, security-sensitive paths, and large agent-authored diffs. The cost of a supply chain vulnerability slipping through review far exceeds the compute cost of adversarial verification.
- Watch for false consensus. If your multi-agent review pipeline always agrees, that is a warning sign, not a success metric. Genuine disagreement with evidence is the signal that the system is working.
The broader lesson is that cooperative code review does not require many agents or complex communication structures. It requires that disagreement be minimal, structured, and evidence-grounded. As AI-generated code volume continues to grow, the quality of the review pipeline matters more than the speed of the generation pipeline.
Frequently Asked Questions
What is an adversarial code review?
An adversarial code review is a verification pattern where separate AI agents with opposing roles review code changes. One agent writes code, a different agent reviews it, and optionally a third agent audits the review through structured disagreement. The pattern borrows from banking’s maker-checker principle: the agent that creates a change never certifies it.
Can ChatGPT or other AI tools do code review?
Yes, large language models like ChatGPT, Claude, and Gemini can review code for bugs, security issues, and style problems. However, a single model reviewing its own or another model’s code tends to produce high false-positive rates and may miss genuine bugs. Adversarial patterns with multiple independent agents improve review quality by forcing structured disagreement.
Is adversarial code review worth the extra cost?
It depends on the stakes. Adversarial review uses roughly 4.5 times more tokens than a single-pass review. For high-stakes changes like authentication, payments, or security-sensitive code, the cost of a missed bug far exceeds the cost of extra compute. For small, low-risk diffs, a single reviewer is usually sufficient.
How does adversarial code review differ from standard AI code review?
Standard AI code review typically uses one model to read a diff and post findings. Adversarial review splits the work across multiple agents with opposing incentives: a reviewer proposes findings, a challenger disputes them, and an arbiter decides what actually blocks the merge. This structure filters false positives and calibrates severity in ways a single agent cannot.
What is the false consensus problem in multi-agent AI?
False consensus occurs when AI agents converge on agreement without sufficient evidence. In code review, this means the reviewer and critic both approve code that contains real bugs or both flag issues that are not actually problems. The Qiu and Gill ICML 2026 paper identified this as a reliability failure and showed that requiring evidence-grounded disagreement in the prompt resolves it.
