Back to News & Insights
Artificial Intelligence August 29, 2026 · 12 min read

The Best Model Pair in My Field Test Was Also the Least Trustworthy

v0.2.1 RELEASED — Aug 28, 2026. Release notes · Field test report v0.2.1 Key Finding: The Mistral...

The Best Model Pair in My Field Test Was Also the Least Trustworthy

v0.2.1 RELEASED — Aug 28, 2026. Release notes · Field test report v0.2.1 Key Finding: The Mistral effect is confirmed. DeepSeek+GPT (two different labs, no Mistral) converged at 0.246 — same as the homogeneous GPT+GPT control (0.273). Mistral, not lab diversity, drives productive debate. The recommendation changes from "pick from different labs" to "always include Mistral." Also new in v0.2.1: Row-count invariants at all 5 pipeline seams (the 2,333->359 collapse is eliminated), false-negative measurement (1.7-3.4% missed-issue rate, first recall data ever), 55 new unit tests.

AdversarialDebate v0.2.0 is released — v0.2.1 shipped Aug 28. GitHub: https://github.com/deghosal-2026/adversarial-debate v0.2.1 Release Notes: https://github.com/deghosal-2026/adversarial-debate/blob/main/docs/reference/release-notes-v0.2.1.md v0.2.1 Field Test Report: https://github.com/deghosal-2026/adversarial-debate/blob/main/docs/field-test/v0.2.1/FIELDTESTREPORT.md README: https://github.com/deghosal-2026/adversarial-debate/blob/main/README.md CHANGELOG: https://github.com/deghosal-2026/adversarial-debate/blob/main/CHANGELOG.md PyPI: https://pypi.org/project/adversarial-debate/ v0.2.0 field test report: https://github.com/deghosal-2026/adversarial-debate/blob/main/docs/field-test/v0.2.0/FIELDTESTREPORTfullcorpus.md v0.1.0 field test report: https://github.com/deghosal-2026/adversarial-debate/blob/main/docs/field-test/v0.1.0/FIELDTESTREPORTfullcorpus.md

It had the best average convergence score. It had the best verdict rate. It was the cheapest to run efficiently. It looked like the clean proof of the whole project thesis.

What looked like the best debate pair was often doing something worse than disagreement: it was surrendering.

This post is about the most dangerous lesson from AdversarialDebate v0.1.0: a multi-agent system can produce excellent aggregate metrics for the wrong reason.

In v0.1.0, the strongest pair by raw metrics was DeepSeek + Mistral, and the danger was capitulation cascade.

In v0.2.0, DeepSeek + Mistral moved into a validation role while GPT + Mistral became the best full-corpus default. That means the exact pair ranking changed, but the core warning stayed the same:

Strong-looking multi-agent metrics are not automatically trustworthy multi-agent metrics.

If you only look at verdict counts, convergence scores, or API efficiency, you can mistake low-quality reasoning for success.

| Pair | Avg Score | Verdict Rate | Concessions | |---|---:|---:|---:| | DeepSeek + Mistral | 0.982 | 97% | 2,352 | | GPT + Mistral | 0.754 | 48% | 1,728 | | GPT + GPT | 0.688 | 57% | 1,444 | | Gemini + DeepSeek | 0.622 | 10% | 1,470 | | Gemini + Mistral | 0.512 | 4% | 1,073 | | GPT + Gemini | 0.357 | 4% | 727 |

I would have written: diversity wins cross-region pairs are best the most diverse pair is the best production default

Because the same pair that hit 97% verdict rate also had a 65% capitulation rate.

Relevant v0.1.0 sources: Full report: https://github.com/deghosal-2026/adversarial-debate/blob/main/docs/field-test/v0.1.0/FIELDTESTREPORTfullcorpus.md Small-corpus report: https://github.com/deghosal-2026/adversarial-debate/blob/main/docs/field-test/v0.1.0/FIELDTESTREPORTsmallcorpus.md Learnings log: https://github.com/deghosal-2026/adversarial-debate/blob/main/docs/field-test/v0.1.0/learnings.md

v0.2.0 gave me a cleaner pair story: GPT + Mistral became the best full-corpus default: 0.536 average convergence, 2/150 verdicts, 2,927 concessions DeepSeek + Mistral stayed stronger than the negative control on its validation subset: 0.572 average convergence, 1/36 verdicts, 936 concessions GPT + Gemini remained the weak negative control: 0.033 average convergence, 0/24 verdicts

So the exact v0.1.0 headline, “the best pair was the least trustworthy,” is most true for v0.1.0 itself.

You cannot trust pair-level success metrics unless you also inspect how that success was produced.

Relevant v0.2.0 sources: Full report: https://github.com/deghosal-2026/adversarial-debate/blob/main/docs/field-test/v0.2.0/FIELDTESTREPORTfullcorpus.md Field test plan: https://github.com/deghosal-2026/adversarial-debate/blob/main/docs/field-test/v0.2.0/field-test-plan.md CHANGELOG: https://github.com/deghosal-2026/adversarial-debate/blob/main/CHANGELOG.md

That is what I call a debate where one side concedes almost everything immediately, usually in round 1, with zero real rebuttal pressure.

Want to discuss this further?

Book a free strategy call with our team to see how these insights apply to your specific business goals.

Book a consultation