I built two systems this year that try to solve the same problem from opposite directions.
PlannerCritic puts an LLM critic in a loop with a deterministic gate layer. The gates own the safety contract. The critic is advisory. A planner decomposes a goal into a typed plan. A critic audits every subtask. The plan is revised until approval — or escalated to a human.
AdversarialDebate puts two LLMs in a structured debate. The protocol owns the independence. The models are the evidence. Two reviewers analyze the same artifact without seeing each other's answers, debate their conclusions point by point, and produce either a converged decision — or a structured disagreement report that preserves the dissent.
Both systems are about the same question: how do you know when an LLM's judgment is wrong?
I thought the answer was architecture. Build the right structure, put code where the model can't be trusted, and the system becomes reliable. That was the thesis of both projects. PlannerCritic bet on code. AdversarialDebate bet on structure.
I was half right. The architecture worked. The systems are reliable. But both had the same blind spot, and it took me months to see it — because the blind spot made the metrics look better, not worse.
PlannerCritic's bet was code over model. The decision path is a frozenset of blocker-eligible families — unsafesequencing, weakrollback, unverifieddependencies, feasibility. The LLM critic can label a finding as blocker, but if the family isn't in the frozenset, the system downgrades it to a warning. The model's severity label is decorative. The family is load-bearing.
AdversarialDebate's bet was structure over consensus. A revelation gate ensures neither reviewer sees the other's output before committing. The debate protocol requires point-by-point responses — CONCEDED, REBUTTED, or CARRIED. The convergence score measures movement, not agreement. Two models that agree without engaging get a low score. Two models that genuinely challenge each other get a high one.
On paper, those bets are compatible. Code guards against structural under-claim. Debate guards against shared hallucination. Together, they cover more ground than either alone.
But each bet had a failure mode that the other system's design would have caught immediately. And neither system caught its own — because the failure made the dashboard look like things were improving.
In PlannerCritic, the deterministic gates own safety. They parse the plan's structure — ordering, preconditions, rollback, verification — and they block anything that doesn't meet the contract. By v0.2.2, the blocker counts across the full 183-goal sweep were dominated by structural families: unsafesequencing at 226 blockers, unverifieddependencies at 185, weakrollback at 86.
Then a community reader named Artjoms Stukans left a comment that reframed how I thought about the entire system:
"If one blocker class stops firing after some refactor, your numbers only look better. 226 becomes 40 and that reads like plans got safer."
He described a Kubernetes incident where four releases in a row never actually ran. An old ReplicaSet kept one pod Running. Every health check passed. Every smoke test was green. Nothing said a word — because the pod was still in a "healthy" state, even though the deployment was broken.
That is exactly the shape of PlannerCritic's problem. A gate that silently stops firing produces better metrics. The blocker count drops. The dashboard reads as improved safety. The variance signal — labelfliprate, evidencedriftrate — the very metrics the project uses to surface critic problems — vanishes exactly where the safety contract moved. The critic is watched. The gates are trusted.
AdversarialDebate already had the pattern for catching this. Each debate produces a transcript, and the capitulation detector reads that transcript for the shape of failure instead of trusting the aggregate convergence score. It doesn't ask "did the debate converge?" It asks "did one side concede everything in round 1 without a single rebuttal?" Those are different questions, and only the second one catches the failure.
PlannerCritic had no equivalent. It tested the gates at build time. It did not monitor them at runtime.
The fix was the Gate Canary — ten fixture pairs, one per gate class, each with a known-good plan and a known-bad plan. The CLI command plancritic gates canary --check runs them in under a second at zero LLM cost and exits 1 if any gate stopped firing on its bad plan. It's a dumb check. It's a cheap check. It catches exactly the failure mode that would have been invisible in every other metric the system produces.