Two weeks ago I counted 204 guards in my repos and found that 89 % had never been shown they can fail. I fixed that for a batch of them. Then tonight I filed a data-loss report about 1,000 files that were never lost — and nobody, including me, had a number to check it against.
Last time I wrote here, the number was 204 and 11 %: of the automated checks in my repositories that draw a conclusion, only about one in nine could prove it was capable of failing. @devmichael's "AI didn't make me a worse coder, it made me a worse reviewer" is what started that count.
So I did the obvious follow-up work: I made a batch of guards prove they can fail. That fix is real and I'll describe it below, because it costs an afternoon and it works.
Every reviewer — a hand-written check, an LLM judge, a second agent grading the first — gets one known-bad case wired through the live path. Not a unit test beside the pipeline. The same entry point real work uses.
Our benchmark harness runs three gates per case, and a case that misses any of them does not run at all: Unsolved state → must go RED. The check must reject the broken repo. Solved state → must go GREEN. It must accept the fix. Known-bad → must go RED again. We re-plant the original mistake deliberately; if the check still passes, it never understood the bug.
Gate three is the one that earns its keep. It has caught real breakage — including, twice, in guards written that same week to catch exactly this class of problem.
This week I pushed it further, because a harness is only as good as the cases in it, and inventing cases is where everyone runs out of imagination. @shreyasht killed his own token-optimization project after discovering his best result — 97 % savings — came from the run that did no work at all: the agent asked a clarifying question, stopped, and the dashboard crowned it. A metric with no way to punish doing nothing eventually rewards it. His fix is one sentence: measure per solved task, never per task.
The imagination problem has a cheap answer, and it was sitting in our database: the incident history is the test suite. We took ten real recorded failures — the kind with a "what went wrong" field written by someone annoyed at the time — and had a model punch each one into a runnable check. Ten lessons in, ten runnable cases out, thirty gates, all thirty passed. Zero discarded.
One thing that surprised me: the first attempt failed the internals filter, and it failed because the example we handed the model contained a product name in a comment. The model copied it faithfully. The filter caught it. A guard I had written to be paranoid about human sloppiness caught a machine being obedient instead.
@wrobeltomasz described this same discipline independently in a comment thread while this draft sat unpublished: identify the checks, then run them "in simulation mode to confirm that they can indeed respond to invalid input" — moving verification "from statistics in the README to actual system resilience." Two people who have never met, same gate, same reason. That usually means the pattern is real and not a personal quirk.
Here is what all of the above proves: on the day I wired it, that reviewer could say no.
@mk023 said the correcting sentence in a thread under my last post, and gave me permission to quote it here:
"Don't just test that the guard can fail — test that it is still guarding the thing you think it is guarding."
That completes the model. A green check can be lying in three orthogonal ways: falsifiability — can it go red at all; liveness — can it go red against today's system; and aim — is the boundary it defends still the one that matters. Part one only bought me the first. Marco's sentence is the other two, and the rest of this post is me finding out the hard way that he was right.
@jamesandersonh, in a thread about AI workspaces that ship with "guardrails — approval gates, an audit trail, a second agent that reviews the first," put the gap better than my question had: most tools hand you the audit log, not proof the veto still fires.
"A checker nobody has ever seen fail is indistinguishable from a checker that approves everything. The two produce identical logs right up until the day the rubber-stamp waves through the thing that hurts you."
My answer to him was a small thing I now think is the most useful idea in this post: the veto heartbeat.
Surface one date. "Last time this reviewer refused something: 2 days ago." First-class, visible, right next to the uptime number. The known-bad case runs on a schedule; if the last-refusal date ages past that schedule, staleness itself is the alarm. No log-diving, no trusting a vendor's dashboard, no archaeology. One date anyone can read in one glance — and the absence of fresh red finally looks like what it is.