I found 27 workflow branches that were being skipped while every run still finished as COMPLETED.
The condition on those branches could never match. So the step was skipped, the run was marked success, and nothing ever told me. They had been running like that for weeks.
That is the part worth sitting with. Not that there was a bug — there is always a bug. That the system reported success 27 times a day, honestly, while doing nothing.
Does it distinguish "green but semantically idle" from "green and actually processed", or is that still something you catch by comparing runs?
I went to check, expecting to say the data is one level down, in the step rows. Each step has its own status, and SKIPPED is one of the values. So the information had to be there.
It was not there. StepExecutionStatus.SKIPPED had existed in the enum since the beginning of the project and had never once been written. Zero occurrences in the codebase.
A skipped step did not create a database row at all. The loop appended a dict to an in-memory results list and continued. The only trace of a skip lived inside a JSON blob on the execution row.
So: the step view reads rows, which meant a skipped step was invisible in the UI too "which runs skipped something?" had no answer short of parsing JSON a run that skipped every branch and a run that did all the work were identical at every level a human or an alert would look
I had built an enum value to describe a thing, and then never recorded the thing.
The fix took an afternoon. A skipped step now writes a real row with the reason. The run reports how many steps it processed, skipped and failed, derived from those rows so the numbers cannot drift from what the step view shows.
A zero can be legitimate. Quiet day, nothing to send, COMPLETED with 0 processed is correct. The alert gets teeth when the count is two-sided: the run says what it processed, the source says what it handed over, and the two have to tie out.
A scheduled workflow with nothing to do processes nothing every quiet day. My badge would have fired on all of them, been muted inside a week, and then not been there on the day it mattered.
The badge came out. The counts stayed, as plain facts, and the judgement went with the badge.
Two days later I built a deploy script that runs the tests locally and refuses to deploy if any are red. It ends by verifying the deploy.
Every one of them true. The feature I had just deployed was dead. The containers had started before I added the environment variable it needed, so the code shipped and did nothing.
I found it by curling the endpoint and reading the value, which the script had not done.
A deploy verifying that it deployed is not the same as verifying the thing works. Health checks are generic. Effect checks are specific, and specific is the part nobody writes.
I posted the 27-branch story and the thread turned into something better than the article I meant to write.