How 41 codified laws, 22 specialist roles, and a file-based memory system stopped an autonomous coding agent from quietly re-breaking the same production defect every few weeks — and why I'm open-sourcing the whole thing as LEO.
Developers reach for Cursor and Copilot to write code ten times faster, and the tools deliver on exactly that promise — which turns out to be most of the problem. Used as advanced autocomplete, an LLM doesn't produce ten times more good code. It produces legacy at ten times the usual rate. You ask for a feature; the model hands back a wall of if/else; you ship it. Two months later the codebase reads like it was assembled by five people who never spoke to each other, the test suite is red more often than green, and the senior engineers who never touched the tool get to point at the wreckage and say, "See? AI is just a toy."
Directing an AI coding agent on real, paying engagements — multi-tenant SaaS platforms, one of them with background AI pipelines — surfaced the same shape of defect more than once, in different files, weeks apart. My own project's changelog (roles/SYSTEMUPGRADEMANIFEST.md — every rule this system has ever added is logged there, with a reason) documents the pattern directly: a rate limiter that could be starved by its own retries because the check-and-consume wasn't atomic at the point of the call. A background worker whose heartbeat proved it was pinging, not that it was making progress — a zombie that looked alive on the dashboard. A held database transaction that outlived the request that opened it and sat there as a lock-holding corpse until something else timed out behind it.
Each time, the agent's code was syntactically perfect. Each time, it passed its own tests. None of this was "the AI is bad at coding" — a frontier model in 2026 writes fine syntax all day. What the log shows is something else: the fix, when it happened, didn't generalize. A defect gets patched in the file where it was noticed; the class of defect — "a heartbeat is not proof of progress," "an if check is not a lock" — has no durable home unless something outside the conversation writes it down as a standing rule. A chat window that scrolls out of context has no mechanism to carry a lesson forward into the next file, the next session, or the next agent entirely. That's not incompetence — it's closer to amnesia at the architecture level, and it is a far more expensive defect than any single wrong line of code, because it recurs on a schedule instead of getting fixed once.
That's the point at which "prompt the AI better" stopped being a credible answer and started looking like an engineering-process problem — the same kind of problem code review, QA, and architecture sign-off were invented to solve, decades before anyone typed "you are a helpful senior engineer" into a text box.
The standard advice for this problem is "treat the AI like a junior developer" — review everything, don't let it merge unsupervised, keep a human in the loop. That's better than nothing. It's also the wrong model, because it assumes the AI already has judgment that mostly just needs correcting on the margins.
It doesn't have judgment to correct. What it has is a high-volume stream of pattern-matched output with no default shape of its own. Pour that stream into an empty context window and it takes whatever shape the last few messages happen to suggest — which is exactly why it drifts. A junior developer is calibrating an internal model of correctness that gets sharper over time. An unconstrained agent has no such model to calibrate; every session starts from the same blank slate, and "use your judgment" is not an instruction it can actually follow, because there's no accumulated judgment there to use.
So the fix isn't heavier supervision of a decision-maker that doesn't fully exist yet. It's building the banks before the water starts flowing: narrow, explicit, interlinked protocols that live inside the project itself — not a paragraph of "best practices" floating somewhere in the model's training data, and not on some vendor's central server you have no visibility into, but files in the actual repository the agent is working in. The agent doesn't need better judgment about which approach fits a given situation. It needs the approach already decided, written down, and sitting in a file it is required to read before it acts — so it isn't guessing by accident which rule applies. It's following the one that was already there.
That's the actual shift LEO makes: not "supervise the junior dev harder," but stop expecting a finished decision-maker and start engineering the channel its decisions are required to flow through.
A single well-crafted prompt is a snapshot of good judgment. It captures what you know to ask for today. It says nothing about the bug you haven't hit yet, the invariant nobody wrote down, or the decision the agent made forty messages ago that it is about to silently re-make, differently, right now.
This is context drift, and it's the actual bottleneck of autonomous coding — not model capability. Give a model enough tokens and it will confidently: Skip the empty state, the 500th test, the timeout on the outgoing HTTP call — because nothing in the conversation made skipping it expensive. Report "should work now" or "practically done" with the same tone it uses for a fact it actually checked. Confidence and correctness are not the same signal, and an unconstrained agent has no mechanism to distinguish them out loud. Never get told no. A single-agent chat has no adversary, no separate reviewer, no QA pass with different incentives than the one that wrote the code. A logic hole ships exactly as fast as the happy path does, because nothing in the loop is structurally positioned to catch it.
None of this is a hypothetical risk. It is the default behavior of an agent operating without a written, load-bearing process — which is precisely the situation almost every "AI pair programmer" setup puts the model in today: one soft system prompt, infinite goodwill, zero enforcement.
Most people trying to control the cost of agentic coding optimize the wrong variable. They shrink each individual prompt and count tokens per request, treating a smaller ask as a cheaper one. That's measuring the wrong unit. A hundred disorganized double-check loops spread across five million tokens produce a worse result than two or three tightly organized ones spread across that same five million tokens. Same spend, radically different outcome — because the cost was never the tokens themselves. It was the absence of a structure telling each loop what it was actually supposed to verify, against what written standard, and who signs off when it's done. Shrink the prompt and you still pay for a hundred aimless loops. Give the loop a gate and a criterion instead, and it often takes two or three.
I come from an unusual angle for this problem. Before I wrote a line of production code, I spent five years in Emergency ICU. That environment teaches you something that transfers directly and uncomfortably well to shipping software with an autonomous agent: zero-error tolerance is not a slogan, it's a set of protocols that exist because someone already paid for the version without them. You don't re-derive the correct response to a cardiac arrest from first principles under pressure — you follow a checklist that encodes every prior failure, because "use your best judgment" is precisely the instruction that fails under load.
I'll be straightforward about where this actually started, because the honesty matters more than the polish. I came into software development as a junior in the middle of the hiring freeze that's reshaped this market over the last few years, and the catch-22 inside it has a specific, brutal shape: you can't get hired without production experience, and you can't get production experience without being hired. LeetCode katas and toy CRUD projects don't close that gap — they were never built to carry the weight of a real system with real users and real consequences for getting an invariant wrong. I felt that wall directly, and it wasn't only a financial problem. It was a quiet, ongoing doubt about whether the path you actually want is still open to you at all, when every door that would let you prove yourself asks for the proof first.
So the bet underneath LEO was never really a career strategy. It was closer to: if the industry won't hand a junior the reps, and a coding agent can write code but has no engineering department to write it inside of, then build the department myself and let the agent supply the reps the market wouldn't. Production-grade SDLC discipline cannot be learned from LeetCode, and it cannot be prompted into existence one clever system message at a time. It has to be architected — as a real, versioned, falsifiable rule system that an agent is required to operate inside of, the same way a hospital shift operates inside protocols nobody re-litigates every time.
LEO is not a library, a plugin, or a hosted product. It's a written constitution — a .cursorrules file plus a 127-file, ~254,000-word role library — that any coding agent with file-system/tool access can load instead of a one-line personality. It gives the agent: A single entry point, @LEAD, that routes every request to a named specialist instead of one model trying to simultaneously be the architect, the developer, and the auditor of its own work in the same breath. Twenty-two roles with real jurisdiction — @ARCH owns the database and API contracts; @PRINCIPLE owns whether a state is logically reachable at all, independent of whether it's technically buildable; @QAARCH audits business logic before anything ships; @PENTEST holds a genuinely blocking security verdict, not an advisory checklist; @DEV is the only role permitted to touch code, and it is allowed to refuse — to raise a MODEL BLOCKER — rather than guess at a hole in the spec. Forty-one Absolute Laws, several written the week a specific defect happened, not from a whiteboard session — the project's own changelog, roles/SYSTEMUPGRADEMANIFEST.md, logs them with dates. The rate-limiter-and-heartbeat incident above (v6.25) became a law about atomic check-and-consume at the point of the call — not a comment, a permanent, greppable rule that a @QAARCH audit now checks for on every future pipeline — and a redefinition of "heartbeat" itself: a pulse without progress is not aliveness. The lock-holding-corpse transaction became Law 35: every session in the architecture spine now carries a numeric idleintransactionsessiontimeout, not a hope. These aren't abstract best practices; they're scar tissue, converted into text the agent cannot argue its way around because it's not a suggestion — it's the operating rule it's loaded under. A gate protocol, not a pipeline. The difference matters enormously: a pipeline lets an agent "complete" a phase by declaring it complete. A gate requires a written artifact — an architecture spine, a security threat model, a QA report — that a separate pass checks against a concrete, numeric criterion. roles/LEADANTICHECKBOXPROTOCOL.md exists specifically to catch phrases like "most likely implemented" and force a real answer: verified, with evidence, or explicitly unknown. A memory that survives the context window. This is the actual fix for the amnesia I started with. Every non-trivial decision — the tenancy model, the threat model, the QA verdict — gets written to a file, not left in chat history. Three weeks and forty conversations later, a completely fresh agent session with zero memory of any prior chat reads that file and inherits the decision exactly. It doesn't re-derive it, and it doesn't quietly re-decide it differently. State, not history — that single principle is doing more work against context drift than any prompting trick I've found. Concretely, this means treating your IDE's own indexing of the project — Cursor's index, a plain file read, a grep across the repo — as the durable memory, and the chat window as what it always was: disposable scratch space. Of every reliability trick I tried, moving memory out of the window and into the file system is the one that actually held up once the project got large. A human hand on the only lever that matters. LEO can evolve its own rules after an incident — but only via an explicit @EVOLVE command from a person, never automatically. And separately, Law 40 — the one I'd call the most important line in the whole system — states plainly that the agent never runs git commit, git push, or git merge, under any framing, even if you paste the exact commands and ask twice. An autonomous agent that can silently rewrite its own rules and publish its own work to history is not a tool anymore; it's an unaccountable process. LEO deliberately keeps a human's hand on both levers.
The obvious way to control both cost and hallucination in a multi-role agent system is isolation: give each role the minimum context it needs, wall roles off from each other, and assume a smaller context means fewer wrong guesses. I built LEO the opposite way, after watching isolation produce more invented behavior, not less.