An AI coding agent finishes the feature. The tests pass. The explanation sounds reasonable.
That question can change the shape of a project. Now “done” needs to mean something observable. The answer needs an experiment, and the experiment needs a way to be wrong.
This tutorial builds a tiny task list around that idea. By the end, you will have a working application, two deliberately broken versions, and a folder of evidence showing which promises each version keeps.
I use evidence-driven development to describe this working habit: write down a claim, decide what would disprove it, run the relevant experiment, and let the result govern the next decision.
The coding agent can help at every step. The evidence must remain inspectable without trusting the agent's summary.
The name predates coding agents. Jan Bosch's 2017 book, Speed, Data, and Ecosystems, includes an Evidence-Driven Development chapter. Its contents connect requirements, hypotheses and experiments. I'm using the term for a practical agent-assisted workflow, without claiming to have invented a methodology. Publisher's catalog.
There is related work in AI, too. Xia and colleagues describe evaluation-driven development and operations, or EDDOps, as a continuing feedback loop for LLM agents. Their paper addresses a broader agent lifecycle than the small application we will build here. EDDOps paper.
TDD still gives us a useful red–green–refactor loop. BDD helps express behavior through examples people can discuss. In this tutorial, EDD names the discipline of keeping the claim, execution conditions, observed result and resulting decision connected. Those practices fit together. TDD, Given–When–Then.
Our app is Tiny Tasks. It adds tasks and lists them. Python and SQLite are enough; the application and experiment runner use only the Python standard library. Use Python 3.10 or newer with SQLite support.
The companion repository contains the complete source, cover artwork and recorded evidence. Clone it to follow along:
The checkout selects the source revision used in this article. The add command returns a task. The list command starts a new process and reads it back. The default database lives at .local/tasks.sqlite.
The request ID belongs to the user's intended action. When the caller retries that same action, it sends the same ID. A genuinely new task gets a new ID—even if the title happens to match.
That small distinction gives us something worth proving. Write a claim that can lose
| Claim | A result that disproves it | |---|---| | Acknowledged tasks survive a process ending | A new process cannot read the saved task | | Repeating a create request returns the original task | A retry creates another task or returns a different ID | | A request ID cannot silently acquire a different meaning | Reusing it with a different title succeeds or changes saved data | | Invalid input leaves the list alone | An empty title adds a task |
The complete contract is embedded here, with a commit-pinned copy in the repository:
{% embed https://gist.github.com/copyleftdev/01e8f0e2b919392f1a7a1a6fc1fa8665 %}
The repository records those claims in experiments/001-reliable-tasks/experiment.json. It also declares three restart rounds, eight concurrent retry processes, two independent repetitions, and the scenarios that must actually execute.
These numbers are a small teaching workload. They are not a statistical reliability estimate or a throughput target.