Back to News & Insights
Artificial Intelligence August 17, 2026 · 11 min read

Don't Give the Model SQL

My health data has six traps in it that have each already produced a wrong answer. Given SQL, a model walks into all six. Told about them in a prompt, it avoids them most of the time, which is worse.

Don't Give the Model SQL

I built a web app to answer questions about my own health data, for a reason that is embarrassingly small. I already had a perfectly good way to ask them: a Claude Code skill that queried the database and reasoned over the results. It worked well. It also can't run on Claude iOS, and the moment I actually want to ask "should I be worried about this" is standing in a kitchen at 6am, not sitting at a desk.

So it lives at a URL, behind Google sign-in allowlisted to exactly one address, with the source public minus the data. It reads the TimescaleDB database I wrote about in The Past Keeps Changing, and it answers two different kinds of question in two deliberately different ways.

(Obligatory: none of this is medical advice, and the thresholds in it are tuned to one person's circumstances. If you copy the rules you will get advice calibrated to somebody else's body.)

The coach view is deterministic. Protein adherence, deficit versus the scale, weight trend, overreaching, stalled lifts, data freshness. Each one is a pure function in lib/signals/, unit-tested against fixtures, no LLM involved anywhere. A verdict traces to a rule instead of to a vibe, and unknown is a first-class result distinct from ok, which matters because this dataset produces unknown constantly.

The ask view is an LLM. Claude, server-side, over thirteen read-only tools. It handles the questions nobody wrote a rule for: am I stalling on squat, how big is my deficit actually, what's my VO2max doing.

The interesting design work is all in the second one, and it's mostly about what I refused to build.

Hand the model SQL. Give it the schema, a read-only connection, and let it write queries. It's the design every "chat with your database" demo uses, it takes an afternoon, and for a database this small the query cost is irrelevant.

I rejected it, because there are six traps in this data and every one of them has already produced a wrong answer in this project, on a page I was looking at, at least once. Today is a Partial Day. Its numbers are still accumulating. Compare it against a completed day and you invent a fast. A gap is not a zero. An unlogged day is unlogged, not a day I ate nothing. Apple shadow-copies every Liftosaur session. Query the workouts view naively and training volume roughly doubles. energybalance overstates the deficit by about 2.7x. Over the last 30 complete days it reports an average intake of 1,602 kcal against 3,216 burned, a net of −1,614 a day, which predicts losing 3.2 lb a week. The scale over the same window says 1.2. Basal energy is a formula estimate from weight, height and age, and watch-measured active energy runs generous. Both are real numbers whose difference is not a measurement. A single weigh-in is noise. Day-to-day scale movement is water. reps: 0 is a set that was attempted and failed, not a set that's missing.

Given SQL, a model walks into all six, which says nothing about the model and everything about the schema. The traps are invisible from it. Nothing about a column called calories tells you today's value is half-formed.

I could put all six traps in the system prompt, and if I do, the model gets them right most of the time. That's what decided the design, because most of the time is the worse outcome.

A tool that's wrong every time gets caught on day one and thrown away. A tool that's right ninety-something percent of the time gets trusted, and then the rare wrong answer arrives wearing exactly the same confident formatting as the right ones. I have no way to spot it, because the whole reason I'm asking is that I don't already know the answer.

So the traps are foreclosed by the shape of the tools rather than by instructions: Windowed queries end in AND observedon < todaylocal(). The Partial Day isn't excluded by the model remembering to exclude it. It isn't reachable. Gaps stay absent rows. Nothing zero-fills, so there's no zero to misread. No tool reaches Apple's workout view at all. The double-counting trap isn't documented for the model's benefit, it's unreachable through the API surface. energybalance cannot be fetched without its reality check arriving in the same payload. You can't get the misleading number on its own.

The prompt still describes all six traps, because a model that understands why a window ends where it does gives better answers than one that just gets truncated data. But the prompt is not what's holding them. If the prompt were deleted tomorrow the answers would get worse and they wouldn't get wrong in those six specific ways.

There's also no write tool, and I mean that literally rather than as shorthand for one that's disabled or gated behind a confirmation. There is nothing to disable. Program changes stay in Liftosaur and macro targets stay MacroFactor's call, so the chat has no legitimate reason to mutate anything. The reasoning is in ADR 0006.

Then I nearly shipped a tool that was wrong in a way none of that structure protects against.

The chat needs to know which metrics exist. There's a metriccatalog table holding canonical units and an attention grade per metric, so listmetrics obviously reads from that. It compiled. It typechecked. It returned rows that looked entirely plausible.

The catalog covers 38 of 81 metrics. The other 43 aren't stale junk: sorted by coverage they start with walking and running distance (3,865 days of it, updated today), flights climbed, walking heart rate average, walking speed, step length, plus the entire micronutrient panel, all of it current because MacroFactor logs micros.

So the tool would have worked exactly as written and made the chat answer "I don't have that" about ten years of walking distance sitting right there in the database.

Want to discuss this further?

Book a free strategy call with our team to see how these insights apply to your specific business goals.

Book a consultation