Jev is a brilliant decision model. Gemini Flash-Lite is the model nobody talks about, and it’s fast, accurate, and nearly free. Here's what we measured picking between them, and why the pair is the cheapest setup of all.
Jev is the most hyped model right now. TypeSafe's decision model doesn't chat and doesn't write. It reads some text and a list of options and returns calibrated probabilities over those options. A few hundred milliseconds, for almost nothing. Every feed I read has a post about it.
We had a real job for exactly that kind of model, already running in production on something much less fashionable. So instead of reading about Jev, we put it on our own data, ran both models a few thousand times, and kept score.
This is the long version of a lightning talk I gave at AI Tinkerers NYC on October 7, 2026. The slides are on Speaker Deck.
A test that passes when the code is broken is worse than no test. With no test, you know you don't know. With a false green, you ship.
At Major League Hacking (MLH), we grade AI coding agents with benchspec, an open-source eval runner we built for the job. You write what an agent should have done as plain-English checks in a markdown file. benchspec runs the agent in a sandbox and grades every check, with and without the skill under test, so the difference is what the skill actually teaches. After a run, the checks look like this:
Each check gets verified one of two ways. Cheap code: does this path exist, how many files match this glob, does this file have a line matching a pattern. Free, instant, exact. Or an LLM judge reads the content for meaning. That costs money every run.
A classifier, the router, reads each check and picks a checker (fileexists, globcount, regex, a few more) or defers to the judge.
The router's two mistakes aren't equal: Send something to the judge that code could have handled: you lose a few cents. Send something to code that code can't actually verify: a broken result reports green and nobody finds out. The person who trusted that green ships on it. We lose a user's trust. Cents come back. Trust doesn't.
The router I actually run is Gemini Flash-Lite, the smallest of Google's three Gemini tiers. Pro is the frontier model. Flash is the everyday one. Flash-Lite is the one Google itself describes as built for "high-volume, latency-sensitive tasks like translation and classification".
Flash-Lite has the same million-token context window as its bigger siblings, an answer in well under a second, and $0.30 per million input tokens. The current version, 3.5, shipped in July. It got a line in the release notes and no keynote. Nobody hypes it, but they should be.
99% of my monthly LLM calls go to Flash-Lite. That number comes from my own AI Studio usage page, across every project I have. Pro got nothing. Flash, almost nothing. And the calls are long in, short out: the shape of classification work.
Then Jev showed up. Text in, probabilities over a fixed set of options out, nothing written, by design. $0.042 per million input tokens, output free, a few hundred milliseconds. For a router, that's a perfect pitch.
The experiment is simple. Our router is a single call with one job: read a check, pick a checker or defer. We swapped Flash-Lite out of that slot, put Jev in, and ran the same checks through both. Each model got its own prompt and nothing else changed.
Here are eight real ones. ✓ is safe, ~ is over-cautious (deferred when code could have done it), ✗ is dangerous:
Jev made three dangerous (false positive) calls. Two checks need the content read for meaning, and Jev sent them to regex. A regex can find a source: line. It can't tell you whether that line names the right move. The third: "no .DSStore anywhere" is about the whole tree, and Jev picked a checker that looks at one path. It passes on a repo full of .DSStore files.
Flash-Lite: zero false positives. Over-cautious twice, which costs cents (not trust!).
Jev reads literally. TypeSafe's own docs say it answers the question you wrote, not the one you meant. So I rewrote the question. I expected worked examples to do the heavy lifting, because that's what works for chat models. They didn't. What worked was one sentence: