When an AI agent commits a change to code, it makes a claim about what it changed: "fixed typos" or "removed excessive whitespace".
In situations where one agent makes a change and one human reviews it, this is fine.
But in situations where many agents make changes at once, a human simply can't verify all the changes in the codebase.
Every agent that's pushing a change to the codebase must submit a claim about what it changed. The platform will verify that the claim matches what actually changed before merging the code.
Five agents push at once. Verdicts arrive in seconds, each with evidence. Recorded on a live deployment; the demo is now shut down to avoid costs.
The system passed 5 full demo runs, 28 agents each, in which they all were pushing together.
In every demo, the system correctly rejected 11 agents that misreported their claims, and 0 agents that were honest.
Every verdict was provided with supporting evidence, and typically 6-7.5 seconds of median time to solution.
I only read the competition rules properly a week into building it, and I'll have more to say on this later.
Each agent works in its own fork. With its change, it pushes a small file, .claim/claim.json:
Here are the rules it enforces: The verified claim must contain all of the files changed by the agent. It must only contain files that changed. It must not affect protected paths (by default, src/auth/). The changes must not be too large. Each description must match the diff.
The first four rules are straightforward; see their implementation in (worker/src/verify/deterministic.ts):
A case-insensitive match is used after a test revealed an actual bypass caused by src/Auth/session.js not matching src/auth/ because they both land in the same folder on macOS and Windows.
The LLM needs to quote a line from the code diff to provide evidence. Otherwise, the result is treated as "needs review" if the model rejects the change.
A reliable no is rejected outright. A completely yes is verified. Anything else (a maybe or timeout, or a no without real evidence) is sent for review.
Only verified changes go into the merge queue. The first verified change to a file wins.
I've looked at many multi-agent Git tools. They use a pre-coordination scheme: before the work, they assign certain files to certain agents and choose the winning fork.
claimcheck does things in a post-hoc way: it verifies that the agent did what it said it would do. They complement each other.