I build a scanner that looks for cross-venue arbitrage in prediction markets. The premise is simple enough to fit in one sentence: Polymarket, Kalshi, PredictIt and Limitless price the same real-world events independently, and when they disagree far enough you can buy YES on one venue and NO on the other for a combined cost under $1.00. Exactly one leg pays out $1.00 whichever way the world goes. It isn't a directional bet — you don't need to know who wins.
That's the theory. This post is about the part of the theory that turned out to be wrong, and what it cost to find out.
A correctly matched hedge has a property you can check mechanically: exactly one leg pays. If you hold YES on one venue and NO on another for the same question, the two outcomes are complementary. One resolves true, one resolves false. Always.
Both won, or both lost. Nearly a third of them. That isn't a rounding error or a fee subtlety — it's a violation of the instrument's defining property. Something in my matching was pairing markets that were not actually complementary.
My first assumption was the obvious one: the matcher is sloppy, tighten the similarity threshold. That assumption was wrong, and it took a classification pass to see why.
Instead of tuning a threshold, I split the 127 by a single question: do the two legs name the same subject?
The split was stark, and it separated two completely different problems that a similarity score had been quietly blending together. (23 named different subjects, 97 named the same subject, and 7 were too ambiguous to classify either way.)
Genuine false matches. Two markets that read alike and are not the same event. The two that hurt most:
Cross-city weather. "Highest temperature in Atlanta 92-93" and "highest temperature in Miami 92-93" differ by exactly one token out of about twelve. IDF-weighted similarity scored them 0.65 — comfortably above threshold — and paired two different cities.
The insight is that a city is not one signal among many in a weather market. It is the market. So it can't be left to a bag-of-words score to weigh; it needs its own guard:
Then, in the scoring function, a disjoint pair of cities is killed rather than merely penalised:
Note the comparison is overlap, not equality. Equality would reject "Portland" against "Portland Timbers" — a real pair. Overlap kills only genuinely disjoint names. That distinction shows up again and again in this codebase; matching on exact sets is almost always too brittle for titles written by different humans at different venues.
A cheap detail that cost real time: my first version of that regex terminated the lazy capture on a bare on, which quietly truncated "Boston" to "bost". The terminators have to be word-bounded — \bon\b|\bbe\b|\bbetween\b.
Opponent inversion. This one is my favourite, because both legs are correct in isolation and the pair is still nonsense. Venues name the same fixture two different ways: Polymarket: "Will Las Vegas win?" Kalshi: "Miami vs Las Vegas — Miami"
Buy YES on the first and NO on the second and you have… one directional bet placed twice. Both pay when Las Vegas wins. Both lose when Miami does. Every token matches. Similarity is high. It settles as a double win or a double loss depending purely on luck.
The fix parses which side each title actually backs — the tail after the dash in "A vs B — pick", otherwise the subject of "Will X win":
These pairs were correctly matched. Same event, same wording, similarity around 0.80. The markets were genuinely identical. And both legs still settled the same way.
The reason: each venue resolves from its own source. Two venues can list "Will the high temperature in Denver exceed 90°F?" and read two different weather stations. One station reads 90.4°F, the other 89.8°F. Both legs settle YES, or both settle NO. The markets agree; the oracles disagree.