Disclosure: I'm the founder of Popgot, which I use as the example below. The problem and the approach apply regardless of what you build on.
Every price comparison project I've seen starts the same way: scrape a bunch of retailers, store the prices, sort ascending. And then it produces garbage rankings, because price is not a comparable field.
| Listing | Price | Count | | --- | --- | --- | | Brand A | $5.99 | 16 | | Brand B | $6.99 | 20 | | Brand C | $11.94 | 40 |
Sort by price and Brand A "wins" at $5.99. Sort by cost per battery and the order flips completely: Brand C is ~29.9c per cell, Brand A is ~37.4c. The cheapest listing is the worst deal on the page.
The naive fix is "just divide price by quantity." The problem is that quantity almost never exists as a clean number. It's buried in the title, and the title is written by whoever uploaded the listing: AA Batteries 24 Pack AA Alkaline Batteries, 1.5 Volts, 24 Count 48-Pack (2 x 24) Double A
So you end up writing a title parser. Then you discover the same product needs a different unit depending on the category: per fluid ounce for detergent, per serving for protein powder, per 100g for coffee, per sheet for paper towels. Then you discover that some categories need a spec filter before unit price is even meaningful. A fish oil at 20c per serving isn't cheaper than one at 34c per serving if the first one has half the EPA+DHA. You're comparing two different products.
That last part is the piece people underestimate. Normalization is only valid within a set of products that actually satisfy the same requirement, which means something has to read the label, not just the title.
This is the problem I ended up building Popgot around, so rather than describe it abstractly, here's the shape of the data. The developer API returns listings with the unit math already done:
The fields that matter for this problem are unitcount and pricecentsperunit. sourcetype tells you which retailer the listing came from, so cross-retailer comparison is a single sort instead of a reconciliation job.
Note the unitcount > 0 guard. Any dataset like this will have listings where the count couldn't be resolved, and you want those excluded from a unit-price sort rather than silently ranked at zero.
I'd rather you hit these in a blog post than in production, so here are the sharp edges — including the ones in my own API.
valuescore is opinionated. It blends unit price with rating signals, so it is not the same as "cheapest." In the sample above the top-ranked-by-value item is not the lowest pricecentsperunit. If your product promises "cheapest," sort on the raw unit price yourself and ignore rank.
Cache aggressively, but treat cached prices as hints. Prices move, and the retailer's price at checkout is the one that actually applies. Never present a stored price as a guarantee.
Units are category-specific. Don't build UI copy that hardcodes "per item." Render whatever unit the category actually uses, or your detergent page will say "$0.06 per item" and mean nothing.
Spec filters belong upstream. If a user needs "at least 1000mg EPA+DHA," express that in the query rather than post-filtering on the title string. Title-based filtering will drop valid products and keep invalid ones.
If you're building anything that ranks products — a deals site, a budgeting tool, an internal procurement dashboard — the interesting engineering isn't collecting prices. It's deciding what the denominator is, and making sure the things you're dividing are genuinely substitutable. Get that wrong and you ship a sorted list that confidently recommends the worst option.
If you want to eyeball the output before writing any code, the search side of the same engine is at popgot.com — useful for sanity-checking your own unit math against ours, and for finding the cases where we get it wrong.
How are you handling this? I'm especially curious whether anyone has found a clean way to normalize multi-pack listings (2 x 24) without a pile of regex.