The first version of my image-to-Minecraft-blocks converter locked up the browser tab for long enough that I assumed something had crashed. I dropped in a photo straight off my phone, hit convert, and the page stopped responding to clicks — no spinner animation, no scroll, nothing. I sat there for a while wondering whether I'd written an infinite loop.
I hadn't. It was just doing exactly what I told it to, on the main thread, one pixel at a time.
This post is about what I got wrong when I tried to fix it, because my first guess was confidently wrong and the actual bottleneck turned out to be somewhere I wasn't looking.
The job is simple to state. You have a photo. You have a fixed list of Minecraft blocks, each with a known average color. For every cell in the output grid you find the block whose color is closest, and you write it down.
The obvious implementation is two nested loops: outer over pixels, inner over the palette, keep the minimum. Textbook nearest-neighbor search, and for small inputs it's genuinely fine.
Then you run the numbers on real inputs. A photo out of a modern phone camera is around 4000×3000 — twelve million pixels before you touch anything. You downscale before matching, sure, since the output grid is maybe 128 or 256 blocks wide. But the decode and resample still happen at full resolution, and then the match loop runs every cell of the output grid against every entry in a palette of several hundred blocks.
My first theory was the color space. Matching happens in OKLab rather than raw RGB, because RGB distance produces the classic "cheap filter" look — most visibly on skin tones and mid-range greens, where a numerically small gap can read as a very different color to a human eye. OKLab conversion involves a matrix multiply and cube roots, and cube roots felt expensive, so I assumed that was where the time was going.
It wasn't. When I actually profiled it instead of guessing, the color conversion barely registered. Nearly everything was sitting in the palette scan — the inner loop — and in the plain fact that all of this was happening on the thread responsible for painting the page.
Two separate problems wearing one costume: The work was on the main thread, so the UI froze regardless of how fast the work was. The work was genuinely redundant, because photos repeat colors constantly and I was re-deriving the same answer thousands of times.
The first fix is the boring correct one: put the pipeline in a Web Worker. The UI thread hands over the pixels, the worker grinds, the UI stays alive and can render progress.
What nobody warns you about is that the handoff itself has a cost. postMessage structured-clones its payload by default, and structured-cloning a multi-megabyte pixel buffer means copying a multi-megabyte pixel buffer, twice per conversion — once in, once out. You can move the buffer instead of copying it by passing it in the transfer list:
That hands ownership across the boundary with no copy. It also neuters the original — after the transfer, buf.byteLength on the sending side is 0.
Which is how I spent an evening on a bug where the source thumbnail rendered as a black rectangle every time, but only after the first conversion. I was holding a reference to the same buffer for the preview, and the transfer had emptied it out from under me. The fix is unglamorous: keep a copy for anything the main thread still needs, or don't transfer that particular buffer. But it's the kind of bug that reads as "canvas is broken" for an hour before it reads as "you gave your array away."
The second fix is where the actual speedup lived, and it comes from a property of photographs rather than a property of JavaScript.
Photos are enormously repetitive at the color level. A sky is thousands of pixels of nearly the same blue. A cheek is hundreds of pixels of nearly the same tone. If you're doing a full palette scan for every one of those pixels independently, you are computing the identical answer over and over.
So quantize the lookup key. Take the source color down to 5 bits per channel and use that as a cache key. Five bits per channel means 32×32×32 = 32,768 possible keys — small enough to keep in a flat typed array, and a real photo touches only a fraction of them. First time you see a bucket, do the full palette scan. Every time after, it's an array index.
The palette scan doesn't get faster. It just stops running for the overwhelming majority of pixels. That's the whole trick, and it's the difference between a frozen tab and a progress bar that finishes while you're still looking at it.
One consequence worth flagging if you build something similar: the cache key has to include the palette identity, not just the color. My block palette is versioned — Java releases from 1.13 up through 26.2, with Bedrock carrying its own separate block and texture set — because a palette frozen at whatever version the tool was written against will eventually disagree with what the game actually renders, and you get a preview that doesn't match your build. That's a correctness win, but it means "nearest block to this color" is not a stable function. Switch edition or version and every cached answer is potentially wrong. Ask me how I know. Version identity goes in the cache key, or the cache gets cleared on switch; either works, silently doing neither does not.