I was recently refactoring some legacy PHP code and noticed that the same data array was being used in completely different ways! Radically different! After some analysis, I came to the conclusion that a PHP array can be a very efficient data structure as long as it remains a packed list, but as soon as it turns into a hash table, memory consumption skyrockets, and for large datasets, associative arrays often consume nearly twice as much memory as regular typed objects.
In this article, I want to explain why this happens and what conclusions I've reached, because as I was told in the PHP community, only God knows where the memory goes 😀
After doing some analysis, I found that a PHP array holding one million integers takes 16.8 MB on PHP 8.4. If you add a single-character key, it takes up 41.9 MB. That means a single key takes up 25 MB. 🤷♂️
I'd read that PHP arrays are hash tables so many times I'd stopped hearing it, it sits in the same drawer as "floats are inexact". So I ended up building arrays of a million elements and watching memorygetusage() while I did things to them. I started with the case PHP optimizes, then the ways that optimization quietly goes away, then a million rows from a query with each row an associative array, because every codebase I've worked on holds its rows that way. The rows are where the real cost is and the fix turns out to be smaller than the problem.
Everything below ran on PHP 8.4.21 in the official php:8.4-cli Docker image on 64-bit Linux with memorylimit set to -1 so nothing died halfway and where the version matters I ran the same script on 8.1.34 too, because 8.2 changed one of these numbers a lot and I wanted the before as well as the after. The figures are deltas of memorygetusage(). The PHP manual says those are rounded up to the allocator's granularity, so treat the last couple of digits as noise.
Here's the sample code I used; this code for measuring memory usage isn't complicated at all. You can try replicating it too! The measuring code is nothing clever. Build the array, subtract two calls to memorygetusage(), divide by the count.
The two loops use the same keys and the same values. One counts up and the other counts down.
Two and a half times the memory for the same million integers, because I filled it backwards.
The first array is what the engine calls packed. The keys are 0 and 1 and 2 and so on in order, so PHP doesn't store them at all, each slot is one 16-byte zval and that's it. The reverse-filled array has the same keys but they arrived out of order, so it's a real hash table with keys and hashes and an index at about 40 bytes per slot.
Then the part that made me sit up. Take the packed million and give it one string key.
One key. Twenty-five megabytes. And as it turns out, removing the key does not undo this action, because nothing on the unset path converts a hash table back into a packed one (sort() does, as it happens, and arrayvalues() builds a fresh packed array, but plain unset only frees the value and leaves the layout alone). A negative key does the same thing as a string key, and I'm not sure why that surprised me since as an unsigned value it's enormous, but it did.
Every PHP array is a zendarray and the C code calls it HashTable too. It's 56 bytes on 64-bit. Most of it is bookkeeping (a refcount header and flags and the table size and the element count and the next free integer key and a destructor pointer) and then there's one pointer to the data, and that pointer is a union, and the union is the whole story:
In hash mode the data block has two parts. In front sits the hash index, an array of uint32t slots with twice as many slots as there are buckets (the mask is -(nTableSize + nTableSize)) so that's 8 bytes of index per bucket and behind it sit the buckets at 32 bytes each:
A lookup hashes the key and masks it into the index and reads a bucket number there and follows a next chain stored inside the zval if two keys collided. Iteration never touches the index, it walks the buckets front to back, and since buckets are appended in insertion order that's how foreach gives you insertion order for free.
So a hash slot costs 32 + 8 = 40 bytes. A million elements need a table of 1,048,576 slots (more on that in a second), and 40 times that is 80 bytes short of what the measurement says. That's the 56-byte struct plus 24 bytes the allocator keeps to track a block that big.
In packed mode there's no index (two placeholder slots and 8 bytes total) and the data is zval arPacked, a bare C array of 16-byte values where the position is the key. Sixteen times the same table size lands about 4 KB under the measured 16.8 MB. It's the same accounting with a smaller slot.
Here's the thing I hadn't tracked: packed arrays only became this cheap in PHP 8.2. Before that they used the same 32-byte buckets as hash arrays and simply skipped the index, with h repeating the slot's position and key always NULL in every slot. That lasted until Dmitry Stogov's PR #7491 ("Use more compact representation for packed arrays") was merged in November 2021 and shipped in 8.2.0 in December 2022. It's an internals change, so the 8.2 UPGRADING notes don't mention it at all (I went looking). Here's the 8.1 run for the before:
Half the slot memory of every list in your application, from one release. Tideways company measured the same thing when 8.2 came out, on a 100,000-element list, and got 4.3 MB down to 2.3 MB. If you're still on 8.1 for some reason, this is a decent argument on its own.