Memory Optimization Still Matters at Cloudflare Scale
Cloudflare published a deep dive on saving 100 terabytes of RAM by optimizing 1.1.1.1's DNS cache. The core insight was embarrassingly simple in retrospect: a Vec field tracking capacity was allocated for every cache entry but never used after writes, costing 8 bytes per entry across a massive cache. Eliminating it, plus rethinking how record data is laid out relative to the CacheEntry struct, produced savings that would cost millions annually in DRAM if unaddressed.
The HN discussion quickly noted that this is a systems programming argument made concrete. The author of MaraDNS showed up to describe similar aggressive memory pooling techniques using a single large malloc traversal. The implication is that most modern services running at scale are leaving significant memory savings on the table by defaulting to convenient abstractions.
This thread is a counterweight to the 'just scale horizontally' culture. At 1.1.1.1 query volumes, there is no horizontal scaling that makes 100TB of unnecessary RAM cheap.
So what?
If you are running any stateful service at meaningful scale with high object counts per server, audit your in-memory data structures the way Cloudflare did. The pattern of 'we write it once and never modify it, so why are we storing capacity?' applies to caches, connection pools, and lookup tables across most production stacks. A few hours of profiling could reveal savings that dwarf any infrastructure discount negotiation.