The Hidden Hop Behind Every Cache Miss · Morning Gist
Wednesday, November 41 story · 4 min
The Hidden Hop Behind Every Cache Miss
A vendor's case for local NVMe over network-attached disks, with real mechanism and numbers, plus a note on what simulation testing trades against formal methods.
Picked from 72 new articles across 34 sources · arrow keys for other days
Today in 30 seconds
30s left
PlanetScale argues network-attached disks add a network hop to every database cache miss; local NVMe removes it from disk reads. story 1 ↓
Its alternative keeps durability through cross-zone replication, costing one network hop per commit, and claims a $551 instance beats a $1,514 setup on IOPS. story 1 ↓
Treat it as a useful mechanism and three diagnostic signals, but remember PlanetScale sells this side of the storage decision. story 1 ↓
Network-attached disks add a hidden hop to every database cache miss
PlanetScale argues cloud databases shouldn't default to network-attached disks. Its Metal product uses local NVMe drives and synchronous cross-zone replicas for durability, removing the network hop from disk reads. Workloads that fit in RAM may not benefit.
Why it matters · Choosing database storage is a latency, cost and failure-model decision; this gives you the mechanism and three diagnostic signals, though from a vendor selling one side.
Know this first
Context from the editor, not the article · 35 sec
1 · Cache misses reach the disk
Databases keep recently used data pages in RAM. When a query needs a page that isn't there, a cache miss, the database must fetch it from storage and the query waits. Storage latency therefore matters mostly for working sets larger than memory, and for writes.
2 · Compute and storage in AWS
A typical cloud database runs on an EC2 virtual machine with an EBS volume, a network-attached disk that looks local to the operating system. Some EC2 instance types instead include NVMe drives inside the host, which are faster but tied to that machine's lifetime.
3 · Commit acknowledgement
A database tells a client a write succeeded only after some durability condition is met. With synchronous replication, that condition includes another server confirming it received the change, so the write survives losing the primary. The price is one network round trip added to each commit.
PlanetScale's query list sorted by block cache hit ratio, a way to spot queries that frequently reach storage.
PlanetScale, a hosted MySQL and Postgres company, has published a post by Simeon Griggs calling network-attached storage "a fine choice and a bad default." It promotes the company's own product, Metal, but the reasoning applies to any database in the cloud.
The hidden hop
Remember
When SSDs made disks about 100 times faster, the network hop that once barely mattered became EBS's main bottleneck.
Local disks don't remove the durability problem; they move it into the database's replication and failover layer.
High latency with low CPU suggests a database waiting on storage, so buying more compute may not help.
A typical cloud database runs on an EC2 virtual machine with an 1 (Elastic Block Store) volume. EBS looks like a local disk, but on current instances a 2, AWS's offload hardware, turns each read and write into a network request.
That design made sense on spinning disks. A random read took 10–20ms while a same-data-center round trip took 1ms or less, so the network barely mattered. When EBS began moving to SSDs in 2012, disk operations got about 100 times faster, and the network became the bottleneck. AWS built custom hardware and a protocol called 3 to cut that latency, but the hop remains.
Why not use local disks?
Some EC2 instances include 4 drives, fast SSDs inside the host, with no network hop. Griggs says they are rarely offered because the disk dies with the machine, its size is rigid, and nothing replicates its data automatically.
Metal moves durability up into the database. Each cluster has a primary and at least two replicas, each with its own NVMe drive, in separate availability zones (isolated data-center locations). A commit must be acknowledged by at least one replica before the client hears success. If the primary fails, a replica is promoted while queries are buffered by 5, a MySQL clustering layer, or 6, a Postgres connection pooler.
So the network hop moves rather than disappears: one per commit, and none for disk reads. On EBS, every disk operation crosses the network, including each cache miss.
The numbers
In PlanetScale's Postgres benchmarks, an r7i instance with a 700 GB io2 EBS volume, provisioned for 16,000 7 (input/output operations per second), cost $1,513.82 a month on demand. An i7i with 1.8 TB of local NVMe, capable of about 300,000 IOPS, cost $551.15. PlanetScale also reports CPU usage rising from 20–25% to 80% or more after migrations, which suggests those databases had been waiting on disk.
When it doesn't matter
The post concedes that if most reads fit in RAM and writes are light, EBS may be fine. The signs you need local disks are frequent cache misses, hitting IOPS limits, or low CPU with high latency.
The post gives no direct latency comparison and no cost for the cross-zone commit. It also doesn't say how failed nodes are rebuilt or how fixed disks get resized, and those are the real operational price of this design.
PlanetScale, a hosted MySQL and Postgres company, has published a post by Simeon Griggs calling network-attached storage "a fine choice and a bad default." The usual advice is to put your application close to your database. Griggs applies the same logic one level down: the database's CPU may be separated from its disk by a network too.
The post promotes PlanetScale's own product, Metal, so treat its framing with care. The mechanism is still worth understanding, and the post concedes when you don't need what it sells.
How storage moved away from compute
Early database servers, such as Sun's SPARC machines, kept disks in the chassis or an adjacent cabinet, connected by SCSI or SAS cables. A single disk is not durable, so 1 (Redundant Array of Inexpensive Disks) kept several disks in sync, letting a failed one be swapped while the others carried on. In the 1990s and early 2000s, enterprises moved storage into dedicated arrays reached over a 2 (Storage Area Network), which was effective and very expensive.
AWS's 3 (Elastic Block Store), launched in 2008, offered the same idea on demand: a volume that mounts like a disk, is replicated across multiple disks within one availability zone (an isolated data-center location in a cloud region), and can be resized in place.
The separation made sense at the time. EBS began on spinning disks, where a random read took 10–20ms and a same-data-center network round trip took 1ms or less. The disk dominated, so the network barely registered.
Then in 2012 EBS began moving to SSDs, and disk operations got about 100 times faster. On those figures, a read falls to very roughly 0.1–0.2ms (my arithmetic, not the article's), so the network became the bottleneck. The post quotes AWS's Marc Olson saying the system "didn't improve nearly as much as we had hoped." AWS responded with custom hardware and a network protocol called 4, which reduced network latency but did not remove the hop. Griggs's summary is that storage and compute were separated when storage was slower than the network, then penalized once the reverse became true.
Where the hop hides today
On current EC2 instances, an EBS volume appears to the operating system as an attached device, but a dedicated 5, AWS's offload hardware, turns each read and write into a network request. Some instance types also include local 6 drives (fast SSDs on the host's own bus) with no network hop at all.
Griggs says local storage is rarely offered because the risk is high without careful management. The disk lives and dies with the machine, its size is rigid, and nothing replicates its data automatically.
Getting durability back with replicas
Metal moves durability from the storage layer to the database layer:
Each cluster has a primary and at least two replicas, each on its own virtual machine with its own NVMe drive, spread across distinct availability zones.
A commit must be acknowledged by at least one replica before the primary confirms it to the client.
Every replica is a candidate primary. If the primary fails, a replica is promoted immediately.
During the switch, queries are buffered: by 7, a sharding and clustering layer for MySQL, or by 8, a Postgres connection pooler, which then directs them to the new primary.
So Metal keeps a network hop, but moves it. Each commit crosses zones once, and disk reads never leave the machine. Griggs contrasts this with EBS, where every disk operation crosses the network, including a read that misses the cache. He notes that reads are probably the majority of most workloads.
An illustration with invented numbers: suppose a query misses the cache on 20 pages and reads them one at a time. At 0.5ms per network-storage read, it spends 10ms waiting on I/O. At 0.1ms per local read, it spends 2ms. If all 20 pages are in RAM, both setups cost about the same, which is why the working-set size decides this question.
The numbers
The post cites PlanetScale's Postgres 17 vs 18 benchmarks, using on-demand monthly prices:
IOPS means input/output operations per second. By this comparison, the local setup costs about 2.7 times less, with roughly 2.6 times the storage and almost 19 times the IOPS. EBS defaults to 3,000 IOPS, a cap Griggs explains as protection against a 10, one tenant swamping shared capacity.
PlanetScale also reports seeing CPU usage rise from 20–25% to 80% or more after customers moved to Metal. That is the database doing work it previously spent waiting on the disk, which suggests the old compute was partly paid for and unusable. The post gives no accompanying latency or throughput figures for those cases.
Metal now starts at $50 a month (since December 2025), with VMs as small as 1/8 vCPU, 1 GB of memory and 10 GB of NVMe.
When the network is fine
The post cites PlanetScale's own documentation: you may not need Metal if most reads fit in RAM and write volume is low, because reads then rarely touch storage. It offers three signs the other way:
Frequent cache misses, because hot data is too large or changes too often for memory.
Hitting EBS IOPS limits, where local disks may be both faster and cheaper.
Low CPU with high latency, which suggests the database is throttled by disk reads.
What the post leaves out
The post contains no direct latency comparison between EBS and local NVMe, and no measurement of what the cross-zone commit adds. It also doesn't state the vCPU or memory of the two benchmark instances, so the price table can't be confirmed as like for like, and it prices single instances rather than full three-node clusters.
The operational hard parts get only a sentence: "we've done the hard work." It doesn't explain how a failed node's data is rebuilt, how long that takes, how storage is resized when a disk is fixed to its host, or what happens to writes when no replica can acknowledge. Those are the costs you take on when durability moves out of the storage layer, and anyone running local disks themselves would have to answer them.
