Two Logs, No Peeking

How MongoDB's Atlas Infinite separates compute from storage

MongoDB's Atlas Infinite runs compute and storage as separate tiers. It keeps WiredTiger's batched checkpoints by writing two logs, a logical one and a physical one. Shared storage only ever holds encrypted bytes it cannot read.

Why it matters · It shows that the Aurora-style "log is the database" design has an alternative when the storage engine batches its writes and storage must not be able to read tenant data.

MongoDB released Atlas Infinite in public preview on September 29, alongside MongoDB 9.0. It separates compute from storage in MongoDB's Atlas cloud service. Murat Demirbas, writing from inside MongoDB, explains the interesting part: where it breaks from the standard disaggregated design.

Why now

MongoDB had , which splits data across machines, for more than a decade, so it never hit single-node limits. The pressure now is "storage gravity." In Atlas's setup, each node of a three-node keeps a full copy of the data on its own disk. Growing storage, adding a replica or restoring a backup means copying everything, which can take hours. Compute and storage are also sold together, so needing more of one means paying for both. Customers also want cheap snapshots and clones.

The standard design, and why it didn't fit

The usual approach, used by Amazon Aurora, sends a physical of every page change to a replicated log service. Page servers replay that log to rebuild any page at any point in time. Demirbas lists three problems for MongoDB:

Batching. , MongoDB's storage engine, lets a page absorb many changes in memory and writes it once per checkpoint. Logging every page change would undo that.
No canonical page. WiredTiger is lock-free and evicts pages speculatively, so two nodes with the same data hold different bytes.
Security. Storage that rebuilds pages has to be able to read them. MongoDB wanted shared, multi-tenant storage that never sees plaintext.

So MongoDB made storage "blind" rather than smart.

How it works

Compute nodes are a primary, a and optional read replicas. They hold only caches and recent changes. Storage is written in Rust and has three parts: a Log Service that only does consensus, a Page Service that spreads pages across servers in every zone, and object storage for durability.

Two logs. A write commits once the , MongoDB's logical log of operations, is on a majority of Log Service replicas in three zones. At each checkpoint, WiredTiger separately emits a "" of physical page deltas, which page servers organize by page. Only the oplog is on the commit path, and pages are written once per checkpoint.

Client-side encryption. Data is encrypted on the compute node before it leaves, using keys the customer can revoke. Storage can't see documents, collection names, indexes or tenant boundaries.

Fresh replicas. Pages only advance at checkpoints, so a replica could lag by seconds. To avoid that, replicas apply the incoming oplog to a local ingest table and merge it with the pages at read time. MongoDB reports millisecond lag.

Caveats

This is an overview with no benchmarks. The read-path cost of the merge and the detailed mechanics are left for upcoming parts 2 and 3 of the engineering series.

MongoDB released Atlas Infinite in public preview on September 29, together with MongoDB 9.0. It is a version of MongoDB's Atlas cloud service in which compute and storage are separate tiers. Plenty of databases already do this. Murat Demirbas's summary, which draws on an engineering post by project lead Abhishek Chauhan, is about where MongoDB chose not to follow the usual design, and why. The writing makes clear that Demirbas works at MongoDB ("bought us time").

Why MongoDB waited

Demirbas argues that MongoDB didn't need disaggregation to scale. It shipped , which splits a dataset across many nodes and includes automatic chunk balancing and routing, more than a decade ago. So it never hit the single-machine limit that pushed other databases to separate storage early.

The pressure now comes from what he calls storage gravity. Atlas uses a design: each of the three nodes in a has its own disk holding a full copy of the data. Growing storage, adding a replica or restoring a backup all mean copying the whole dataset, and on large clusters that takes hours. Compute and storage are also sold as a bundle, so a customer who needs more of one pays for more of both. Customers also want cheap snapshots, clones and branches of production data.

The standard design, and the three ways it conflicts with MongoDB

In the usual disaggregated database, compute is stateless. It sends a physical , a record of every change to every page, to a log service that a quorum of replicas agrees on. Page servers read that log and replay it, so they can produce any page as of any log position. Object storage sits underneath as the durable copy.

According to the article, three things make this a poor fit for MongoDB:

batches its writes. WiredTiger, MongoDB's storage engine, uses checkpoint-based consistency. Writes pile up in memory, a page can take many changes, and it is written once at the next checkpoint. That spreads the cost of turning writes into readable on-disk state across many changes. Logging every page modification so storage could rebuild pages would, Demirbas says, "cripple its efficiency."
No canonical page image exists. WiredTiger is lock-free and parallel, and it evicts pages from memory speculatively. Two nodes holding the same logical data hold different bytes. "Replicate the pages" therefore has no single correct target.

MongoDB owns the engine, the server and the cloud service, so it could design changes across all three. Where the standard design makes storage smart, MongoDB made it blind.

The architecture

Compute consists of a primary, a (a warm replacement ready to take over) and optional read replicas. These nodes hold only caches and recent changes, and all of it can be rebuilt from storage. The shared storage layer is written in Rust and has three parts:

A Log Service that does one job: reaching consensus on the log. Keeping it narrow keeps it simple and predictable.
A Page Service that spreads each customer's pages across many servers in every availability zone. Page materializers feed it from the log.
Object storage for long-term durability, organized by an Object Index Service.

Difference 1: two logs instead of one

A write is durable once its entry is on a majority of Log Service replicas across three zones. The oplog is MongoDB's logical operation log, with entries like "update document X." Separately, at each checkpoint, WiredTiger emits a : physical page changes, mostly deltas. The phylog travels through the Log Service, and page servers organize it by page.

Durability and page materialization are therefore decoupled. Only the oplog sits on the commit path. Pages are written once per checkpoint rather than once per change, so WiredTiger keeps its batching. An illustrative example (not from the article): if a hot page receives 500 updates between checkpoints, a redo-log design ships and replays 500 page records. Here the commit path carries 500 small logical oplog entries, and storage receives one delta for that page at the checkpoint.

Difference 2: storage never sees plaintext

Data is encrypted on the compute node, inside the customer's network, before any byte goes to storage. It is decrypted only there, and the customer can hold and revoke the keys. Storage cannot see documents, collection names, indexes or even tenant boundaries. Demirbas's claim is that root access to the entire storage fleet would give an attacker nothing but opaque bytes. In the traditional design, he says, security is partly "trust us" and partly "best intentions". This only works because storage never needs to interpret a page.

Difference 3: fresh replicas without page-level logging

There is a cost to this. Shared pages only advance at checkpoints, so a replica that read only from page servers could fall seconds behind. The fix is on the compute side. A standby or read replica also applies the oplog to a local ingest table as entries arrive from the Log Service. Reads merge the checkpoint-consistent pages from storage with the recent changes in the ingest table. MongoDB reports millisecond replica lag this way while keeping checkpoint batching.

Availability

Every cluster has a hot standby, so losing the primary means a quick handoff rather than starting a new node from scratch. Data exists in three forms at once: the replicated log, the page servers and object storage. The fleet is split into isolated cells so a failure in one cell can't spread to the next. Recovery runs in parallel across many machines instead of falling on one.

What's left open

This is a high-level overview, and parts 2 and 3 of Chauhan's series are still to come. The article gives no throughput or latency comparisons with the shared-nothing Atlas, and no measured numbers beyond "millisecond replica lag." It leaves several questions unanswered:

How reads merge the ingest table with pages, and what that costs on the read path.
How checkpoint frequency trades phylog volume against how much state lives on compute.
How encrypted page storage interacts with snapshots and branching.
How a new replica bootstraps its ingest table after a failover.