Skip to content

MongoDB Internal Architecture: mongod and WiredTiger

A MongoDB Community mongod with default settings, as a replica-set member, on versions 7.0, 8.0 and 9.0.

Detailed diagram of a mongod with WiredTiger: the network and query layers, concurrency control, the WiredTiger cache and its threads, background threads, replication, the operating-system page cache and the files in the data directory, with the write path numbered 1 to 8.
Detailed diagram: components, threads and default values. Scroll sideways or open the full-size image.

Open full size (PNG, 3600 px)

Open the simplified version (PNG, 2000 px)

In-memory structures

WiredTiger cache

Holds the uncompressed B-tree pages of collections and indexes, their update chains and the history-store pages. The default size is max(50% × (RAM − 1 GB), 256 MB), where RAM is the smaller of physical memory and the cgroup limit. Tune it with storage.wiredTiger.engineConfig.cacheSizeGB. From 8.0 there is also cacheSizePct, a fraction from 0 to 0.8 of RAM that cannot be combined with cacheSizeGB.

Collection pages grow to 10 MB in memory before they split. Index pages are 16 KB on disk and keep their keys prefix-compressed.

Official documentation: MongoDB: WiredTiger Storage Engine, Memory Use · MongoDB: storage.wiredTiger.engineConfig.cacheSizeGB · WiredTiger: Cache

Eviction

There are 4 threads by default. Whichever one wins the pass lock walks the trees and queues pages; the others evict. A clean page is dropped. A dirty page is reconciled and written to its data file, and its older versions go to the history store.

The default thresholds are 80% used as the target and 95% as the trigger, dirty at 5% and 20%, updates at 2.5% and 10%. At the target the eviction threads start working. At the trigger, application threads are drafted to help, which shows up as latency. In 8.0 the thread count can be changed at runtime, from 1 to 64 (wiredTigerEvictionThreadsMin and wiredTigerEvictionThreadsMax).

Official documentation: WiredTiger: Eviction · WiredTiger: Cache and eviction tuning

Update chains (MVCC)

A write does not change the record in place: it prepends a new version to the record's update list on the in-memory page. A reader walks the list to the version its snapshot can see.

Isolation is snapshot with commit timestamps. The oldest timestamp trails the stable timestamp, the majority-committed point, by 300 s (minSnapshotHistoryWindowInSeconds). Older versions move out to the history store.

Official documentation: WiredTiger: Transactions · WiredTiger: Timestamps · MongoDB: minSnapshotHistoryWindowInSeconds

Log slots (journal buffer)

Each commit copies its journal records into shared log slots. Each record is compressed with snappy as it is appended, except small ones. In a replica set only the local.* tables are journaled, so these records are, in practice, oplog entries.

Official documentation: WiredTiger: Logging · MongoDB: Journaling

Spill engine (9.0)

A second, small WiredTiger instance for SBE hash aggregation and hash lookup, window functions and $graphLookup that exceed their memory limit. It has its own cache, outside cacheSizeGB: 2.5% of RAM, between 100 MB and 1 GB, with one eviction thread and zstd compression. It has no journal and is recreated at every startup.

In 7.0 and 8.0, that SBE spill goes to temporary internal-*.wt tables inside the dbPath.

Official documentation: MongoDB: spillPath

Operating-system page cache

WiredTiger uses buffered I/O (there is no direct_io in any of the three versions), so the filesystem cache holds blocks as they are on disk: compressed for collections and the journal, and without block compression, with prefix-compressed keys, for indexes.

Official documentation: MongoDB: WiredTiger Storage Engine, Memory Use

On-disk structures

Data files

Each collection and each index is one file, a WiredTiger table. Collections use snappy block compression and checksums (storage.wiredTiger.collectionConfig.blockCompressor). Indexes use prefix compression, 16 KB pages and checksums, with no block compression by default.

In 7.0 and 8.0 a collection's file name has a number and a random suffix (collection-<n>-<random>.wt). In 9.0, tables created since 8.2 are named collection-<UUID>.wt and index-<UUID>.wt; an upgraded dbPath keeps the old names.

Official documentation: WiredTiger: Data File Format · WiredTiger: B-Trees · MongoDB: WiredTiger Storage Engine, Compression

Oplog

local.oplog.rs is an ordinary WiredTiger table, capped by size: 5% of the free space on the dbPath filesystem when it is created, between 990 MB and 50 GB (replication.oplogSizeMB and storage.oplogMinRetentionHours). It is truncated oldest-first, in sections (truncate markers). Recent entries are usually hot in the cache, for secondaries and change streams.

Official documentation: MongoDB: Replica Set Oplog

History store

WiredTigerHS.wt holds the older versions of records that eviction moved out of the cache. They are needed for snapshot reads and for timestamps.

Official documentation: WiredTiger: History Store

Journal

The WiredTiger write-ahead log lives in journal/, in 100 MB WiredTigerLog.NNNNNNNNNN files, with the next one already preallocated (WiredTigerPreplog.*). In a replica set only the local.* tables are logged (the oplog, not user collections). A standalone logs every table. Files older than the last checkpoint are removed.

The JournalFlusher thread flushes every 100 ms (1 to 500, changeable at runtime, storage.journal.commitIntervalMs), or at once when a j:true or majority write asks. A w:1 write without j can lose up to one journal interval of oplog in a crash and can be rolled back on failover.

Official documentation: MongoDB: Journaling · WiredTiger: Logging

Checkpoints and recovery

The Checkpointer thread takes a database-wide checkpoint at the stable timestamp, sleeping syncdelay (60 s, storage.syncPeriodSecs) between checkpoints. Every dirty table is written (data, indexes, history store, _mdb_catalog.wt and sizeStorer.wt), then WiredTiger.wt and WiredTiger.turtle. Journal that is no longer needed is removed.

Checkpoints are copy-on-write: WiredTiger never overwrites a block that the live checkpoint references, blocks are checksummed, and a checkpoint is published by rewriting WiredTiger.wt and atomically replacing WiredTiger.turtle. On restart WiredTiger opens the last checkpoint, replays the journal (local.* in a replica set) and rolls back to the stable timestamp. Then mongod replays the oplog from that point.

Official documentation: MongoDB: WiredTiger Storage Engine, Snapshots and Checkpoints · WiredTiger: Checkpoint

Metadata and catalog

WiredTiger.wt is the metadata table, with the files and their checkpoints. WiredTiger.turtle points to the checkpoint of that table itself, WiredTiger holds the version and WiredTiger.lock prevents a second process.

On the MongoDB side, _mdb_catalog.wt maps collections and indexes to their files. sizeStorer.wt keeps document counts and data sizes, flushed by whichever thread returns a WiredTiger session, every 100,000 releases or 60 s, with no dedicated thread. storage.bson records the engine and mongod.lock holds the process id.

Official documentation: WiredTiger: Metadata

Temporary files

Sorter spill files go in _tmp/, for $sort, classic $group, $bucketAuto and index builds. The directory only appears when something spills. In 9.0 it exists from the start because of _tmp/spilldb/, which is deleted and recreated at every startup.

Official documentation: MongoDB: Aggregation Pipeline Limits, Memory Restrictions

diagnostic.data (FTDC)

FTDC metric archives, in 10 MB files. The directory has a size cap: 200 MB in 7.0, 250 MB in 8.0 (500 MB with --shardsvr) and 500 MB in 9.0 (diagnosticDataCollectionDirectorySizeMB).

Official documentation: MongoDB: diagnosticDataCollectionDirectorySizeMB

Summary