MongoDB in 8 Slides

Post 3 opens the hood. The previous post made claims — fast reads, durability, scale — and an engineer's instinct should be to ask how those claims are actually delivered. This post answers that with the four real components: the storage engine, indexes, replica sets, and the oplog that ties replication together.

The goal is to demystify. MongoDB can feel like a magic JSON bucket where you throw data in and queries come out, but underneath it's recognizable database engineering — B-trees, write-ahead logging, leader election. Seeing the machinery is what lets you debug it when it misbehaves.

WiredTiger is the default storage engine and it explains a lot of Mongo's runtime behavior. Two features matter most. First, compression: documents are compressed on disk, which trades a little CPU for substantially less storage and I/O. Second, document-level concurrency control — two writers modifying different documents don't block each other, which is a big improvement over the collection-level locking of Mongo's early days.

The write-ahead journal is the durability backbone. Before a change is applied to the main data files, it's recorded in the journal. If the server crashes mid-write, on restart the engine replays the journal to recover any committed changes. This is the same write-ahead-logging idea that relational databases use, and it's why a properly configured Mongo doesn't lose acknowledged writes to a crash.

Indexing is the single most important performance concept in any database, and Mongo is no exception. Without an index, a query that filters on a field must examine every document in the collection — a collection scan, O(n) in the number of documents. That's invisible on a tiny dev dataset and catastrophic on a production collection with millions of records.

An index is a separate, sorted B-tree structure keyed on one or more fields. To answer a filter, Mongo walks the tree to jump straight to matching documents, turning a linear scan into a logarithmic seek. Every collection automatically indexes _id, but you are responsible for indexing the other fields you filter and sort on. This slide sets up the explain() demo two slides later and the COLLSCAN mistake in post 5.

The pipeline diagram traces a query's life: parse and validate the query, plan by choosing the best available index, fetch by either seeking through an index or scanning the collection, and return the documents to the client. It mirrors how a relational query planner works, which reinforces that Mongo is doing recognizable database work rather than something exotic.

The 'Plan' stage is the interesting one. Mongo's query planner evaluates candidate plans — which index to use, or whether to scan — and caches the winner for similar queries. When a query is slow, this is the stage you interrogate with explain(), because it tells you whether the planner found a useful index or fell back to scanning everything.

Replica sets are how MongoDB delivers durability and high availability. A replica set is a group of nodes that all hold the same data. One node is the primary and is the only one that accepts writes; the others are secondaries that copy the primary's data. Reads can be served by the primary or, if you allow it, by secondaries.

The magic is automatic failover. The nodes constantly heartbeat each other. If the primary stops responding, the remaining nodes hold an election and promote a secondary to primary, usually within a few seconds. Your driver detects the change and reconnects to the new primary. From the application's perspective there's a brief blip rather than an outage, and no committed data is lost — which is exactly the durability the previous post promised.

The flow diagram makes the replica-set topology concrete: the application writes to the primary, which is the source of truth, and two secondaries replicate from it. One of those secondaries is the one most likely to be promoted if the primary fails.

This shape is why a production MongoDB deployment is almost never a single server. A standalone node has no failover and no redundancy — if it dies, you're down and possibly losing data. A three-node replica set is the standard minimum because it gives you a clear majority for elections (two out of three) and a spare copy of the data. The diagram is the mental picture to keep when reasoning about availability.

The oplog — operations log — is the mechanism that actually keeps secondaries in sync, and it's worth understanding because it shows up in backups, monitoring, and change streams. Every write the primary commits is recorded as an idempotent entry in a special capped collection called the oplog. 'Capped' means it's a fixed-size ring buffer: old entries are overwritten as new ones arrive.

Secondaries continuously tail the oplog and replay its operations in order, which is how they converge on the same state as the primary. Because the operations are idempotent, replaying them is safe. This same oplog powers change streams — the feature that lets your application subscribe to a live feed of changes — and many backup and migration tools that watch it to capture every modification.

This code slide turns the indexing theory into a habit you can actually practice. createIndex builds a B-tree on the email field. Then explain('executionStats') asks Mongo to describe how it would run the query and how much work it did.

The single most useful thing to look for in that output is the winning plan's stage. IXSCAN means the query used an index — good. COLLSCAN means it scanned the whole collection — usually bad on a large collection and a signal you're missing an index. This one diagnostic loop — run explain, read the stage, add an index if needed — is the core performance-tuning skill for MongoDB, and it directly sets up the first and most common mistake in post 5.

Read and write concerns are the dials that let you tune the consistency-versus-speed trade-off per operation, and they're how MongoDB lets you choose where you sit on that spectrum rather than forcing one answer. Write concern controls how many nodes must acknowledge a write before the operation returns. w:1 means just the primary confirmed — fast, but a primary failure right after could lose the write. w:'majority' means a majority of the replica set confirmed — slower, but the write survives a failover.

Read concern is the mirror image: it controls how fresh and how durable the data you read must be — for instance, only reading data that has been committed to a majority of nodes. Together these let a single application use fast, relaxed settings for low-stakes data and strict, safe settings for data it cannot afford to lose. The fire-and-forget mistake in post 5 is precisely what happens when someone ignores write concern.

The recap compresses the engine into five lines: WiredTiger gives compression and document-level locking, indexes turn O(n) scans into B-tree seeks, a replica set is a primary plus secondaries with automatic failover, the oplog lets secondaries replay the primary's writes, and read/write concerns dial consistency against speed.

These five facts are the working model of how MongoDB behaves at runtime. They explain why an unindexed query is slow, why a single-node setup is risky, and why a sloppy write concern can lose data — all of which become the concrete failure modes in the final post.

This cover closes the mechanics post and points to the hands-on tour. Theory about indexes, replica sets, and the engine is necessary, but you cement it by running queries.

The teaser frames post 4 as a code-heavy walkthrough — CRUD operations, real query operators, and the aggregation pipeline. After understanding how the engine works, the next step is driving it directly and seeing the behavior firsthand.