Redis: Caching & Beyond

This cover poses the question that exposes how Redis really works: if everything lives in RAM, what happens when the server restarts or the power fails? RAM is volatile — it's wiped on reboot. A naive in-memory store would lose all its data. The fact that Redis can survive a restart tells you there's more going on under the surface.

Post three is the architecture post. It covers the single-threaded event loop that defines Redis's execution model, the two persistence mechanisms that let it recover from disk, the expiration system behind TTLs, the eviction policies that decide what to drop when memory fills, and the replication that keeps it available. Understanding these turns Redis from a black box into a system you can reason about and operate.

This slide explains the heart of Redis's execution model: a single-threaded event loop. The loop continuously accepts new client connections, reads an incoming command, executes it to completion, and writes the reply, then repeats. Crucially, one command fully finishes before the next begins. There's no preemption mid-command and no concurrent execution of two commands.

That design is why every Redis command is atomic with no locking required — there's literally no way for two commands to interleave and corrupt shared state. It also explains a key operational rule that returns in later slides: a single slow command blocks every other client, because they're all waiting their turn on the same thread. Fast, in-memory commands keep the loop spinning; one expensive command stalls everyone.

This cycle diagram animates the event loop as the four repeating phases it actually runs: accept a connection, read and parse a command, execute it against the in-memory data, and send the reply back. Then the loop returns to the top and handles the next ready event.

Seeing it as a tight cycle reinforces why throughput is so high and latency so low: each phase is cheap and there's no waiting on disk in the common path. It also makes the single-threaded constraint tangible. Anything that makes the 'execute' step slow — a command over a huge key, a blocking operation — extends the time before the loop can serve the next client, which is felt as latency by everyone connected.

This slide covers persistence, the answer to the cover's question. Redis offers two complementary mechanisms. RDB takes point-in-time snapshots of the entire dataset on a schedule (or on demand), producing a compact binary file that's fast to load on restart. AOF, the append-only file, logs every write command as it happens, so on restart Redis replays the log to reconstruct the exact state.

They trade off differently. RDB is compact and restarts fast, but you can lose any writes made since the last snapshot. AOF is more durable — with per-second fsync you lose at most about a second of writes — but the file is larger and replaying it is slower. Many production setups enable both: AOF for durability and RDB for quick restarts and backups. The right choice depends on how much recent data loss you can tolerate.

This comparison puts RDB and AOF side by side so the trade-off is clear. RDB forks the process to write a point-in-time snapshot — a compact binary file that reloads quickly, at the cost of potentially losing writes made after the snapshot. AOF appends every write command to a log that's replayed on restart — more durable, but a larger file that's slower to load and grows until it's rewritten.

The decision hinges on your durability requirements. If Redis is a pure cache whose data can be rebuilt, you might run RDB alone or even disable persistence for maximum speed. If it holds data you'd rather not lose, AOF with everysec fsync is the safer default, often paired with RDB for fast restarts. Knowing both mechanisms lets you make that choice deliberately rather than accepting whatever the default happens to be.

This slide explains expiration, the feature that makes Redis ideal for caches, sessions, and locks. Any key can be given a TTL — a time to live — after which Redis considers it gone. Redis enforces expiry in two ways working together. Lazy expiration: when you access a key, Redis checks whether it has expired and removes it if so. Active expiration: a background process periodically samples a batch of random keys with TTLs and removes the expired ones.

The combination is a pragmatic balance. Lazy expiration costs nothing until you touch a key, but a key that's never accessed again would otherwise linger forever — so active sampling cleans those up over time. The practical upshot is that you can rely on TTLs to bound how long stale data survives, which is exactly why they underpin caching and session management.

This snippet shows TTLs in practice. SET with EX 3600 stores a value that expires in one hour — the typical pattern for a cached value or a session. TTL queries how many seconds remain on a key, useful for debugging and monitoring. SET with EX 30 and NX combines a TTL with 'only set if the key does not exist,' which is the building block of a simple distributed lock that auto-releases after thirty seconds even if the holder crashes.

PERSIST removes the TTL from a key, making it permanent again. Together these commands show the full lifecycle control Redis gives you over key lifetime: set an expiry, inspect it, atomically create-with-expiry, or remove the expiry entirely. This fine-grained control is why so many ephemeral-data patterns map cleanly onto Redis.

This slide addresses what happens when RAM runs out, which it inevitably will if you keep writing without bounds. You set a maxmemory limit, and when Redis reaches it, it applies an eviction policy you've chosen to decide which keys to drop to make room. allkeys-lru evicts the least recently used key across the whole keyspace — a sensible default for a pure cache. volatile-ttl evicts the key with the nearest expiry among those that have a TTL. noeviction refuses new writes and returns errors instead of dropping anything.

The correct policy depends entirely on Redis's role. If it's a cache where everything is rebuildable, an allkeys policy that freely evicts is right. If some keys are precious and others disposable, a volatile-* policy that only evicts keys with TTLs protects the important ones. Choosing this consciously is critical — the wrong policy is a common cause of production surprises, as post five details.

This decision tree captures the eviction logic as a clear yes/no flow. First: has maxmemory been reached? If not, the write is simply accepted. If it has, the next question is the policy. If the policy is noeviction, Redis rejects the write and returns an error to the client. Otherwise, it evicts one or more keys according to the configured policy — for example, the least recently used — and then accepts the write.

Tracing your configuration through this tree is the fastest way to predict how Redis will behave under memory pressure. The branch that catches people off guard is noeviction: writes start failing rather than data being dropped, which can look like a mysterious application error rather than a memory problem. Knowing the tree means you can diagnose 'why are my writes failing?' in seconds.

This snippet shows the configuration directives that tie the previous concepts together, as they'd appear in redis.conf. maxmemory caps how much RAM Redis may use. maxmemory-policy sets the eviction strategy — here allkeys-lru for a cache. appendonly yes enables the AOF log, and appendfsync everysec controls how often it's flushed to disk, balancing durability against performance. The save directive defines RDB snapshot triggers — here, snapshot if at least one key changed in 900 seconds.

These few lines encode the operational personality of a Redis instance: how much memory it uses, what it discards under pressure, and how durably it persists. Setting them deliberately — rather than running with defaults you haven't examined — is the difference between a Redis you operate and one that surprises you. Each maps directly to a concept from earlier in this post.

This slide covers availability — how Redis keeps serving when a server dies. A replica maintains a live copy of a primary by continuously receiving the stream of writes the primary applies. Replicas can serve read traffic to spread load, and they stand ready to be promoted if the primary fails. On top of this, Redis Sentinel monitors the primary and automatically promotes a replica to primary when it detects a failure, handling failover without manual intervention.

For scaling beyond one machine's memory or throughput, Redis Cluster shards the keyspace across multiple primaries, each with its own replicas. Together these give Redis high availability and horizontal scale. The mental model to keep is layered: replication for copies and read scaling, Sentinel for automatic failover, and Cluster for sharding — each solving a distinct availability or scale problem.

This recap pins the mechanics into five lines: one thread makes commands atomic; RDB snapshots plus the AOF log provide durability; TTLs expire keys both lazily and actively; maxmemory with a policy controls eviction; and replicas with Sentinel or Cluster provide high availability. Each maps to a concrete operational lever you now understand.

The payoff of post three is that Redis stops being magic. You know why it's fast, how it survives restarts, how it manages finite memory, and how it stays available — which means you can configure and troubleshoot it deliberately rather than by superstition.

Post three explained how Redis works internally. Post four gets hands-on: we wire Redis into real Python code, implement the cache-aside pattern you'll use most of the time, set TTLs, invalidate on writes, and use hashes for structured objects — turning slow database calls into microsecond memory hits.