Redis: Caching & Beyond

This cover names the uncomfortable truth about caches: a cache that passes every test can still take down production, and not by malfunctioning — by failing in the specific ways caches fail under real load. The disasters are quiet and they cluster around the moments a cache is supposed to help: an expiry fires, an invalidation is needed, memory fills up.

Post five is the field guide to these traps. Each one — the stampede, stale data, mistaking Redis for durable storage, missing eviction config, and oversized keys — survives code review because the code looks correct and works fine at low volume. They show up as 3am pages months after they shipped, which is exactly why knowing them in advance is so valuable.

The cache stampede, also called the thundering herd, is the most dangerous caching failure because it weaponizes the cache's own behavior. Picture a very popular key — a homepage, a trending product — with a TTL. The moment it expires, every concurrent request that would have been a hit becomes a miss simultaneously. If a thousand requests arrive in that instant, all thousand miss together and all thousand hit the database to rebuild the same value.

The database, which the cache was supposed to protect, suddenly receives a thousand identical expensive queries at once. It can buckle under that synchronized load, causing the very outage caching was meant to prevent. The cruel irony is that the more popular and well-cached a key is, the bigger its stampede when it expires. The next slides show how to defuse it.

This flow diagram traces the stampede as a chain of cause and effect. A hot key's TTL reaches zero and it's gone. Because it was hot, a large number of requests are in flight for it, and they all miss the cache at the same instant. With nothing in the cache, every one of them falls through to the database to rebuild the value. The database receives that synchronized flood and overloads.

Seeing it as a chain reveals where to intervene. You can break it at the 'all hit DB' link by ensuring only one request rebuilds while the others wait or serve a slightly stale value — the lock approach in the next slide. Or you can break it earlier by avoiding synchronized expiry altogether, for example by jittering TTLs so hot keys don't all expire at the same moment. Either way, the fix is to prevent the simultaneous rebuild.

This snippet implements the standard stampede defense: a rebuild lock. The function checks the cache first and returns on a hit, as usual. On a miss, instead of every caller rebuilding, it tries to acquire a short-lived lock with SET nx=True (set only if absent) and a 10-second expiry. Only one caller wins the lock; it rebuilds the value from the database, writes it back with the real TTL, and releases the lock. The other callers, having failed to get the lock, can wait briefly and retry, or serve a stale value.

The nx flag is what makes this safe under concurrency — it's an atomic test-and-set, so exactly one of the thousand simultaneous callers acquires the lock. The ex on the lock prevents a deadlock if the rebuilding process crashes before releasing it. This pattern turns a thousand simultaneous database queries into one, which is precisely the protection the cache was supposed to provide.

This slide covers the stale data trap, the failure mode behind 'why is the user still seeing the old value?' It happens when a write successfully updates the database but the corresponding cache invalidation fails or is simply forgotten on some code path. The cache keeps serving the old value, and users see outdated data. If the key has no TTL, it serves that stale value forever — until someone notices and manually clears it.

The fix has two parts, and you want both. First, invalidate on every write path — audit your code so that anything mutating the database also deletes the relevant cache keys. Second, always keep a TTL as a backstop, so that even a missed invalidation self-heals within minutes when the key expires. Belt and suspenders: explicit invalidation for correctness now, TTL for recovery when invalidation inevitably gets missed somewhere.

This slide tackles a dangerous architectural mistake: treating Redis as your durable system of record. Redis can persist to disk, and that lulls people into trusting it with data they can't afford to lose. But persistence in Redis is best-effort relative to a real database — a crash between an AOF fsync, an eviction under memory pressure, or a misconfigured FLUSHALL can lose recent or even all data. Its design optimizes for speed, with durability as a configurable add-on.

The rule is firm: never let Redis hold the only copy of anything critical — orders, payments, account records. Those belong in a durable database that's built to never lose committed data. Redis should hold copies, derived data, and genuinely ephemeral state like sessions and rate-limit counters that you can afford to lose or rebuild. Keep the source of truth in the database; let Redis be the fast layer in front of it.

This comparison crystallizes the source-of-truth-versus-cache distinction. The database is durable on disk, survives crashes, owns the critical data, and is allowed to be slower because correctness is its job. Redis is fast and in memory, can be lost safely, holds copies, and stores data that's rebuildable from the truth. Keeping these roles distinct in your head is what prevents the previous slide's mistake.

The practical test when deciding where data lives: ask 'if this vanished right now, is it a catastrophe or an inconvenience?' If catastrophe — it must be in the durable database. If inconvenience — a cache miss, a re-login, a reset counter — Redis is fine. Data that fails this test by being both critical and Redis-only is a latent disaster waiting for the next crash or eviction.

This slide warns about a configuration omission that causes real outages: not setting a maxmemory policy. If you leave maxmemory unset, Redis will happily allocate until it consumes all available RAM, at which point the operating system's out-of-memory killer terminates the process — a hard crash. Conversely, if maxmemory is set but the policy is the default noeviction, Redis stops accepting writes once full and returns errors, which surfaces as mysterious write failures in your application.

Neither default is what you want for a cache. You must consciously choose a policy that matches Redis's role. For a pure cache where everything is rebuildable, allkeys-lru lets Redis evict the least recently used keys to make room. When only some keys are safe to drop, a volatile-* policy evicts only keys that carry a TTL, protecting the rest. The point is to decide deliberately rather than discover your eviction behavior during an incident.

This snippet shows how to set the eviction policy on purpose and verify it. CONFIG SET applies maxmemory and maxmemory-policy at runtime (you'd also put them in redis.conf so they survive a restart). Here we cap memory at 2gb and choose allkeys-lru, the sensible default for a pure cache. The verification commands matter just as much: INFO memory reports used_memory and the evicted_keys counter, so you can confirm eviction is actually happening and watch how aggressive it is.

CONFIG GET maxmemory-policy reads back the active policy, a good thing to check before assuming what your instance does under pressure. The habit to build is treating eviction as something you configure and monitor, not something you discover. A rising evicted_keys count under load tells you the cache is undersized; flat memory with rejected writes tells you the policy is wrong. Both are visible if you look.

This slide addresses big keys and slow commands, a failure mode that interacts dangerously with the single-threaded model from post three. A single oversized key — a list of millions of elements, a hash with enormous fields — makes any command operating on it slow. And because Redis runs one command at a time on one thread, that slow command blocks every other client until it finishes. One bad key becomes everyone's latency spike.

The related trap is the KEYS command. KEYS * scans the entire keyspace in one blocking operation; on a large database it can freeze Redis for seconds while every other request waits. The fixes are structural: keep individual keys bounded in size, split giant collections into smaller ones, and never use KEYS in production — use SCAN, which iterates in small cursor-based batches without monopolizing the thread. The next slide shows SCAN in action.

This snippet contrasts the dangerous KEYS command with the safe SCAN alternative. KEYS user:* would match and return every matching key in a single blocking pass — fine on a tiny dev database, catastrophic on a production one with millions of keys, because it stalls the single thread for the entire scan. The commented-out line is the trap to avoid.

SCAN solves this with cursor-based iteration. You call SCAN with a cursor (starting at 0), a MATCH pattern, and a COUNT hint for batch size; it returns a new cursor plus a small batch of keys. You repeat the call with each returned cursor until it comes back as 0, signaling completion. Each individual SCAN call is fast and doesn't block other clients, so you can iterate a huge keyspace without freezing Redis. The rule is simple: SCAN for iteration, never KEYS — and the same cursor pattern exists for HSCAN, SSCAN, and ZSCAN on large collections.

This recap turns the five traps into a checklist: guard hot-key rebuilds against stampedes with a lock or jittered TTLs; invalidate on every write and keep a TTL as a backstop; remember Redis is a cache, not your source of truth; always set a maxmemory policy that matches its role; and avoid big keys and KEYS *, using SCAN to iterate. Run a Redis design or incident through this list and you'll catch the cause more often than not.

These mistakes account for the bulk of real-world Redis incidents precisely because each leaves code that works fine at low volume and fails under production load or at the worst moment. Knowing them in advance turns vague 'Redis fell over' reports into specific, preventable failure modes.

That completes the deep dive on Redis — from its in-memory data structures and why it matters, through how it works under the hood and how to cache with it, to the traps that bite in production. Day 86 continues the NoSQL track with a higher-level question: how to choose the right database for the job among the many options now on the table.