Cache Invalidation & Eviction

There’s a famous line: “There are only two hard things in computer science — cache invalidation and naming things.” It’s funny because it’s true. Getting stale copies out of the cache at the right moment is genuinely one of the hardest problems we deal with. Let’s break it into the two separate jobs people usually mix up.

• Invalidation — removing data because it’s wrong now (the source changed).
• Eviction — removing data because we’re out of room (memory is full).

Different problems, different tools.

Invalidation: keeping copies honest

TTL / expiry

The simplest approach. We stamp every cache entry with a time-to-live and the cache auto-deletes it when the clock runs out. No cleanup code, no coordination — the entry just expires.

// "trust this copy for 60 seconds, then forget it"
await cache.set("product:42", data, "EX", 60);

TTL is a bet on staleness: we’re saying “being up to 60 seconds out of date is fine here.” Short TTL = fresher but more misses. Long TTL = more hits but staler data. It’s the same trade-off we always tune.

Active invalidation on write

TTL alone means we serve stale data until it expires. When we can’t tolerate that, we actively kill the entry the moment the underlying data changes — usually right after the DB write.

async function updatePrice(id, price) {
  await db.products.update(id, { price });  // source of truth first
  await cache.del(`product:${id}`);         // then evict the stale copy
}

Delete, don’t overwrite. Deleting lets the next read repopulate from the fresh DB (cache-aside), which sidesteps a whole class of race conditions where two writers overwrite each other’s cached value in the wrong order.

Eviction: making room

TTL removes stale things. Eviction removes things when memory fills up, even if they’re still valid. The cache has to pick a victim — and the rule it uses is the eviction policy.

LRU is the go-to. It leans on the idea that data used recently will probably be used again soon (temporal locality), and it’s cheap to approximate. Reach for LFU when some keys are always hot and shouldn’t be evicted just because they went quiet for a moment. FIFO and random are mostly there when we want dead-simple bookkeeping. In Redis this is the maxmemory-policy setting (allkeys-lru, allkeys-lfu, and friends).

The cache stampede (thundering herd)

Here’s the failure that takes down real systems. A super-popular key expires. In that instant, every request that wanted it misses at the same time — and they all pile onto the database together to recompute it. The DB, sized for a trickle of misses, gets a flood. It falls over, which slows everything, which causes more misses. A stampede.

Two fixes we should be able to name:

• Locking (single-flight) — the first request to miss grabs a lock and does the recompute; everyone else waits for that one result instead of all hitting the DB. Only one trip to the database, no matter how many callers.
• Jittered TTL — instead of a fixed 60s, we set 60s ± a random few seconds. Now keys expire at slightly different times, so misses spread out instead of all landing together. Cheap and effective.

A third trick worth a mention: early/probabilistic refresh — recompute a hot key a little before it expires, in the background, so it never actually goes cold.

Interview soundbite

Invalidation removes wrong data (TTL expiry, or an active del on write — delete rather than overwrite to dodge races). Eviction removes data when memory is full, driven by a policy: LRU (default, evict least-recently-used), LFU (least-frequently-used), FIFO, or random. And watch for the cache stampede — a hot key expiring sends every request to the DB at once; fix it with a lock (single-flight) or a jittered TTL so expirations don’t sync up.