Caching Patterns

Caching sounds simple on the surface — store stuff in memory so we don’t hit the database every time. But the devil is in the details. How we cache matters a lot. Do we populate the cache on reads or writes? What happens when the cache and database go out of sync? What if a popular key expires and a thousand requests hit the database at once?

Let’s look at the main caching patterns and the problems we’ll inevitably run into.

Cache-Aside (Lazy Loading)

This is the most common pattern. The application manages the cache itself:

1. Read: Check the cache first. If found (cache hit), return it. If not (cache miss), read from the database, put it in the cache, then return it.
2. Write: Write directly to the database. Optionally invalidate (delete) the cache entry.

# Pseudocode for cache-aside
def get_user(user_id):
    # Step 1: Check cache
    cached = redis.GET(f"user:{user_id}")
    if cached:
        return deserialize(cached)    # cache hit!

    # Step 2: Cache miss — read from DB
    user = db.query("SELECT * FROM users WHERE id = %s", user_id)

    # Step 3: Populate cache with TTL
    redis.SETEX(f"user:{user_id}", 3600, serialize(user))

    return user

Pros: Only caches data that’s actually requested. Simple to implement. Cons: First request is always slow (cache miss). Data can become stale if the DB changes and we don’t invalidate.

Write-Through

With write-through, every write goes to both the cache and the database at the same time. The cache is always up to date.

# Pseudocode for write-through
def update_user(user_id, data):
    # Write to DB
    db.query("UPDATE users SET ... WHERE id = %s", user_id, data)

    # Write to cache at the same time
    redis.SETEX(f"user:{user_id}", 3600, serialize(data))

Pros: Cache is always consistent with the DB. No stale reads. Cons: Higher write latency (two writes per operation). We cache data that might never be read.

Write-Behind (Write-Back)

This is the performance-optimized version. We write to the cache immediately and asynchronously flush to the database later (in batches or after a delay).

# Pseudocode for write-behind
def update_user(user_id, data):
    # Write to cache immediately — fast!
    redis.SETEX(f"user:{user_id}", 3600, serialize(data))

    # Queue the write for async processing
    queue.push({"action": "update_user", "id": user_id, "data": data})

# Background worker picks up the queue and writes to DB

Pros: Super fast writes (only cache). Batching reduces DB load. Cons: If the cache crashes before flushing, we lose data. More complex to implement. Debugging is harder.

Read-Through

Similar to cache-aside, but the cache itself is responsible for fetching from the DB on a miss. The application doesn’t talk to the DB directly — it always goes through the cache layer.

In simple language, the only difference from cache-aside is who does the fetching. In cache-aside, our app code fetches from DB. In read-through, the cache library handles it.

Cache Invalidation Strategies

Phil Karlton famously said there are only two hard things in computer science: cache invalidation and naming things. He wasn’t wrong.

• TTL-based — set an expiration time. Simple but the data can be stale until the TTL expires.
• Event-driven — when the DB changes, publish an event to invalidate the cache. More accurate but more complex.
• Version-based — include a version number in the cache key. When data changes, bump the version. Old entries naturally become orphans.

Cache Stampede (Thundering Herd)

This is a nasty problem. Imagine a popular cache key (say, the homepage data) expires. Suddenly, thousands of requests all see a cache miss at the same time and all hit the database simultaneously. The DB gets crushed.

Solutions

1. Distributed lock (SETNX) Only one request fetches from the DB. Others wait for the cache to be repopulated.

# Only one process gets the lock and rebuilds the cache
def get_with_lock(key):
    value = redis.GET(key)
    if value:
        return value

    # Try to acquire a lock
    if redis.SET(f"lock:{key}", "1", NX=True, EX=10):
        # We got the lock — rebuild the cache
        value = db.query(...)
        redis.SETEX(key, 3600, value)
        redis.DEL(f"lock:{key}")
        return value
    else:
        # Someone else is rebuilding — wait and retry
        sleep(0.1)
        return get_with_lock(key)

2. Probabilistic early expiration Each request has a small random chance of refreshing the cache before it actually expires. The hotter the key, the more likely someone refreshes it early.

3. Background refresh A background job refreshes popular keys before they expire. The cache never actually becomes empty.

Cache Penetration

This happens when requests keep asking for keys that don’t exist — not in the cache AND not in the database. Every request is a cache miss that hits the DB, and the DB always returns nothing.

Think of it like someone querying user:99999999 over and over. The cache never helps because there’s nothing to cache.

Solutions

1. Cache null values If the DB returns nothing, cache a null/empty marker with a short TTL.

# Cache the "not found" result too
value = db.query("SELECT * FROM users WHERE id = %s", user_id)
if value is None:
    redis.SETEX(f"user:{user_id}", 300, "NULL")   # cache for 5 min

2. Bloom filter A bloom filter is a space-efficient data structure that tells us “this key DEFINITELY doesn’t exist” or “this key MIGHT exist.” We check the bloom filter before hitting the cache or DB.

# Before hitting cache or DB, check bloom filter
if not bloom_filter.might_contain(user_id):
    return None    # definitely doesn't exist, skip everything

# Otherwise, proceed with normal cache-aside logic

Cache Avalanche

Similar to a stampede, but worse. This happens when a large number of keys expire at the same time — maybe because they were all set with the same TTL during a cache warm-up.

Solution: Add a random jitter to TTL values.

# Instead of a fixed TTL
redis.SETEX(key, 3600, value)

# Add random jitter (3600 ± 300 seconds)
import random
ttl = 3600 + random.randint(-300, 300)
redis.SETEX(key, ttl, value)

Quick Reference