Gyaan

Why & Where We Cache

beginner caching performance latency cdn

A cache is a fast copy of expensive-to-get data, kept close to whoever needs it. That’s the whole idea.

In simple language — instead of asking the slow database the same question again and again, we save the answer somewhere fast and hand out that copy.

Why we cache

Three reasons, and honestly every interview answer boils down to these:

  • Latency — reading from memory is nanoseconds; hitting a database across the network is milliseconds. A cache turns a slow trip into a fast one.
  • Load — every request the cache answers is a request the database never sees. We shield the expensive backend from getting hammered.
  • Cost — fewer queries, fewer compute cycles, less bandwidth. Serving a cached byte is dirt cheap compared to recomputing it.

Think of it like keeping a snack in your desk drawer instead of walking to the shop every time you’re hungry. Same snack, way less effort.

Where caching happens

Caching isn’t one thing in one place. It happens at every hop between the user and our data. A request can be answered — and stop travelling — at any of these layers.

A request travels down until something can answer it
Browser cache — disk/memory on the user's device. Images, JS, CSS.
↓ if not there
CDN / edge cache — a server near the user. Static assets, cacheable pages.
↓ if not there
Reverse proxy — nginx/Varnish in front of our app. Full HTTP responses.
↓ if not there
Application / in-memory cache — Redis, Memcached, or a local map. Query results, sessions.
↓ if not there
Database — the slow source of truth (it caches too, buffer pool + query cache).

The higher up the stack we answer, the faster and cheaper it is. The browser cache is the fastest possible cache — the request doesn’t even leave the device.

Cache hit vs miss

Two words that come up in every caching conversation.

  • Cache hit — the data is in the cache. We return it straight away. Fast, cheap, done.
  • Cache miss — the data isn’t there. We go fetch it from the slower source, usually store a copy on the way back, then return it.

The number we care about is the hit ratio — hits divided by total lookups. A cache with a 95% hit ratio means only 1 in 20 requests actually bothers the database. A cache with a 5% hit ratio is just overhead — we pay to check it and almost always miss anyway.

// The shape of nearly every cache read
let value = cache.get(key);   // check the fast layer first
if (value === undefined) {    // MISS
  value = db.query(key);      // pay the slow cost once
  cache.set(key, value);      // remember it for next time
}
return value;                 // HIT on every future call

The one trade-off: staleness

Here’s the catch. A cache holds a copy, and copies drift out of date. The moment the real data changes, our cached copy is a lie until we refresh it. That gap is staleness.

Every caching decision is really one question: how stale can we afford to be? A product price might need to be fresh within seconds. A user’s avatar can be minutes or hours old and nobody cares. We tune the cache lifetime around that tolerance — short lifetimes stay fresh but miss more, long lifetimes hit more but risk serving old data.

There’s an old joke that the two hardest problems in computer science are cache invalidation and naming things. It’s a joke because keeping copies in sync is genuinely hard — we cover the strategies for it in the next notes.

Practical takeaway

A cache is a fast copy of expensive data, and we place copies at every layer from the browser down to the database. We win on latency, load, and cost — we pay in staleness. The whole art of caching is deciding how fresh the copy needs to be, and which layer is the cheapest place to answer from.

References