Timeouts, Retries & Circuit Breakers

When our service calls another service, that call can be slow, fail, or hang forever. Timeouts, retries, and circuit breakers are the three patterns that stop one flaky dependency from dragging our whole service down with it. In simple language — they’re how we call something unreliable without betting our own uptime on it.

The why is a chain reaction. A downstream service gets slow → our requests to it pile up waiting → our threads/connections get exhausted → we go down → whatever calls us goes down. This is a cascading failure, and these three patterns are the standard defense.

Timeouts — never wait forever

A timeout is a hard limit on how long we’ll wait for a response before giving up. This is the most important one and the most often forgotten.

Why it matters: without a timeout, a hung downstream call holds our resources hostage indefinitely. One stuck dependency slowly consumes every connection in our pool, and now we’re down too — even though our own code is fine.

// Every outbound call gets a deadline. No exceptions.
const res = await fetch(url, { signal: AbortSignal.timeout(2000) }); // 2s cap

Rule of thumb: every network call — HTTP, DB query, cache lookup — gets a timeout. “Wait forever” is never the right default.

Retries — but carefully

A retry is trying again after a failure, on the bet that the failure was transient (a blip, a brief network hiccup). Sometimes that’s exactly right. But naive retries are dangerous, so two rules:

Rule 1 — only retry idempotent operations. An operation is idempotent if doing it twice has the same effect as doing it once (a GET, or a PUT that sets a value). Retrying a non-idempotent POST /charge might charge the customer twice. If we can’t guarantee idempotency, we don’t retry — or we add an idempotency key so the server dedupes.

Rule 2 — back off with exponential backoff + jitter. Don’t retry immediately, and don’t retry on a fixed interval. Wait longer each time (1s, 2s, 4s…) and add jitter (randomness) so a thousand clients don’t all retry in sync.

// exponential backoff with full jitter
const base = 100; // ms
const delay = Math.random() * (base * 2 ** attempt); // attempt = 0,1,2...
await sleep(delay);

The retry storm

Here’s the trap interviewers love. A downstream service is struggling under load. Every caller retries. Retries triple the traffic hitting an already-overloaded service. It falls over completely. This is a retry storm (or retry amplification) — retries turning a small blip into a full outage.

The jitter above spreads retries out. But the real fix for “the downstream is genuinely overloaded” is the next pattern.

Circuit breaker

A circuit breaker wraps a downstream call and stops calling it when it’s clearly failing — so we fail fast instead of piling on. Think of it exactly like the electrical breaker in our house: when there’s a fault, it trips and cuts the circuit rather than letting the wiring burn.

It’s a little state machine with three states.

• Closed — normal. Requests pass through. We count failures.
• Open — the failure threshold was crossed, so the breaker trips. Now every call fails instantly without touching the downstream. This gives the sick service room to recover instead of getting hammered.
• Half-open — after a cooldown, we cautiously let a few probe requests through. If they succeed, close the breaker (back to normal). If they fail, snap back open and wait again.

The magic is the open state: by refusing to call a failing service, we stop the retry storm and fail fast (users get an instant error or fallback instead of waiting for a timeout on every request).

Bulkheads, briefly

A bulkhead isolates resources so a failure in one area can’t drain everything — named after the sealed compartments in a ship’s hull that stop one flooded section from sinking the whole boat. In practice: give each downstream dependency its own connection pool / thread pool. If service X hangs and exhausts its pool, services Y and Z keep working because they have separate pools.

Interview soundbite

Three layers of defense against a cascading failure: timeouts (never wait forever — every call gets a deadline), retries (only for idempotent/transient failures, always with exponential backoff + jitter to avoid a retry storm), and the circuit breaker (a closed → open → half-open state machine that trips after repeated failures and fails fast so the sick dependency can recover). Add bulkheads — separate resource pools per dependency — so one hung service can’t drain the rest.