Backend Engineering

“What happens when you type a URL and hit enter?” is the most classic interview warm-up there is. In simple language — a whole relay race of steps kicks off before a single byte of the page shows up.

Let’s walk through it end-to-end. Same thing happens whether it’s a browser loading a page or our mobile app calling an API.

The client-server model

The web runs on a simple deal. A client (browser, app, curl) asks for something. A server listens, does the work, and answers back. The client always starts the conversation — servers never call us out of the blue over plain HTTP.

Think of it like ordering at a restaurant. We (the client) ask the waiter for a dish. The kitchen (the server) cooks it and sends it out. We don’t walk into the kitchen ourselves.

The full journey of a request

Here’s the whole relay, top to bottom.

Step 1 — DNS resolution

Computers talk in IP addresses, not names. DNS (Domain Name System) is the phone book that maps api.shop.com to an IP.

Our machine checks caches first (browser cache → OS cache → the router), and only if it misses does it ask a resolver which walks the DNS hierarchy (root → .com → shop.com). The answer comes back with a TTL — how long we’re allowed to cache it. This is why a DNS change can take a while to spread everywhere.

Step 2 — TCP handshake

Now we have an IP. Before sending any data, TCP does a 3-way handshake to open a reliable pipe: SYN → SYN-ACK → ACK. After those three packets, both sides agree the connection is live.

We cover TCP in depth in its own note. For now — this is the “are you there? yes I’m here. great, let’s talk” bit.

Step 3 — TLS handshake

Because it’s https://, we layer encryption on top. The client and server verify the server’s certificate, prove the server owns the domain, and agree on secret keys so nobody snooping the wire can read the traffic. Also its own note — just know it happens right after TCP and before any HTTP.

Step 4 — The HTTP request

Finally the actual ask. A plain-text request with a method, a path, headers, and maybe a body.

GET /orders HTTP/1.1
Host: api.shop.com
Authorization: Bearer eyJhbGc...
Accept: application/json

Step 5 — Where the backend sits

This is our world. The request lands, and the backend does the heavy lifting:

• Routing — match /orders to a handler
• Auth — is this token valid? is this user allowed?
• Business logic — validate input, run the rules
• Data layer — query the database or a cache, call other services
• Serialize — turn the result into JSON

The backend is the brain between the network and the data. Everything in this whole collection is about doing steps here well.

Step 6 & 7 — Response and render

The server sends back a status code (200, 404, 500…), headers, and a body. The client reads the status, parses the body, and does its thing — a browser paints the page, our app updates state.

Often the first response is HTML that references more assets (CSS, JS, images), and each of those kicks off its own mini request lifecycle. A single page load can be dozens of these.

Practical takeaway

• DNS → TCP → TLS → HTTP → server → response is the skeleton — memorize the order.
• The client always starts the conversation; the server responds.
• DNS is cached at every layer, gated by TTL.
• TCP gives a reliable pipe, TLS wraps it in encryption — both happen before the first HTTP byte.
• The backend is steps 5’s routing, auth, logic, and data access — that’s the job.

HTTP is the language clients and servers speak. In simple language — it’s a set of rules for “here’s what I want” and “here’s what you get”. Let’s nail the parts interviewers poke at.

Request & response anatomy

Every HTTP message has the same shape: a start line, headers, a blank line, then an optional body.

POST /orders HTTP/1.1        <- method, path, version
Host: api.shop.com           <- headers
Content-Type: application/json
                             <- blank line separates headers from body
{ "item": "book", "qty": 2 } <- body

The response mirrors it — a status line, headers, blank line, body.

HTTP/1.1 201 Created
Content-Type: application/json
Location: /orders/42

{ "id": 42, "status": "confirmed" }

The methods

Each method is a verb — what we want to do with the resource.

• GET — read something. No body, shouldn’t change anything.
• POST — create something new, or “run this action”. Not repeatable safely.
• PUT — replace a resource entirely at a known URL.
• PATCH — partially update a resource (just the fields we send).
• DELETE — remove a resource.

Two words that come up constantly:

Safe means the method doesn’t change server state — GET is safe, we can call it all day and nothing happens. Idempotent (fancy word, plain meaning) means calling it once or ten times lands the same final result. GET, PUT, DELETE are idempotent — deleting order 42 twice still leaves it deleted. POST is not — POST twice and you might create two orders. That’s the whole reason double-clicking “Buy” can be dangerous.

Status codes — the families

The first digit tells us the category. In simple language: 1xx = hold on, 2xx = worked, 3xx = go elsewhere, 4xx = you messed up, 5xx = we messed up.

The ones worth memorizing cold:

• 200 OK — generic success (GET with a body).
• 201 Created — POST made a new resource; include a Location header.
• 204 No Content — success but nothing to send back (a DELETE, say).
• 301 vs 302 — 301 is permanent (browsers cache it, SEO moves), 302 is temporary.
• 304 Not Modified — “your cached copy is still good” (see ETag below).
• 400 Bad Request — the request itself is malformed.
• 401 Unauthorized — we don’t know who you are (missing/bad credentials).
• 403 Forbidden — we know who you are, you’re just not allowed.
• 404 Not Found — no such resource.
• 409 Conflict — the request clashes with current state (duplicate signup, version conflict).
• 422 Unprocessable Entity — syntax is fine but the data fails validation.
• 429 Too Many Requests — you’re rate-limited, slow down.
• 500 Internal Server Error — our code threw and we didn’t handle it.
• 502 Bad Gateway — a proxy got a garbage response from upstream.
• 503 Service Unavailable — server is down/overloaded, try later.

The 401 vs 403 and 502 vs 503 pairs are favorite trick questions. 401 = “who are you?”, 403 = “no.” 502 = “the server behind me is broken”, 503 = “I’m alive but can’t serve right now.”

Headers worth knowing

Headers are the metadata bolted onto every request and response.

• Content-Type — what the body is (application/json, text/html).
• Accept — what the client wants back (content negotiation).
• Authorization — credentials, usually Bearer <token>.
• Cache-Control — caching rules (max-age=3600, no-store).
• ETag — a fingerprint of a resource. The client sends it back as If-None-Match; if it still matches, the server replies 304 and skips resending the body. Saves bandwidth big time.

HTTP/1.1 vs 2 vs 3

Same HTTP semantics (methods, status codes), different plumbing underneath.

HTTP/1.1 opens one request at a time per connection. Send a request, wait for the response, then the next. That waiting is head-of-line blocking — one slow response stalls everything behind it. Browsers hacked around it by opening ~6 connections per host.

HTTP/2 adds multiplexing — many requests share one TCP connection, interleaved as independent streams. Plus header compression and server push. Big win. But there’s a catch: it still rides on TCP, so a single lost packet stalls all streams (head-of-line blocking moved down to the TCP layer).

HTTP/3 fixes that by ditching TCP for QUIC, a protocol built on UDP. Streams are truly independent, so one lost packet only stalls its own stream. QUIC also folds the TLS handshake into the connection setup, so connecting is faster.

Interview soundbite

HTTP is a request-response text protocol. Methods are verbs (GET reads, POST creates); GET/PUT/DELETE are idempotent, POST is not. Status families: 2xx worked, 3xx go elsewhere, 4xx client’s fault, 5xx server’s fault — know 401 (who are you) vs 403 (not allowed) and 502 vs 503. HTTP/2 added multiplexing over one TCP connection; HTTP/3 moved to QUIC on UDP to kill TCP-level head-of-line blocking.

HTTPS is just HTTP with a security layer bolted underneath. That layer is TLS (Transport Layer Security — the modern name for what people still call SSL). In simple language — TLS wraps the whole conversation in a locked box so nobody in the middle can read or tamper with it.

Why HTTPS at all

HTTPS gives us three guarantees. Worth memorizing the trio:

• Encryption — snoopers on the wire (coffee-shop wifi, your ISP) see gibberish, not your password.
• Integrity — if someone flips even one bit in transit, the receiver notices and rejects it. No silent tampering.
• Authentication — we’re actually talking to bank.com, not an imposter pretending to be it.

Plain HTTP gives us none of these. That’s why browsers now shame every http:// site with a “Not Secure” badge.

Symmetric vs asymmetric — the two kinds of crypto

TLS uses both, and knowing the difference is the whole game.

Symmetric encryption — one shared secret key encrypts and decrypts. Fast. The catch: both sides need the same key, and shouting a secret key across the internet is obviously a bad idea.

Asymmetric encryption — a key pair: a public key and a private key. Anything locked with the public key can only be opened by the private key. The public key can be shared with the world; the private key never leaves the server. Slower, but it solves the “how do we agree on a secret over an open wire” problem.

Think of it like this: asymmetric is an expensive armored truck we use once to safely hand over a secret. Symmetric is the cheap fast car we use for every trip after that.

So TLS does both: use slow asymmetric crypto during the handshake just to agree on a shared symmetric key, then use fast symmetric crypto for the actual data. Best of both worlds.

The TLS handshake

This happens right after the TCP handshake, before any HTTP. Here’s the flow (simplified TLS 1.2-style so the steps are visible).

Step by step:

1. ClientHello — client says “here are the TLS versions and cipher suites I speak” plus a random number.
2. ServerHello — server picks a version and cipher, sends its own random number.
3. Certificate — server sends its certificate, which contains its public key.
4. Key exchange — using that public key (asymmetric), both sides securely agree on a shared secret, then derive matching symmetric session keys from it. Modern TLS 1.3 uses an ephemeral Diffie-Hellman exchange here so past traffic stays safe even if the private key later leaks (forward secrecy).
5. Finished — both confirm, and every byte after this is encrypted with the fast symmetric keys.

TLS 1.3 trimmed this to a single round trip (and 0-RTT for resumed connections), so it’s noticeably faster than the older two-round-trip dance.

Certificates, CAs, and the chain of trust

A certificate is basically a signed ID card. It says “this public key belongs to bank.com” and it’s signed by a Certificate Authority (CA) — a trusted org like Let’s Encrypt or DigiCert.

But why trust the CA? Because our OS and browser ship with a built-in list of root CAs they trust. Here’s the clever part: CAs don’t sign your site directly with the precious root key. They sign intermediate certs, which sign your site’s cert. That’s the chain of trust.

The browser walks this chain upward. If it reaches a root it already trusts and every signature checks out, the cert is valid. If any link is broken, expired, or self-signed, we get the scary “Your connection is not private” page.

Interview soundbite

HTTPS = HTTP over TLS, giving us encryption, integrity, and authentication. TLS uses asymmetric crypto (a public/private key pair) during the handshake only, to agree on a fast symmetric session key used for the real data. The handshake goes ClientHello → ServerHello → Certificate → key exchange → session keys → encrypted traffic. We trust the server’s certificate because it chains up through intermediate CAs to a root CA our browser already trusts — that’s the chain of trust.

TCP and UDP are the two ways data actually moves across the internet. Both sit on top of IP (which just gets packets from A to B). In simple language — TCP is the careful courier that guarantees delivery, UDP is the guy who throws the package over the fence and hopes for the best.

TCP — reliable and ordered

TCP (Transmission Control Protocol) is connection-oriented. Before any data moves, both sides shake hands and set up a connection. Then TCP promises three things:

• Reliable — every packet is acknowledged; lost ones get retransmitted.
• Ordered — packets are reassembled in the exact order they were sent, even if they arrive scrambled.
• Error-checked — corrupted packets are caught and resent.

The cost of all this bookkeeping is overhead and latency. But when we can’t afford to lose a byte, TCP is the answer.

UDP — fast and fire-and-forget

UDP (User Datagram Protocol) is connectionless. No handshake, no acknowledgements, no reordering. We just fling datagrams at the destination and move on.

That means UDP is lean and fast, but it makes no guarantees — packets can arrive out of order, duplicated, or not at all. If that matters, the application has to handle it itself.

Think of it like this: TCP is a registered letter with tracking and a signature. UDP is a postcard — cheap, quick, and if it gets lost, oh well.

When to use which

The rule of thumb: does losing data break things, or does waiting for it break things?

• Use TCP when every byte matters — web pages (HTTP), database connections, file transfers, emails. A missing chunk of a JSON response is useless.
• Use UDP when speed beats completeness — live video, voice calls, online gaming, DNS lookups. In a video call, a dropped frame from 200ms ago is worthless; we’d rather skip it and stay live than pause to re-fetch it.

Fun fact: HTTP/3 runs on UDP (via QUIC) and rebuilds reliability itself on top — getting UDP’s speed without giving up ordering. DNS uses UDP for quick lookups but falls back to TCP for large responses.

The 3-way handshake

This is how TCP opens a connection. Three packets, and interviewers love it.

SYN → SYN-ACK → ACK. After those three packets, both sides have agreed on starting sequence numbers and the pipe is live. (Closing is a separate 4-way FIN/ACK dance.) UDP does none of this — it just starts sending, which is exactly why it’s faster.

Ports & sockets

An IP address gets us to the right machine. A port gets us to the right program on that machine. One server can run a web server on 443, a database on 5432, and SSH on 22 all at once — the port number keeps the traffic sorted.

Well-known ports worth remembering: 80 (HTTP), 443 (HTTPS), 22 (SSH), 53 (DNS), 5432 (Postgres), 6379 (Redis), 3306 (MySQL).

A socket is the actual connection endpoint — the full combo of IP + port + protocol. A unique TCP connection is identified by four things: source IP, source port, destination IP, destination port. That’s why one server on port 443 can hold thousands of simultaneous connections — each client’s source IP/port makes the tuple unique.

# see what's listening on which ports
lsof -i -P -n | grep LISTEN
# curl an explicit port
curl http://localhost:5432

Interview soundbite

TCP is connection-oriented, reliable, and ordered — it does a SYN/SYN-ACK/ACK handshake and retransmits lost packets, so we use it for HTTP, databases, and file transfer. UDP is connectionless and fire-and-forget with no guarantees, so it’s faster — great for video, VoIP, DNS, and gaming where being late is worse than being lossy. A port identifies which program on a host gets the traffic; a socket is the IP+port endpoint, and a TCP connection is uniquely keyed by the source/destination IP+port tuple.

Here’s the core problem. Plain HTTP is request-response — the client asks, the server answers, done. The server can’t just tap the client on the shoulder and say “hey, new message arrived”. So how do we build a live chat, a stock ticker, or a notifications badge?

In simple language — we need a way for the server to push data to the client instead of waiting to be asked. There are four common approaches, each a step up in real-time-ness.

Short polling — “are we there yet?”

The dumbest-but-simplest option. The client asks on a timer: every few seconds it fires a request, “anything new?” Server answers immediately, usually “nope”.

It works, but it’s wasteful. Most requests come back empty, and there’s always lag — a message that lands right after a poll waits until the next one. Lots of traffic, little payoff.

// short polling — hammer the server every 3s
setInterval(async () => {
  const res = await fetch("/api/messages?since=" + lastId);
  const msgs = await res.json();
  render(msgs); // mostly empty responses
}, 3000);

Long polling — “hold the line”

Smarter. The client asks, but the server holds the request open until it actually has something to say (or a timeout hits). The moment there’s data, it responds. The client immediately opens a new request and waits again.

Same request-response model, but way fewer empty responses and near-instant delivery. It’s how a lot of “real-time” was done before WebSockets existed, and it’s still a solid fallback.

// long polling — server holds the connection until there's data
async function poll() {
  const res = await fetch("/api/updates"); // may hang for ~30s
  render(await res.json());
  poll(); // immediately re-open
}
poll();

SSE — a one-way stream from the server

Server-Sent Events open a single long-lived HTTP connection over which the server streams messages one way: server → client. The client can’t send data back on that channel (it uses normal HTTP requests for that).

It’s built into browsers via EventSource, auto-reconnects for us, and rides over regular HTTP — so proxies and firewalls don’t complain. Perfect for feeds, notifications, live scores, progress bars.

// SSE — the browser handles reconnection for us
const es = new EventSource("/api/stream");
es.onmessage = (e) => render(JSON.parse(e.data));
// one-way: server pushes, we just listen

WebSockets — a two-way pipe

WebSockets give us a full-duplex connection — both sides can send messages any time, independently. In simple language: a real two-way phone call instead of walkie-talkie turns.

It starts life as an HTTP request that gets upgraded to the WebSocket protocol (ws:// or wss://), then stays open as a persistent TCP connection. Once upgraded, it’s no longer HTTP — just a raw message pipe both ways with tiny per-message overhead.

const ws = new WebSocket("wss://api.chat.com/room/42");
ws.onopen = () => ws.send("hello");     // client -> server
ws.onmessage = (e) => render(e.data);   // server -> client, any time

The upgrade handshake

WebSockets sneak in through a normal HTTP request so they work everywhere HTTP works.

GET /room/42 HTTP/1.1
Host: api.chat.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==

The server agrees with a 101 Switching Protocols, and from that point the same TCP connection carries WebSocket frames instead of HTTP.

Which one, when?

Here’s the whole picture — direction of data flow tells the story.

Rules of thumb:

• Only the server needs to push, one-way? → SSE. Simpler than WebSockets, rides plain HTTP, free reconnection.
• Both sides chatter constantly (chat, games, collab editing)? → WebSockets.
• Can’t use either / need a dead-simple fallback? → long polling.
• Short polling? Only for low-stakes stuff where a few seconds of lag is fine and you want zero moving parts.

Interview soundbite

HTTP is request-response, so the server can’t push on its own. Short polling asks on a timer (wasteful, laggy). Long polling holds the request open until there’s data (near real-time, good fallback). SSE is a one-way server→client stream over plain HTTP with auto-reconnect — great for feeds and notifications. WebSockets upgrade an HTTP connection (via Upgrade header → 101 Switching Protocols) into a persistent full-duplex pipe — use them when both sides need to send freely, like chat or multiplayer.

REST is the style almost every HTTP API claims to follow, and interviewers love poking at whether we actually understand it. In simple language — REST is a set of rules for exposing our data as resources that clients read and change using plain HTTP.

REST stands for Representational State Transfer. Scary name, simple idea: we have things (resources), the client asks for a copy of a thing (a representation), and it changes state by sending representations back.

The three ideas that matter

Resources

A resource is any “thing” our API talks about — a user, an order, a product. Each one gets its own URL, called a URI. Think of it like — every noun in our system gets an address.

Representations

We never hand over the actual database row. We hand over a representation of it, usually JSON. The same user resource could be sent as JSON, XML, or CSV — the resource is the concept, the representation is the format on the wire.

Statelessness

This is the big one. Every request carries everything the server needs to handle it. The server keeps no memory of previous requests — no “session” sitting in server RAM tying requests together.

Why do we care? Because a stateless server is easy to scale. Any of our 10 servers behind a load balancer can handle any request, since none of them holds special context. We’ll dig into this more below.

Resource naming — nouns, not verbs

The single most common mistake. URLs should name things (nouns), not actions (verbs). The HTTP method already says the action.

A few naming habits worth locking in:

• Use plural nouns — /users, not /user. A collection is many things; /users/5 is one item inside it.
• Nest for relationships — /users/5/orders reads as “orders belonging to user 5”. Don’t nest more than one or two levels deep, though; deep nesting gets ugly fast.
• Hyphens, lowercase — /order-items, not /orderItems or /Order_Items.
• No file extensions — /users/5, not /users/5.json. Let the Accept header pick the format.

Use HTTP methods the way they’re meant

The method is the verb. Each one has an agreed meaning — use it and clients (and caches, and proxies) instantly understand our intent.

POST /users HTTP/1.1
Content-Type: application/json

{ "name": "Manish", "email": "m@example.com" }

# Server responds:
# 201 Created
# Location: /users/5

Notice we POST to the collection (/users) to create, and the server tells us the new item’s URL in the Location header. That’s the RESTful handshake.

Statelessness in practice

Because the server remembers nothing, the client must send its identity on every request — typically a token in the Authorization header.

GET /users/5/orders HTTP/1.1
Authorization: Bearer eyJhbGci...

There’s no “I logged in three requests ago so you know who I am”. Each request stands on its own. The trade-off: slightly bigger requests, but massively easier scaling and failover.

HATEOAS (the part nobody fully does)

HATEOAS — Hypermedia As The Engine Of Application State — means our responses include links telling the client what it can do next, so it doesn’t hardcode URLs.

{
  "id": 5,
  "status": "pending",
  "_links": {
    "self":   { "href": "/orders/5" },
    "cancel": { "href": "/orders/5/cancel", "method": "POST" }
  }
}

In theory this is the top level of REST maturity. In practice, most “RESTful” APIs skip it — so know what it is, but don’t sweat that your API lacks it.

So what makes an API “RESTful”?

Boiling it down: resources with clean noun URLs, correct HTTP methods, stateless requests, and standard status codes. Hit those and you’re 95% of the way there — HATEOAS is the bonus round.

Interview soundbite

REST models our system as resources addressed by noun-based URLs, manipulated with standard HTTP methods, where every request is stateless — it carries all the context the server needs. Use plural nouns, let the method be the verb (DELETE /users/5, never POST /deleteUser), return proper status codes, and optionally add hypermedia links (HATEOAS) so clients discover actions instead of hardcoding them.

Happy-path APIs are easy. What separates a professional API from a hobby one is how it behaves when things go wrong. In simple language — good error handling means the caller always knows what broke, whose fault it was, and what to do next.

Two things make this work: the right status code (the machine-readable signal) and a consistent error body (the human/developer-readable detail).

Get the status code right

The status code is the first thing every client, proxy, and monitoring tool looks at. The number range alone tells a story.

The golden rule: 4xx means the client should fix its request; 5xx means the client should retry or wait for us to fix things. Mixing these up is a classic bug — returning 200 OK with {"error": "..."} in the body is the worst offender, because it lies to every tool watching the response.

401 vs 403 — the perennial mixup

Easy way to remember: 401 = “who are you?” (we don’t know you, log in), 403 = “I know you, and no” (you’re authenticated but not authorized). The only difference is whether identity is the problem or permission is.

Validation errors: 400 vs 422

Both mean “your input is bad”, and honestly people argue about this forever. The common convention:

• 400 Bad Request — the request is malformed. Broken JSON, missing a required field, wrong content type. The server can’t even parse it properly.
• 422 Unprocessable Entity — the JSON parsed fine, but the values fail our business rules. Email isn’t a valid email, age is negative.

Think of it like — 400 is “I can’t read this”, 422 is “I read it, and it’s wrong”.

A consistent error shape

Every error our API returns should look the same. Nothing is more painful than an API where every endpoint fails differently. A solid minimum:

{
  "code": "VALIDATION_ERROR",
  "message": "One or more fields are invalid.",
  "details": [
    { "field": "email", "issue": "must be a valid email address" },
    { "field": "age",   "issue": "must be >= 0" }
  ],
  "requestId": "req_a1b2c3d4"
}

Three parts do the heavy lifting:

• code — a stable, machine-readable string. Clients branch on this, not on the human message. We can reword message anytime; code is the contract.
• message — a short, human-friendly sentence for a developer reading logs.
• details — the field-by-field breakdown, so a frontend can highlight exactly what’s wrong.

Standardize with problem+json (RFC 9457)

Instead of inventing our own shape, there’s a standard: Problem Details for HTTP APIs, originally RFC 7807, now updated by RFC 9457. It defines a JSON body served with Content-Type: application/problem+json.

HTTP/1.1 422 Unprocessable Entity
Content-Type: application/problem+json

{
  "type": "https://api.example.com/errors/validation",
  "title": "Validation failed",
  "status": 422,
  "detail": "The 'email' field must be a valid email address.",
  "instance": "/users",
  "requestId": "req_a1b2c3d4"
}

The nice bit: type is a URL that can point to docs about that error, and tooling already understands the format. We can add our own fields (like requestId) freely.

Never leak internals

This one’s both a security and a professionalism issue. Never send stack traces, SQL, or file paths to the client. A raw 500 dump tells an attacker our framework, our DB, our directory layout.

// ✗ BAD — leaks everything
{ "error": "ER_DUP_ENTRY at /app/src/db/users.js:42: duplicate key 'email'" }

// ✓ GOOD — safe outside, full detail in our logs
{ "code": "EMAIL_TAKEN", "message": "That email is already registered.",
  "requestId": "req_a1b2c3d4" }

Keep the juicy details in our server logs, keyed by the same requestId.

Correlation / request IDs

That requestId is the glue between the client and our logs. We generate one per request (or accept an incoming X-Request-Id header), attach it to every log line, and return it in every error body — and ideally every response header.

When a user reports “it failed at 3pm”, we ask for the request ID and jump straight to those exact log lines. In a microservices world we pass this correlation ID downstream so we can trace one request across five services.

Interview soundbite

Return honest status codes — 4xx for the caller’s mistake, 5xx for ours, never 200 with an error inside. Ship a consistent error body with a stable machine code, a human message, and field-level details; the application/problem+json standard (RFC 9457) is a ready-made shape. Never leak stack traces or SQL, and stamp every request with a correlation ID you also write into your logs so a reported failure maps to exact log lines.

Once real clients depend on our API, we can’t just change it whenever we feel like it. And once our tables have millions of rows, we can’t return everything at once. Versioning and pagination are the two tools that keep a growing API from falling over. In simple language — versioning lets us change without breaking people, pagination lets us hand out big data in bite-sized pages.

Why version at all

An API is a contract. The moment someone builds against /users returning a name field, renaming that field to fullName breaks their app. We can’t ship breaking changes silently.

Versioning gives us an escape hatch: keep v1 alive for existing clients while v2 carries our shiny new shape. Old clients keep working, new clients opt in.

One nuance worth saying out loud: additive changes don’t need a new version. Adding a new optional field or a new endpoint doesn’t break anyone. We only bump the version for breaking changes — removing a field, renaming one, changing a type.

Three ways to version

Honest take for interviews: URI versioning (/v1/) is the most common because it’s the least surprising. Everyone can see it, test it, and cache it. Header and media-type versioning are “cleaner” in theory, but the invisibility makes them harder to work with day to day. Pick one and be consistent.

Pagination: offset vs cursor

If /users has 2 million rows, returning all of them is a disaster. We hand out pages instead. There are two schools.

Offset / limit — the simple one

GET /users?limit=20&offset=40

“Skip 40, give me the next 20.” Dead simple, and it lets us jump to any page (offset=40 is page 3). Great for a numbered pager on a small dataset.

The catch is two-fold. First, it gets slow at scale — the database still has to walk and count past all 40 (or 40,000) skipped rows before it can return the ones we want. Second, it’s unstable: if someone inserts a row while we page, everything shifts and we see a duplicate or skip an item.

Cursor / keyset — the scalable one

GET /users?limit=20&cursor=eyJpZCI6MTIzfQ

Instead of “skip N”, a cursor says “give me the 20 items after this specific item”. The cursor is usually an encoded pointer to the last row we saw (like its id or a created_at + id pair).

# response
{
  "data": [ ... 20 users ... ],
  "nextCursor": "eyJpZCI6MTQzfQ"
}

The query behind it becomes WHERE id > 123 ORDER BY id LIMIT 20, which uses the index directly — no counting past skipped rows. Fast and stable no matter how deep we page. The trade-off: we lose random page jumps. We can only go next/previous, not “jump to page 500”.

Why cursor scales better

Rule of thumb: offset for small, admin-y lists where page numbers matter; cursor for big feeds, infinite scroll, and public APIs. This is a favorite interview question, so know the “why”.

Filtering & sorting

Neither of these needs new endpoints — they’re just query parameters on the collection.

GET /orders?status=shipped&minTotal=100&sort=-createdAt&limit=20

A few conventions that keep this sane:

• Filter by field name — ?status=shipped filters where status = "shipped". One param per field.
• Sorting with a sign — sort=-createdAt means descending by createdAt; drop the minus for ascending. Some APIs use sort=createdAt&order=desc instead — either is fine, just be consistent.
• Combine freely — filters, sort, and pagination all live together on the same collection URL.
• Whitelist everything — only allow filtering/sorting on fields we’ve explicitly indexed and approved. Blindly turning query params into SQL is both a performance and a security hole.

Practical takeaway

Version only on breaking changes, and /v1/ in the path is the pragmatic default. For paging, reach for offset/limit when users need page numbers on small data, and cursor/keyset for big or public datasets because it uses the index and stays fast and stable no matter how deep we scroll. Filtering and sorting are just query params on the collection — always whitelist the fields we allow.

Networks are flaky. Requests time out, connections drop, clients retry. If a retried “charge the customer” request runs twice, someone gets billed twice — and that’s a very bad day. Idempotency is how we make retries safe. In simple language — an idempotent operation gives the same result whether we run it once or ten times.

Safe vs idempotent — two different words

People smush these together, but they mean different things.

• Safe — the request doesn’t change anything on the server. Pure reads. GET and HEAD are safe. We can hammer them all day and nothing on the server moves.
• Idempotent — the request can change things, but running it again lands in the same final state. Doing it twice is no worse than doing it once.

Every safe method is automatically idempotent (reading twice changes nothing), but not every idempotent method is safe (it can still write).

PUT /users/5 {name:"Manish"} is idempotent — sending it five times leaves user 5 named Manish, same as sending it once. POST /users {name:"Manish"} is not — five calls create five users.

Why this matters: retries + network failures

Here’s the nasty scenario. Our client sends POST /charges. The server processes it, charges the card, and starts sending back 200 OK… but the connection dies before the response arrives.

The client has no idea what happened. Did the charge go through? It has two bad options: retry (maybe double-charge) or give up (maybe never charge). This is the classic at-least-once delivery problem — the only way to guarantee a message arrives is to be willing to send it again, which means duplicates are inevitable.

For safe/idempotent methods, retrying is fine by definition. But POST — creating things, charging cards — is exactly where retries are dangerous. That’s the gap idempotency keys fill.

Idempotency keys for POST

The trick: the client generates a unique key (a UUID) for the operation and sends it in a header. The server remembers keys it has already processed and refuses to do the work twice.

POST /charges HTTP/1.1
Idempotency-Key: 4f8c-6b2a-9d1e-7c3f
Content-Type: application/json

{ "amount": 5000, "currency": "usd", "card": "tok_visa" }

Server logic, in plain words:

1. Look up the key. Never seen it? Do the work, store key → result, return the result.
2. Seen it before? Skip the work entirely, return the stored result from last time.

So the client can retry the same request as many times as it wants — same key, same single charge, same response.

This is exactly how Stripe’s payments API works, and it’s a very common system-design question. A few practical notes:

• The client owns the key. It must be the same across retries of the same logical operation, and different for genuinely new ones.
• Store keys with a TTL (say 24h) — we don’t keep them forever.
• Watch for the in-flight race. If two retries land at the same instant, a unique DB constraint on the key (or a lock) keeps only one from doing the work.

Making retries polite

On the client side, retries should be exponential backoff with jitter — wait 1s, then 2s, then 4s, plus a little randomness so a thousand clients don’t retry in lockstep and stampede our server. And only auto-retry idempotent operations (or POSTs guarded by an idempotency key). Retrying a bare POST blindly is how double-charges happen.

Interview soundbite

Safe = no server change (GET, HEAD); idempotent = repeating it lands the same final state (GET, PUT, DELETE). POST is neither, which is a problem because networks force at-least-once retries and duplicates are inevitable. The fix is an idempotency key: the client sends a unique key per operation, the server does the work once and replays the stored result on any retry — so a lost-response retry of a payment charges the card exactly once. Pair it with exponential backoff and jitter on the client.

“Should this be REST or GraphQL or gRPC?” is a question we’ll actually get asked at work, and definitely in interviews. In simple language — they’re three different styles of API, each great at a different job. There’s no “best”, only “best for this”.

Let’s take them one at a time, then compare.

REST — resources over HTTP

We covered REST already: resources with noun URLs, standard HTTP methods, stateless requests. It’s the default for a reason — simple, cacheable, works in any browser, understood by everyone.

Its main pain is data shape. Because each endpoint returns a fixed structure, we hit two problems:

• Over-fetching — the mobile app needs a user’s name, but GET /users/5 sends the whole fat object (address, preferences, timestamps). Wasted bytes.
• Under-fetching — to render one screen we need a user and their orders and each order’s items, so we make three or four round trips. This is the classic “N+1 requests” chattiness.

For a lot of APIs, that’s totally fine. But when clients have wildly different data needs, it bites.

GraphQL — the client picks the fields

GraphQL flips it around. There’s one endpoint (POST /graphql), and the client sends a query describing exactly the fields it wants — no more, no less.

query {
  user(id: 5) {
    name
    orders {
      total
      items { title }
    }
  }
}

One request, one round trip, exactly the fields asked for. Over-fetching and under-fetching basically vanish — that’s GraphQL’s superpower. It’s fantastic when many different clients (web, iOS, Android) each want a different slice of the same data.

The catch is on our side, the server. Because the client can ask for anything, a naive resolver for orders → items can fire a fresh database query per order — the dreaded N+1 query problem. We fix it with batching tools like DataLoader, but it’s real work. GraphQL is also harder to cache (everything’s a POST to one URL, so plain HTTP caching doesn’t apply) and needs a schema and query layer that REST just doesn’t.

gRPC — fast binary RPC

gRPC isn’t about resources at all — it’s RPC, “remote procedure call”. We define services and methods in a .proto file, and it feels like calling a local function that happens to run on another machine.

service UserService {
  rpc GetUser (GetUserRequest) returns (User);
}
message GetUserRequest { int32 id = 1; }
message User { int32 id = 1; string name = 2; }

Two things make it fast. It serializes data as Protocol Buffers — a compact binary format, much smaller than JSON — and it runs over HTTP/2, which multiplexes many calls over one connection and supports streaming (client-stream, server-stream, or full bidirectional). That combo makes it excellent for chatty service-to-service traffic inside our backend, and for real-time streams.

The trade-off: it’s binary, so we can’t just eyeball it in a browser or curl it easily, and browsers can’t call it directly (they need a grpc-web proxy). That’s why gRPC lives mostly inside the datacenter, between our own services, not as a public-facing API.

Side by side

When to pick each

• Reach for REST when we want a simple, public, cacheable API that any client can consume — CRUD over resources, third-party integrations, anything a browser or curl should hit easily. It’s the safe default.
• Reach for GraphQL when many different clients need different shapes of the same data, and we’re tired of building a dozen bespoke endpoints or watching mobile over-fetch. Great for rich frontends and aggregating several data sources.
• Reach for gRPC for internal service-to-service calls where speed and efficiency matter, or when we need streaming. Two microservices chattering millions of times a day love the compact binary + HTTP/2 combo.

Real systems mix them: gRPC between backend services, and a REST or GraphQL layer at the edge for the outside world.

Interview soundbite

REST is resources over HTTP — simple and cacheable, but clients over- and under-fetch. GraphQL gives one endpoint where the client picks exactly the fields it wants (killing over/under-fetch) at the cost of caching and N+1 resolver work on the server. gRPC is binary protobuf over HTTP/2 — fast, strongly typed, supports streaming, ideal for internal service-to-service traffic but not browser-friendly. Default to REST for public APIs, GraphQL for flexible clients, gRPC for internal high-throughput calls.

These two get mixed up constantly, and interviewers love catching us on it. In simple language — authentication answers “who are you?” and authorization answers “what are you allowed to do?”

Same prefix, totally different jobs. Authn always comes first: we can’t decide what someone’s allowed to do until we know who they are.

Authentication — proving who you are

Authentication (authn) is the process of verifying identity. We hand over some proof — a password, a token, a fingerprint — and the server checks it’s really us.

Think of it like showing your passport at the airport. The officer confirms the face matches the photo. That’s it — no decision yet about where you’re allowed to fly.

Common ways we authenticate:

• Something we know — password, PIN
• Something we have — a phone for OTP, a hardware key
• Something we are — fingerprint, face scan

Combine two of those and it’s MFA (multi-factor authentication).

Authorization — deciding what you can do

Authorization (authz) happens after we know who someone is. It’s the permission check — can this identity access this resource or perform this action?

Back to the airport: authentication was the passport check. Authorization is the boarding pass and the security guard checking you’re allowed into the business-class lounge. Same person, different gates.

Where each happens in a request

They run at different points in the request pipeline, and knowing the order matters.

401 vs 403 — the classic gotcha

The status codes trip people up, mostly because the names are misleading.

• 401 Unauthorized — we don’t know who you are. The name says “unauthorized” but it really means unauthenticated. Missing or invalid credentials. The fix is: log in.
• 403 Forbidden — we know exactly who you are, and you still can’t do this. Valid identity, insufficient permissions. Logging in again won’t help.

So the only difference is: 401 is “who are you?”, 403 is “not allowed”. If someone hits an admin route with a good token but a regular-user role, that’s a 403, not a 401.

// Express-style middleware showing the two gates
function authenticate(req, res, next) {
  const user = verifyToken(req.headers.authorization); // decode + check signature
  if (!user) return res.status(401).json({ error: "invalid token" }); // authn fail
  req.user = user;
  next();
}

function requireAdmin(req, res, next) {
  if (req.user.role !== "admin") return res.status(403).json({ error: "forbidden" }); // authz fail
  next();
}

RBAC vs ABAC — two ways to model permissions

Once we’re doing authorization, we need a model for who can do what. Two common ones.

RBAC (Role-Based Access Control) — we assign users roles (admin, editor, viewer), and roles carry permissions. Simple and easy to reason about. Most apps start here. The downside is roles explode when rules get granular (“editor, but only for their own team’s docs”).

ABAC (Attribute-Based Access Control) — decisions use attributes of the user, the resource, and the context. Rules like “allow if user.department == resource.department and time is business hours”. Way more flexible, more complex to build and audit.

The only difference in one line: RBAC checks what role you have, ABAC checks the attributes of you plus the thing you’re touching.

Interview soundbite

Authentication verifies identity (who you are), authorization decides permissions (what you can do), and authn always runs first. A failed authn is 401 (we don’t know you — go log in); a failed authz is 403 (we know you, you still can’t). For modeling permissions, RBAC assigns roles, ABAC evaluates attributes and context — RBAC is simpler, ABAC is more granular.

After someone logs in, how does the server remember them on the next request? HTTP is stateless — every request arrives with no memory of the last one. Two big approaches: server-side sessions and JWTs. This is a super common interview topic.

Session-cookie auth — server keeps the state

In simple language — the server remembers who’s logged in, and the browser just carries a ticket stub.

On login, the server creates a session (a record in memory, Redis, or a DB), generates a random session ID, and sends it back in a cookie. On every later request, the browser auto-sends that cookie, the server looks up the session, and knows who we are.

Set-Cookie: sid=8f3a...random...; HttpOnly; Secure; SameSite=Lax

The session ID is meaningless on its own — it’s just a lookup key. All the real data (user id, roles) lives on the server.

JWT — the token carries the state

A JWT (JSON Web Token) flips it around. Instead of storing state on the server, we pack the user info into the token itself, sign it, and hand it over. The server keeps nothing. That’s what “stateless” means.

On the next request, the client sends the token back (usually Authorization: Bearer ...). The server just verifies the signature — no database lookup needed. If the signature checks out, we trust the claims inside.

Think of it like the difference between a coat-check ticket (session ID — number means nothing without the server’s records) and a signed, tamper-proof concert wristband (JWT — everything’s printed right on it).

JWT structure — three parts, dot-separated

A JWT is just three Base64URL chunks glued with dots: header.payload.signature.

Two things people always get wrong: the payload is Base64, not encrypted — anyone can decode and read it, so never put secrets in there. And the signature doesn’t hide anything; it just proves the token wasn’t tampered with, because only the server knows the signing secret.

The revocation problem

Here’s the JWT catch interviewers dig into. Because the server keeps no state, it can’t easily un-issue a token. If a token is stolen, or a user is banned, that JWT stays valid until it expires on its own. There’s no session to delete.

Workarounds:

• Short-lived access tokens (5–15 min) + a long-lived refresh token to get new ones. Small blast radius.
• A denylist of revoked token IDs — but now we’re keeping server state again, which defeats the “stateless” selling point.

Sessions don’t have this problem: delete the session row and the user is instantly out.

Where to store the token

This is the security part, and it’s all about picking your poison.

• localStorage — easy, but readable by any JavaScript on the page. One XSS (cross-site scripting — attacker runs JS in our page) and the token is stolen.
• Cookie with HttpOnly — JS can’t read it, so XSS can’t grab it. But cookies auto-send on every request, which opens CSRF (cross-site request forgery — another site tricks the browser into sending our cookie). We defend that with the SameSite attribute and CSRF tokens.

So the trade-off in one line: localStorage dodges CSRF but is exposed to XSS; HttpOnly cookies dodge XSS but need CSRF protection. The common recommendation leans toward HttpOnly; Secure; SameSite cookies.

Interview soundbite

Sessions store state on the server and send a random session ID in a cookie — easy to revoke, needs a lookup. JWTs are stateless: signed claims live in the token (header.payload.signature, Base64 not encrypted), so the server just verifies the signature — scales flat but is hard to revoke before expiry. Fix revocation with short access tokens plus refresh tokens. Store tokens in HttpOnly cookies to dodge XSS, then add SameSite/CSRF tokens for CSRF.

“Sign in with Google” is OAuth. In simple language — OAuth 2.0 lets one app access some of our stuff on another service without ever seeing our password. It’s about delegated access, not login (that’s a common mix-up we’ll clear up at the end).

The problem OAuth solves

Say a photo-printing site wants our Google Photos. The bad old way: hand them our Google password. Now they can read our email, delete our account, everything — and we can’t take it back without changing the password.

OAuth fixes this. Instead of a password, the printing site gets a scoped, revocable token that says “read photos only”. No password shared, limited access, and we can revoke it anytime from our Google settings.

Think of it like a hotel key card. We don’t give the guest our master key — the front desk issues a card that only opens their room, only until checkout.

The four roles

Every OAuth flow has the same four players. Learn these names — interviewers ask directly.

• Resource Owner — us, the user who owns the data.
• Client — the app that wants access (the photo-printing site).
• Authorization Server — issues tokens after we approve (Google’s auth server).
• Resource Server — holds the data and honors the token (the Google Photos API).

Authorization Code flow (+ PKCE)

This is the main flow and the one to know cold. Here it is in plain steps.

Why the two-step dance (code, then token) instead of just handing over a token? The code comes back through the browser’s URL, which is a leaky place — logs, history, referer headers. The code alone is useless. The real token is fetched in a direct server-to-server call in step 4, out of the browser’s reach.

PKCE (Proof Key for Code Exchange, say “pixy”) plugs a hole for apps that can’t keep a secret — mobile and single-page apps. The client makes a random code_verifier, sends its hash (the code_challenge) in step 1, then reveals the original verifier in step 4. So even if someone steals the code, they can’t exchange it without the verifier. PKCE is now recommended for every client, not just public ones.

Access tokens vs refresh tokens

The token exchange usually gives us two.

• Access token — short-lived (minutes). Sent to the Resource Server on every call. If leaked, it expires fast.
• Refresh token — long-lived. Kept private by the client and used only to get a fresh access token when the old one expires — no need to bug the user again.

{
  "access_token": "ya29.a0Af...",
  "token_type": "Bearer",
  "expires_in": 3600,
  "refresh_token": "1//0gLo...",
  "scope": "photos.read"
}

The split is a blast-radius trade: the token flying around everywhere is short-lived, the powerful long-lived one stays hidden.

OIDC — the identity layer on top

Here’s the mix-up to clear: plain OAuth 2.0 is about authorization (access to resources), not logging you in. It never actually tells the client who you are.

OpenID Connect (OIDC) is a thin layer on top of OAuth that adds identity. Same Authorization Code flow, but the Auth Server also returns an ID token — a JWT with verified identity claims (sub, email, name). That’s what makes “Sign in with Google” a real login.

So the only difference in one line: OAuth gives an access token (what you can do), OIDC adds an ID token (who you are).

Interview soundbite

OAuth 2.0 grants delegated, scoped access without sharing a password. Four roles: resource owner, client, authorization server, resource server. The Authorization Code flow returns a short-lived code through the browser, then swaps it server-to-server for an access token — PKCE protects that exchange for public clients. Access tokens are short-lived, refresh tokens are long-lived and hidden. OAuth alone is authorization; OpenID Connect adds an ID token on top to give you authentication.

Someone signs up with a password. We need to check it again next time they log in — but we must never be able to read it back. This tension is the whole topic, and it’s a favourite interview question because so many people get it subtly wrong.

Rule zero — never store plaintext

If our database stores raw passwords and it leaks, every account is instantly owned. Worse, people reuse passwords, so we just handed the attacker their email and bank too. In simple language — we should never be able to see a user’s password, not even us.

So instead of storing the password, we store a one-way fingerprint of it.

Hashing vs encryption — one-way vs reversible

People mix these up constantly, so let’s nail it.

• Encryption is reversible. We scramble data with a key, and with that key we can unscramble it back. Good for data we need to read later (credit card numbers we charge again).
• Hashing is one-way. We run the input through a function that produces a fixed-size digest, and there’s no “un-hash”. Given the hash, we can’t get the password back — we can only hash a guess and compare.

Passwords want hashing, not encryption. We never need the original back; we only need to check “does this login attempt hash to the same thing we stored?” Encryption is the wrong tool here — a stolen key would reverse everything.

Why fast hashes (MD5, SHA-256) are bad here

MD5 and SHA-256 are cryptographic hashes, so surely they’re fine? No — and this trips people up. They’re built to be fast, which is exactly what we DON’T want for passwords.

An attacker with a leaked hash database just guesses. A modern GPU can compute billions of SHA-256 hashes per second. So they run every word in the dictionary, every common password, every leaked password from past breaches, hash each one, and compare. Fast hash = fast guessing.

There’s a second problem: fast hashes are deterministic and unsalted by default. md5("password123") is always the same string. So attackers precompute giant lookup tables — rainbow tables — mapping common hashes back to their password, and a “crack” becomes a single table lookup.

Salt — kills rainbow tables

A salt is a random value we generate per user and mix into the password before hashing. We store the salt right next to the hash (it’s not a secret).

Now hash("password123" + saltA) and hash("password123" + saltB) are completely different. So:

• Two users with the same password get different hashes — no more “oh, these 400 accounts share a hash”.
• Precomputed rainbow tables are useless, because the attacker didn’t know our random salt ahead of time.

The salt doesn’t need to be hidden. Its whole job is to be unique and unpredictable per user, forcing the attacker to crack every account separately instead of all at once.

Pepper — the salt’s secret cousin

A pepper is like a salt, but it’s a single secret value shared across all users and stored outside the database (in an env var or secret manager, not the DB). The only difference from salt is where it lives and that it’s secret.

So if just the database leaks but the app server’s pepper doesn’t, the attacker is still stuck — they’re missing an ingredient. It’s a nice defence-in-depth layer, but it’s optional; salt + a slow hash is the non-negotiable part.

Slow, adaptive hashes — bcrypt, scrypt, argon2

The real fix for “GPUs guess too fast” is to use a hash that’s deliberately slow. These functions bundle salting in and add a tunable work factor (also called cost or rounds) that controls how much CPU/memory each hash costs.

• bcrypt — battle-tested, has a cost factor (each +1 doubles the work). Safe default choice.
• scrypt — also memory-hard, so it resists GPU/ASIC attacks that have lots of cores but limited RAM.
• argon2 — the modern winner (Password Hashing Competition, 2015), tunable on time, memory, and parallelism. OWASP’s current top recommendation.

The work factor is the key idea. We tune it so one hash takes ~250ms on our server — barely noticeable at login, but it turns “billions per second” into “a handful per second” for the attacker. And it’s adaptive: as hardware gets faster, we just bump the factor up. Same code, more resistance.

Register vs login flow

The trick that surprises people: the salt lives inside the bcrypt hash string, so at login we don’t fetch it separately — compare reads it out for us and re-hashes the attempt with it.

const bcrypt = require("bcrypt");

// Register: cost factor 12, salt is auto-generated and baked into the result
const hash = await bcrypt.hash(plainPassword, 12);
// store `hash` in the DB, e.g. "$2b$12$Np3q...<salt+digest>"

// Login: bcrypt pulls the salt out of the stored hash and re-hashes the attempt
const ok = await bcrypt.compare(loginAttempt, hash);
if (!ok) throw new Error("invalid credentials"); // constant-time compare, no leak

Notice we never wrote a salt column ourselves and never compared strings with === — compare does a constant-time check so an attacker can’t time our response to guess characters.

Interview soundbite

Never store plaintext. Hashing is one-way (right for passwords); encryption is reversible (wrong here). Fast hashes like MD5/SHA are guessable at billions/sec and vulnerable to rainbow tables. Add a per-user salt to kill rainbow tables and an optional pepper (secret, kept out of the DB) for defence in depth. Use a slow adaptive hash — bcrypt/scrypt/argon2 — with a work factor tuned so each hash costs ~250ms, and bump it up as hardware improves.

Interviewers love asking “name a few web vulnerabilities and how you’d fix them”. The good news — most of them share one root cause: we trusted input we shouldn’t have. Let’s walk the classics. Each one gets a definition, why it happens, and the fix.

SQL Injection — attacker writes our query

In simple language — SQL injection is when user input sneaks out of the “data” slot and becomes part of the SQL command itself.

It happens whenever we build a query by gluing strings together. The database can’t tell where our intended query ends and the attacker’s text begins, so it just runs the whole thing.

// VULNERABLE — user input is concatenated straight into SQL
const email = req.body.email; // attacker sends:  ' OR '1'='1
db.query(`SELECT * FROM users WHERE email = '${email}'`);
// becomes:  SELECT * FROM users WHERE email = '' OR '1'='1'  → returns everyone

That ' OR '1'='1 closes our quote and adds a condition that’s always true. A nastier payload like '; DROP TABLE users; -- can delete data outright.

The fix is parameterized queries (prepared statements). We send the query and the values on separate channels, so the driver treats input as pure data — it can never become SQL.

-- FIXED — the ? / $1 is a placeholder, the value is bound separately
SELECT * FROM users WHERE email = $1;

// The value never touches the query string — it's bound as data
db.query("SELECT * FROM users WHERE email = $1", [req.body.email]);

An ORM does this for us under the hood, but the moment we drop to raw SQL, parameterize. Never concatenate. Never “just escape it myself”.

XSS — attacker runs JS in our page

Cross-Site Scripting (XSS) is when an attacker gets their JavaScript to run in our user’s browser. Once their script runs on our page, it can read cookies, steal tokens, or act as the user.

Same root cause as SQLi, different target: we took untrusted input and dropped it into HTML without escaping, so the browser executed it as code instead of showing it as text.

Two flavours interviewers want:

• Stored XSS — the payload is saved on the server (a comment, a profile bio) and served to everyone who views it. Most dangerous, it’s persistent.
• Reflected XSS — the payload rides in the request (a search query in the URL) and is echoed straight back in the response. Needs the victim to click a crafted link.

The fix is output escaping plus a Content-Security-Policy:

• Escape on output — turn <script> into <script> so the browser renders it as harmless text. Modern frameworks (React, Vue) auto-escape by default; the danger is when we bypass them with things like dangerouslySetInnerHTML or innerHTML.
• CSP header — a browser-enforced allowlist of where scripts may load from. Even if a payload slips through, Content-Security-Policy: script-src 'self' blocks inline and third-party scripts from running.

CSRF — attacker rides our logged-in session

Cross-Site Request Forgery (CSRF) tricks a logged-in user’s browser into firing a request we didn’t intend — like a hidden form on a malicious site that POSTs to ourbank.com/transfer. The only difference from other attacks: the attacker never sees the response, they just want the side effect to happen.

Why it works: cookies auto-send on every request to our domain, no matter who triggered it. So the browser happily attaches our valid session cookie to the attacker’s forged request, and the server thinks it’s us.

Fixes:

• CSRF tokens — the server plants a random unpredictable token in our form/page, and requires it back on state-changing requests. The attacker’s site can’t read our token (same-origin policy), so it can’t forge a valid request.
• SameSite cookie attribute — SameSite=Lax or Strict tells the browser “don’t send this cookie on cross-site requests”, which kills most CSRF at the source. This is the modern first line of defence.

Note the flip side: CSRF is a cookie-auth problem. Token-in-header auth (JWT in an Authorization header) isn’t auto-sent by the browser, so it sidesteps CSRF — but then it’s exposed to XSS instead. Pick your poison.

SSRF — the server makes the request for them

Server-Side Request Forgery (SSRF) — briefly — is when we let user input control a URL that our server then fetches. Think an “import from URL” or a webhook feature.

The attacker points it at internal stuff they can’t reach from outside: http://169.254.169.254/ (the cloud metadata endpoint that hands out credentials) or internal admin services on localhost. Our server, sitting inside the trusted network, fetches it for them.

Fix: validate and allowlist which hosts/schemes we’ll fetch, block private/internal IP ranges, and never blindly follow redirects to them.

Where these sit — OWASP Top 10

The OWASP Top 10 is the industry’s periodically-updated list of the most critical web risks — worth naming in an interview. Recent lists put Broken Access Control at #1 (users reaching data/actions they shouldn’t) and Injection (which folds in SQLi and XSS) near the top. We don’t need to memorize all ten; we need to show we know the categories and their fixes.

Interview soundbite

Most web vulns come from trusting input. SQLi: input becomes SQL → fix with parameterized queries, never string concat. XSS: input becomes runnable JS → escape output and add a CSP; stored (saved server-side) is worse than reflected (echoed from the request). CSRF: the browser auto-sends our cookie on a forged request → fix with CSRF tokens and SameSite. SSRF: user input steers a server-side fetch toward internal services → allowlist hosts. And know the OWASP Top 10 exists, with Broken Access Control and Injection near the top.

Three practical topics that show up in backend and system-design interviews. They’re unrelated but all live under “keeping our API safe and well-behaved”. Let’s take them one at a time.

Rate limiting — capping how often someone hits us

Rate limiting caps how many requests a client can make in a window of time. Why we do it — to stop abuse (brute-force login, scraping), protect against traffic spikes, and share capacity fairly. It’s the difference between one bad actor taking us down and one bad actor just getting throttled.

There are four algorithms interviewers ask about. The only real differences are how they count and how bursty they allow.

• Fixed window — count requests per fixed slot (e.g. 100 per minute). Simple, but has a nasty edge: someone can send 100 at 0:59 and 100 at 1:00 — 200 requests in two seconds — because the counter resets at the boundary.
• Sliding window — instead of a hard reset, the window slides with time, so it smooths out that boundary burst. More accurate, a bit more bookkeeping.
• Token bucket — a bucket refills with tokens at a steady rate; each request spends one. If the bucket has tokens, we allow a burst; when it’s empty, we reject. Great for “steady rate but tolerate short spikes”.
• Leaky bucket — think of requests dripping out of a bucket at a fixed rate (a queue). It smooths traffic into a constant outflow — no bursts allowed, unlike the token bucket. That’s the one difference to remember between the two buckets.

When we do throttle someone, we respond with HTTP 429 Too Many Requests, and politely tell them when to come back with a Retry-After header.

HTTP/1.1 429 Too Many Requests
Retry-After: 30
X-RateLimit-Remaining: 0

Token bucket, visualized

The tokens sitting in the bucket are what allow a burst — save up 10 tokens while idle, then fire 10 requests at once. That’s the token bucket’s whole personality.

CORS — the browser’s cross-origin gatekeeper

First, the thing CORS is built on: the same-origin policy. By default, a browser blocks JavaScript on site-a.com from reading responses from site-b.com. An origin is the triple of scheme + host + port — change any one and it’s a different origin. This stops a malicious page from quietly reading our logged-in bank data.

CORS (Cross-Origin Resource Sharing) is the controlled relaxation of that rule. It’s how a server says “actually, I allow site-a.com to call me”. In simple language — CORS isn’t a lock the server puts on itself; it’s the server handing the browser a permission slip.

Key point people miss: CORS is enforced by the browser, for browser JS. Server-to-server calls, curl, Postman — none of them care about CORS. So it’s not a server-side security control; it’s a browser safety feature.

The server grants access with response headers:

Access-Control-Allow-Origin: https://site-a.com
Access-Control-Allow-Methods: GET, POST
Access-Control-Allow-Headers: Authorization, Content-Type

Preflight — the browser asks first

For “non-simple” requests (a PUT/DELETE, or a custom header like Authorization), the browser sends a preflight — an automatic OPTIONS request that asks “am I allowed to do this?” before sending the real one. If the server’s response headers say yes, the real request goes; if not, the browser blocks it and we see that classic CORS error in the console.

OPTIONS /api/orders HTTP/1.1
Origin: https://site-a.com
Access-Control-Request-Method: DELETE

The fix for a CORS error is always server-side — configure the allowed origins/methods/headers. We can’t fix it from the frontend, and slapping Access-Control-Allow-Origin: * on everything is a lazy, unsafe reflex.

Secrets management — keys, tokens, DB passwords

Secrets are the credentials our app needs but must never expose — DB passwords, API keys, signing keys. The whole game is keeping them out of places that leak.

• Never commit secrets to git. Once pushed, they’re in history forever (and scraped by bots within minutes). Use a .gitignore’d .env file locally and commit a .env.example with blank values instead.
• Environment variables are the baseline — the classic 12-factor approach. Config comes from the environment, not code, so the same image runs in dev and prod with different secrets injected.
• Secret managers (AWS Secrets Manager, HashiCorp Vault, Doppler) are the step up. Env vars are fine but they’re plaintext, visible to anything in the process, and awkward to rotate. Secret managers add encryption at rest, access control (who/what can read which secret), and audit logs.
• Rotation — secrets should be rotated (changed) on a schedule and immediately if one leaks. This is exactly where env vars hurt: rotating means a redeploy, while a secret manager can rotate and distribute the new value without touching the app.

# Local: keep real values in .env (gitignored), commit only the template
echo ".env" >> .gitignore
# .env.example (committed) shows the shape, not the secrets:
#   DATABASE_URL=
#   JWT_SECRET=

The one-line rule: secrets live in the environment or a manager, never in the code or the repo, and we should be able to rotate any of them without a scramble.

Interview soundbite

Rate limiting caps request frequency — fixed window (simple, boundary-burst bug), sliding window (smoother), token bucket (allows bursts via saved tokens), leaky bucket (constant drain, no bursts); throttle with 429 + Retry-After. CORS relaxes the browser’s same-origin policy so a server can allow specific cross-origin JS; browser-enforced only, “non-simple” requests trigger an OPTIONS preflight, and it’s always fixed server-side. Secrets go in env vars or a secret manager — never in git — with rotation, access control, and a committed .env.example template.

A SQL database stores data in tables with a fixed structure and lets us query it with SQL. A NoSQL database is basically everything else — anything that doesn’t use the classic table-and-rows model.

In simple language — SQL is a spreadsheet with strict columns; NoSQL is a big flexible bag where each item can look a little different.

This is one of the most common system-design warmups. Interviewers don’t want a religious war — they want to hear us pick the right tool for a workload and explain the trade-off.

The relational (SQL) model

SQL databases — Postgres, MySQL, SQL Server — store data in tables (rows and columns). A few core ideas:

• Schema — we define the columns and their types up front. Every row must fit.
• Joins — we split data across tables and stitch them back together at query time. One users table, one orders table, joined on user_id. No duplication.
• ACID — transactions are atomic, consistent, isolated, durable. In plain words: money doesn’t vanish mid-transfer. (We cover ACID in depth in its own note.)

The big win is data integrity. The schema and foreign keys stop us from writing garbage. If a query needs data from five tables, SQL is genuinely great at it.

-- one order, but user + product data pulled in via joins
SELECT u.name, p.title, o.quantity
FROM orders o
JOIN users u    ON u.id = o.user_id
JOIN products p ON p.id = o.product_id
WHERE o.id = 42;

The NoSQL families

“NoSQL” is an umbrella, not one thing. There are four families, and knowing which is which is the interview gold.

The document family is the one people usually mean by “NoSQL”. Instead of joining tables, we store the whole thing as one document — user plus their orders nested inside.

// a document — everything about the order in one place, no joins
{
  _id: 42,
  user: { name: "Asha", email: "asha@x.com" },
  items: [{ title: "Keyboard", qty: 1 }],
  total: 4999
}

Schema-on-write vs schema-on-read

This is the real philosophical difference, and a great line to drop in an interview.

• Schema-on-write (SQL) — the database checks our data against the schema when we write it. Bad shape? Rejected. Structure is guaranteed on the way in.
• Schema-on-read (NoSQL) — we write whatever we want, and the reading code figures out the shape. Flexible, but now the burden of “is this field even here?” lives in our application.

Think of it like — SQL is a bouncer checking IDs at the door; NoSQL lets everyone in and sorts it out at the bar.

Scaling differences

• SQL scales vertically — mostly a bigger box (more CPU, RAM). Horizontal scaling (sharding across machines) is possible but painful, because joins and transactions across nodes are hard.
• NoSQL scales horizontally — built from day one to spread data across many cheap machines. That’s often the whole point. The catch: to get that, most NoSQL stores relax consistency (eventual consistency — reads might briefly see stale data).

When to use which

• Reach for SQL when data is relational, integrity matters, and we run complex queries — payments, orders, anything financial. Default here unless we have a reason not to.
• Reach for NoSQL when the shape is fluid, we need massive write throughput or horizontal scale, or the access pattern is dead simple (key → value) — caches, event firehoses, flexible catalogs.

Honestly, most systems end up using both: Postgres for the source of truth, Redis for caching, maybe Elasticsearch for search.

Interview soundbite

SQL = tables, fixed schema, joins, ACID, schema-on-write, scales up. NoSQL = flexible model in four flavors (document, key-value, wide-column, graph), schema-on-read, scales out with eventual consistency. Default to SQL for relational data with integrity needs; pick NoSQL for scale, flexibility, or simple key-based access — and most real systems mix both.

An index is a separate, sorted data structure the database keeps so it can find rows fast — without scanning the whole table.

In simple language — it’s the index at the back of a book. Want the page about “goroutines”? We don’t read all 400 pages; we jump to the “G” section and follow the pointer.

That’s the whole idea. Without it, the database does a full table scan — reads every single row to find the ones we want.

Why it matters — and what it costs

The why is speed. On a million-row table, a WHERE email = ... with no index reads a million rows. With an index it reads maybe 3–4. That’s the difference between 800ms and 0.2ms.

But there’s no free lunch. An index is extra data the database must keep in sync. So:

• Reads get faster.
• Writes get slower — every INSERT, UPDATE, DELETE now also has to update the index.
• Disk usage grows — the index is a real structure taking real space.

Think of it like a book index — great for the reader, but every time we add a paragraph, someone has to update the index too.

How a B-tree index works

Most indexes are B-tree (balanced tree) indexes. Picture a tree of sorted values. We start at the top, and at each level we go left or right based on comparison, narrowing the range until we hit the row’s pointer.

Because the tree is sorted, B-trees are great for equality (=), ranges (>, <, BETWEEN), and ORDER BY. They’re useless for things like WHERE email LIKE '%son' — a leading wildcard can’t use the sorted order.

Composite indexes and column order

A composite index covers multiple columns, e.g. (last_name, first_name). The catch that trips everyone up is the leftmost prefix rule: the index can only be used from the left.

An index on (a, b, c) helps queries filtering on a, or a + b, or a + b + c — but not b alone, and not c alone. It’s like a phone book sorted by last name then first name: great for finding “Sharma, Asha”, useless for finding everyone named “Asha”.

So column order is a design decision. Put the column we filter on most — usually the equality column — first.

Covering index

A covering index is one that contains every column the query needs. When that happens, the database answers straight from the index and never touches the table at all — an “index-only scan”. Fastest thing going.

-- if we index (user_id, status), this query is fully covered:
-- it only needs user_id (filter) and status (returned)
CREATE INDEX idx_orders_user_status ON orders (user_id, status);
SELECT status FROM orders WHERE user_id = 42;

When NOT to index

Indexes aren’t free, so don’t sprinkle them everywhere:

• Low-cardinality columns — a gender or is_active boolean has few distinct values; the index barely narrows anything.
• Small tables — a full scan is already instant.
• Write-heavy tables with rarely-queried columns — we’d just pay the write tax for nothing.
• Columns we never filter, join, or sort on.

Reading EXPLAIN

EXPLAIN shows the plan the database will use. This is how we prove an index is actually being hit.

EXPLAIN SELECT * FROM users WHERE email = 'asha@x.com';

-- Bad  → Seq Scan on users   (full scan, no index used)
-- Good → Index Scan using idx_users_email on users

If we see Seq Scan on a big table where we expected an index, something’s off — wrong column order, a function wrapping the column, or a type mismatch.

Interview soundbite

An index is a sorted side-structure (usually a B-tree) that turns O(n) scans into O(log n) lookups — faster reads at the cost of slower writes and more disk. Composite indexes obey the leftmost-prefix rule, so column order matters; a covering index answers the query without touching the table. Don’t index low-cardinality or tiny tables, and always confirm with EXPLAIN that the plan says Index Scan, not Seq Scan.

A transaction is a group of database operations that succeed or fail together, as one unit. Either all of them happen, or none of them do.

In simple language — it’s an “all or nothing” wrapper around a bunch of statements. No half-done state.

The classic example: transfer ₹1000 from Asha to Rohan. That’s two writes — subtract from Asha, add to Rohan. If the power dies between them, Asha lost money that never reached Rohan. A transaction makes sure that can never happen.

The WHY — ACID

The four guarantees a transaction gives us spell ACID. This is the interview bread and butter, so let’s nail each one in plain words, using the bank transfer throughout.

Let’s unpack each with the transfer in mind:

• Atomicity — the “all or nothing” bit. Subtract-from-Asha and add-to-Rohan are one indivisible action. If step two fails, step one is undone.
• Consistency — the total across both accounts is the same before and after. Constraints like “balance can’t go negative” are never violated by a committed transaction.
• Isolation — if two transfers run at once, each behaves as if it’s alone. Nobody sees another transaction’s uncommitted, half-written state. (How much isolation is a dial — that’s a whole topic on its own.)
• Durability — the moment we get “committed”, it’s written to durable storage. A crash one millisecond later can’t lose it.

The HOW — COMMIT and ROLLBACK

We open a transaction, run our statements, then either COMMIT (make it permanent) or ROLLBACK (throw it all away). Until we commit, nothing is visible to the outside world.

BEGIN;                                              -- start the transaction

UPDATE accounts SET balance = balance - 1000 WHERE id = 'asha';
UPDATE accounts SET balance = balance + 1000 WHERE id = 'rohan';

COMMIT;   -- both writes become permanent, together
-- if anything had failed above, we'd run ROLLBACK instead and lose both

And the failure path — if a check fails, we bail and nothing sticks:

BEGIN;
UPDATE accounts SET balance = balance - 1000 WHERE id = 'asha';
-- oops, Asha's balance would go negative — abort the whole thing
ROLLBACK;   -- Asha's 1000 is restored, as if nothing happened

In application code it’s usually a try/catch: commit on success, rollback in the catch.

await client.query("BEGIN");
try {
  await client.query("UPDATE accounts SET balance = balance - 1000 WHERE id = 'asha'");
  await client.query("UPDATE accounts SET balance = balance + 1000 WHERE id = 'rohan'");
  await client.query("COMMIT");   // success — make it stick
} catch (err) {
  await client.query("ROLLBACK"); // any failure — undo everything
  throw err;
}

One gotcha — auto-commit

By default, most databases run in auto-commit mode: each lone statement is its own tiny transaction that commits immediately. That’s fine for single writes, but the moment two writes must move together, we have to wrap them in an explicit BEGIN ... COMMIT. Forgetting this is a real production bug, not just an interview one.

Interview soundbite

A transaction bundles multiple operations into one all-or-nothing unit, guaranteed by ACID: Atomicity (all or nothing), Consistency (constraints stay valid), Isolation (concurrent transactions don’t see each other’s half-done work), Durability (committed data survives crashes). We wrap the work in BEGIN, then COMMIT to persist or ROLLBACK to undo — the classic example being a bank transfer where both the debit and credit must land together or not at all.

An isolation level is a dial that controls how much one running transaction can see of another running transaction’s work. Turn it up and transactions feel more alone; turn it down and they run faster but can trip over each other.

In simple language — it’s the setting for “how strictly do we keep concurrent transactions from stepping on each other”.

The “I” in ACID (Isolation) isn’t all-or-nothing — it’s a spectrum. To understand the levels, we first need the bad things they’re protecting us from.

The three anomalies

When transactions overlap, three classic weirdnesses can happen. All of them come down to reading data that’s changing under our feet.

• Dirty read — we read another transaction’s uncommitted change. It might roll back, and now we acted on data that never really existed. The worst one.
• Non-repeatable read — we read a row, someone else commits an UPDATE to it, we read the same row again in the same transaction and get a different value. The row changed mid-transaction.
• Phantom read — we run a range query (WHERE age > 30), someone commits a new row that matches, we run it again and get an extra row that wasn’t there before. The set of rows changed.

Think of it like re-reading a paragraph in a book someone else is editing: dirty = you read a pencilled note they later erase; non-repeatable = a word you read got swapped; phantom = a whole new sentence appeared.

The four isolation levels

The SQL standard defines four levels. Each one prevents more anomalies than the last — and costs a bit more performance. Here’s the map to memorize.

  <div style="border-top: 1px solid var(--color-border); padding-top: 6px;">Read Uncommitted</div>
  <div style="border-top: 1px solid var(--color-border); padding-top: 6px; text-align: center; color: var(--color-tag-advanced);">yes</div>
  <div style="border-top: 1px solid var(--color-border); padding-top: 6px; text-align: center; color: var(--color-tag-advanced);">yes</div>
  <div style="border-top: 1px solid var(--color-border); padding-top: 6px; text-align: center; color: var(--color-tag-advanced);">yes</div>

  <div>Read Committed</div>
  <div style="text-align: center; color: var(--color-accent);">no</div>
  <div style="text-align: center; color: var(--color-tag-advanced);">yes</div>
  <div style="text-align: center; color: var(--color-tag-advanced);">yes</div>

  <div>Repeatable Read</div>
  <div style="text-align: center; color: var(--color-accent);">no</div>
  <div style="text-align: center; color: var(--color-accent);">no</div>
  <div style="text-align: center; color: var(--color-tag-intermediate);">maybe*</div>

  <div>Serializable</div>
  <div style="text-align: center; color: var(--color-accent);">no</div>
  <div style="text-align: center; color: var(--color-accent);">no</div>
  <div style="text-align: center; color: var(--color-accent);">no</div>
</div>
<div style="color: var(--color-text-muted); margin-top: 8px;">* The standard allows phantoms at Repeatable Read; Postgres (via MVCC) actually blocks them here.</div>

-- set the level explicitly for one transaction
BEGIN TRANSACTION ISOLATION LEVEL REPEATABLE READ;
SELECT balance FROM accounts WHERE id = 'asha';
-- ... same row read again here returns the SAME value, guaranteed
COMMIT;

The N+1 problem is when we run 1 query to fetch a list, then N more queries — one for each item in that list — to fetch related data. So a list of 100 posts becomes 101 database round-trips.

In simple language — instead of asking the database once, we ask it in a loop, hammering it once per row.

It’s the single most common performance bug in ORM-heavy backends, and interviewers love it because the fix shows we actually understand what our ORM is doing under the hood.

Why it happens

It almost always sneaks in through an ORM (a library that maps objects to rows — think Prisma, Sequelize, ActiveRecord). The code looks totally innocent:

const posts = await db.query("SELECT * FROM posts LIMIT 100");  // 1 query
for (const post of posts) {
  // this fires a fresh query EVERY loop iteration → 100 queries
  post.author = await db.query("SELECT * FROM users WHERE id = $1", [post.userId]);
}
// total: 1 + 100 = 101 queries

Each .author access looks like a property, but it’s secretly a database call. One list of 100 posts → 101 queries. On a real page that’s the difference between 20ms and 2 seconds.

The fixes

The core idea is always the same: stop looping, start batching. Ask for everything in one (or two) queries.

1. JOIN — let the database do the stitching. One query brings back posts with their authors.

SELECT posts.*, users.name AS author_name
FROM posts
JOIN users ON users.id = posts.user_id
LIMIT 100;   -- one round-trip, done

2. Eager loading — the ORM way to say “fetch the relation up front, not lazily”. Most ORMs expose this as include, with, or preload. Under the hood it’s usually one JOIN or one follow-up WHERE id IN (...) — either way, not a loop.

// Prisma-style: one extra IN-query instead of 100
const posts = await db.post.findMany({ include: { author: true }, take: 100 });

3. Batching / DataLoader — the go-to for GraphQL. A DataLoader collects all the individual .author requests fired during one tick, dedupes them, and fires a single WHERE id IN (1,2,3,...). It turns N calls into one behind the scenes, so our code can stay simple.

-- what batching collapses the N author lookups into:
SELECT * FROM users WHERE id IN (7, 12, 12, 30, 41);  -- deduped keys

Other quick wins

The N+1 fix is the headline, but a few habits keep queries fast in general:

• Select only the columns we need — SELECT id, name beats SELECT *. Less data over the wire, and it opens the door to covering indexes.
• Avoid SELECT * — it breaks covering indexes and drags along big columns (blobs, JSON) we didn’t want.
• Index the columns we filter and join on — an unindexed JOIN or WHERE is a full scan hiding in plain sight.
• Paginate — LIMIT/OFFSET or keyset pagination. Never fetch 100k rows to show 20.
• Run EXPLAIN — it’s the truth serum. It shows whether a query is a fast index scan or a slow sequential scan, and whether a JOIN is doing something silly.

Practical takeaway

If a page feels slow, the first thing to check is the query count in the logs. Seeing “1 + N” repeated queries is the smoking gun for N+1 — fix it with a JOIN, eager loading, or a DataLoader. Then tidy up the basics: select only needed columns, index the filter/join columns, paginate, and let EXPLAIN confirm the plan. Turning 101 round-trips into 1 is often the biggest single win we’ll ever ship.

Two everyday-backend topics that share nothing except being about databases at scale: connection pooling (reusing DB connections) and migrations (versioning our schema changes). Both come up constantly once we run real services.

Why connections are expensive

Opening a database connection isn’t cheap. Each one means a TCP handshake, then TLS negotiation, then authentication, then the database forks or assigns a backend process/thread and allocates memory for it.

In simple language — opening a connection is like hiring and onboarding a new employee. Doing it fresh for every single query would be madness.

So if every web request opened its own connection, ran one query, and closed it, we’d waste most of our time on setup and teardown. Worse, databases cap how many connections they’ll accept (Postgres defaults to ~100). A traffic spike could open thousands and knock the database over.

What a connection pool is

A connection pool is a fixed set of already-open connections that we keep around and hand out on demand. A request borrows one, runs its query, and returns it to the pool instead of closing it. The next request reuses the same warm connection.

Think of it like a fleet of taxis idling at a stand — riders hop in and out, but the cars never go home. No re-hiring per trip.

// one pool for the whole app — created once, reused forever
const pool = new Pool({ max: 10 });          // never opens > 10 connections
const { rows } = await pool.query("SELECT 1"); // borrows + returns automatically

Pool sizing gotchas

Bigger is not better. The classic mistake is setting max to some huge number “to be safe”.

• Every app instance has its own pool. Run 10 app pods with max: 20 and that’s 200 connections aimed at the database — easily past its limit.
• A pool bigger than the database can handle just moves the queue from the pool to the database, which handles contention worse.
• A common starting point is small — often around (cores * 2) per instance — then tune with real metrics.
• Always set a connection timeout so a request fails fast instead of hanging forever when the pool is drained.

Migrations — versioning the schema

A migration is a versioned script that changes the database schema — add a table, add a column, create an index. We check them into git alongside the code, so the schema evolves in lockstep with the app.

In simple language — migrations are “git for our database structure”. Every change is a numbered step everyone runs in the same order.

Key properties:

• Versioned & ordered — each migration has a number/timestamp; they run in sequence. The database records which ones it’s already applied, so re-running is safe (idempotent at the runner level).
• Forward-only in production — we roll forward with a new migration to fix a mistake, rather than un-applying old ones on a live system.
• Up / down — an up applies the change, a down reverses it. down is mostly a dev-time safety net; in prod we prefer a new forward migration.

Tools that do this: Flyway and Liquibase (JVM world), knex/Prisma Migrate (Node), Alembic (Python), Rails migrations. They all share the same idea — a folder of ordered scripts plus a tracking table.

-- 0007_add_last_login.up.sql
ALTER TABLE users ADD COLUMN last_login timestamptz;

-- 0007_add_last_login.down.sql
ALTER TABLE users DROP COLUMN last_login;

Zero-downtime migrations — expand/contract

Here’s the senior-level bit. If we deploy a schema change and the app change at the same instant, there’s a window where old code hits the new schema (or vice versa) and things break. The fix is the expand-contract (a.k.a. parallel-change) pattern — never break compatibility in a single step.

Say we’re renaming name to full_name:

1. Expand — add the new full_name column. Old code ignores it, still works.
2. Migrate — deploy code that writes to both columns and backfill existing rows.
3. Switch — deploy code that reads from full_name.
4. Contract — once nothing touches name, drop it in a later migration.

Each step is backward-compatible, so at no point are the running app and the schema out of sync. The golden rule: additive changes first, destructive changes last, and never in the same deploy.

Interview soundbite

Connections are expensive (TCP + TLS + auth + a server-side process), so we keep a connection pool of warm connections and borrow/return them per request — sized small and tuned, because every app instance has its own pool and databases cap total connections. Migrations are versioned, ordered, forward-only schema scripts (Flyway, Liquibase, knex) with up/down steps, checked into git. For zero downtime we use expand-contract: add the new thing, dual-write and backfill, switch reads, then drop the old thing in a later deploy — additive first, destructive last, never in one step.

A cache is a fast copy of expensive-to-get data, kept close to whoever needs it. That’s the whole idea.

In simple language — instead of asking the slow database the same question again and again, we save the answer somewhere fast and hand out that copy.

Why we cache

Three reasons, and honestly every interview answer boils down to these:

• Latency — reading from memory is nanoseconds; hitting a database across the network is milliseconds. A cache turns a slow trip into a fast one.
• Load — every request the cache answers is a request the database never sees. We shield the expensive backend from getting hammered.
• Cost — fewer queries, fewer compute cycles, less bandwidth. Serving a cached byte is dirt cheap compared to recomputing it.

Think of it like keeping a snack in your desk drawer instead of walking to the shop every time you’re hungry. Same snack, way less effort.

Where caching happens

Caching isn’t one thing in one place. It happens at every hop between the user and our data. A request can be answered — and stop travelling — at any of these layers.

The higher up the stack we answer, the faster and cheaper it is. The browser cache is the fastest possible cache — the request doesn’t even leave the device.

Cache hit vs miss

Two words that come up in every caching conversation.

• Cache hit — the data is in the cache. We return it straight away. Fast, cheap, done.
• Cache miss — the data isn’t there. We go fetch it from the slower source, usually store a copy on the way back, then return it.

The number we care about is the hit ratio — hits divided by total lookups. A cache with a 95% hit ratio means only 1 in 20 requests actually bothers the database. A cache with a 5% hit ratio is just overhead — we pay to check it and almost always miss anyway.

// The shape of nearly every cache read
let value = cache.get(key);   // check the fast layer first
if (value === undefined) {    // MISS
  value = db.query(key);      // pay the slow cost once
  cache.set(key, value);      // remember it for next time
}
return value;                 // HIT on every future call

The one trade-off: staleness

Here’s the catch. A cache holds a copy, and copies drift out of date. The moment the real data changes, our cached copy is a lie until we refresh it. That gap is staleness.

Every caching decision is really one question: how stale can we afford to be? A product price might need to be fresh within seconds. A user’s avatar can be minutes or hours old and nobody cares. We tune the cache lifetime around that tolerance — short lifetimes stay fresh but miss more, long lifetimes hit more but risk serving old data.

There’s an old joke that the two hardest problems in computer science are cache invalidation and naming things. It’s a joke because keeping copies in sync is genuinely hard — we cover the strategies for it in the next notes.

Practical takeaway

A cache is a fast copy of expensive data, and we place copies at every layer from the browser down to the database. We win on latency, load, and cost — we pay in staleness. The whole art of caching is deciding how fresh the copy needs to be, and which layer is the cheapest place to answer from.

A caching strategy is just the rule for who talks to the cache and when. The tricky part is never the reads — it’s keeping the cache and the database agreed on what’s true. Let’s walk the five patterns interviewers actually ask about.

Cache-aside (lazy loading)

The most common one by far. In simple language — the application manages the cache directly. The cache doesn’t know the database exists.

On a read: check the cache first. Hit? Return it. Miss? Load from the DB, stash a copy in the cache, return it. We only cache data that’s actually been asked for — hence “lazy loading”.

async function getUser(id) {
  const cached = await cache.get(`user:${id}`);
  if (cached) return JSON.parse(cached);      // HIT

  const user = await db.users.findById(id);   // MISS → source of truth
  await cache.set(`user:${id}`, JSON.stringify(user), "EX", 300); // 5 min TTL
  return user;
}

When to use: read-heavy workloads where the app is happy owning the logic. Downside: the first read of anything is always a miss (a “cold cache”), and our code has to remember to keep the cache in sync on writes.

Read-through

Same read behaviour as cache-aside, but the cache does the DB fetch on a miss, not our app. We only ever talk to the cache; it’s the cache’s job to fill itself from the backing store.

The only difference from cache-aside is who loads on a miss — the cache library instead of our code. It keeps the app simpler, but we need a cache that supports it (a provider or a loader function).

Write-through

Now the writes. Write-through means every write goes to the cache and the database together, synchronously, before we tell the caller “done”.

The cache is always fresh — it can never be more stale than the DB, because they’re updated in the same breath. The price is a slower write: we pay two writes on the critical path.

Write-behind (write-back)

Same idea, but we cheat on timing. The write hits the cache and returns immediately. The cache flushes to the database later — batched, async, in the background.

Writes feel instant and the DB sees far fewer, bigger writes. The danger is obvious: if the cache dies before it flushes, those writes are gone. We trade durability for speed.

Write-around

The odd one out. On a write, we skip the cache entirely and go straight to the database. The cache only fills up later, on a read miss (cache-aside style).

Why bother? Because sometimes we write data that won’t be read again soon — logs, bulk imports, one-off records. Writing it into the cache would just evict genuinely-hot data for nothing. Write-around keeps the cache full of stuff people actually read.

When to use: write-heavy data with low re-read rates. Downside: a just-written record is a guaranteed miss if someone reads it right away.

Interview soundbite

Reads: cache-aside (app fills the cache) and read-through (cache fills itself) — same behaviour, different owner. Writes: write-through is safe but slow (cache + DB together), write-behind is fast but risky (cache now, DB later), and write-around skips the cache on writes to avoid polluting it with data nobody reads. Most systems run cache-aside for reads plus write-through or write-around for writes.

There’s a famous line: “There are only two hard things in computer science — cache invalidation and naming things.” It’s funny because it’s true. Getting stale copies out of the cache at the right moment is genuinely one of the hardest problems we deal with. Let’s break it into the two separate jobs people usually mix up.

• Invalidation — removing data because it’s wrong now (the source changed).
• Eviction — removing data because we’re out of room (memory is full).

Different problems, different tools.

Invalidation: keeping copies honest

TTL / expiry

The simplest approach. We stamp every cache entry with a time-to-live and the cache auto-deletes it when the clock runs out. No cleanup code, no coordination — the entry just expires.

// "trust this copy for 60 seconds, then forget it"
await cache.set("product:42", data, "EX", 60);

TTL is a bet on staleness: we’re saying “being up to 60 seconds out of date is fine here.” Short TTL = fresher but more misses. Long TTL = more hits but staler data. It’s the same trade-off we always tune.

Active invalidation on write

TTL alone means we serve stale data until it expires. When we can’t tolerate that, we actively kill the entry the moment the underlying data changes — usually right after the DB write.

async function updatePrice(id, price) {
  await db.products.update(id, { price });  // source of truth first
  await cache.del(`product:${id}`);         // then evict the stale copy
}

Delete, don’t overwrite. Deleting lets the next read repopulate from the fresh DB (cache-aside), which sidesteps a whole class of race conditions where two writers overwrite each other’s cached value in the wrong order.

Eviction: making room

TTL removes stale things. Eviction removes things when memory fills up, even if they’re still valid. The cache has to pick a victim — and the rule it uses is the eviction policy.

LRU is the go-to. It leans on the idea that data used recently will probably be used again soon (temporal locality), and it’s cheap to approximate. Reach for LFU when some keys are always hot and shouldn’t be evicted just because they went quiet for a moment. FIFO and random are mostly there when we want dead-simple bookkeeping. In Redis this is the maxmemory-policy setting (allkeys-lru, allkeys-lfu, and friends).

The cache stampede (thundering herd)

Here’s the failure that takes down real systems. A super-popular key expires. In that instant, every request that wanted it misses at the same time — and they all pile onto the database together to recompute it. The DB, sized for a trickle of misses, gets a flood. It falls over, which slows everything, which causes more misses. A stampede.

Two fixes we should be able to name:

• Locking (single-flight) — the first request to miss grabs a lock and does the recompute; everyone else waits for that one result instead of all hitting the DB. Only one trip to the database, no matter how many callers.
• Jittered TTL — instead of a fixed 60s, we set 60s ± a random few seconds. Now keys expire at slightly different times, so misses spread out instead of all landing together. Cheap and effective.

A third trick worth a mention: early/probabilistic refresh — recompute a hot key a little before it expires, in the background, so it never actually goes cold.

Interview soundbite

Invalidation removes wrong data (TTL expiry, or an active del on write — delete rather than overwrite to dodge races). Eviction removes data when memory is full, driven by a policy: LRU (default, evict least-recently-used), LFU (least-frequently-used), FIFO, or random. And watch for the cache stampede — a hot key expiring sends every request to the DB at once; fix it with a lock (single-flight) or a jittered TTL so expirations don’t sync up.

In theory caching is one idea; in practice we reach for two specific tools over and over. A CDN for caching content close to users, and Redis for caching data close to our app. Let’s make both concrete.

CDN — caching at the edge

A CDN (Content Delivery Network) is a fleet of servers spread across the globe that cache copies of our content near the people requesting it. In simple language — instead of every user in Tokyo fetching an image from our server in Virginia, the CDN keeps a copy on a machine in Tokyo and serves it from there.

The win is distance. Bytes travel at a fixed speed, so a request that stays in-city is dramatically faster than one crossing an ocean. Those nearby servers are called edge locations (or PoPs, points of presence).

What does the CDN cache? Mostly static assets — images, JS, CSS, videos, fonts — anything identical for every user. We control it with Cache-Control headers on our responses. This is the shared language between our origin and the CDN (and the browser):

Cache-Control: public, max-age=31536000, immutable

public means any cache may store it, max-age=31536000 says “fresh for a year,” and immutable promises it’ll never change (so don’t even revalidate). That combo is how we cache hashed asset files like app.9f2c.js forever — a new build just gets a new hash, so a new URL.

Redis — caching in memory

Redis is an in-memory key-value store. We hand it a key, it hands back a value, and it lives entirely in RAM — which is why it answers in well under a millisecond. It’s the default choice for an application cache and for session storage.

Why is it so fast? Everything is in memory (no disk seek on the read path), it’s single-threaded for commands (no lock contention), and it speaks a tiny, efficient protocol. The catch is that RAM is smaller and pricier than disk, so we cache the hot subset, not everything.

The GET/SET basics look exactly like you’d hope:

SET user:42 "Manish" EX 300   # store, expire in 300 seconds
GET user:42                    # -> "Manish"  (a HIT)
TTL user:42                    # -> 287        (seconds left)
DEL user:42                    # invalidate on write

Redis isn’t just strings, either — that’s the other reason we love it. It ships real data types: hashes (objects), lists (queues), sets and sorted sets (leaderboards, rate limiters), and more. So a session or a “top 10 today” list is a native structure, not something we serialize by hand.

On persistence: Redis is memory-first but can save to disk so it survives a restart — RDB takes periodic point-in-time snapshots, AOF logs every write for finer-grained recovery. For a pure cache we often don’t even need it; for a session store we usually do.

CDN vs Redis — which one?

They cache different things at different layers, so it’s rarely either/or — most stacks run both. But the distinction matters in interviews:

Rule of thumb: CDN for content, Redis for data. If it’s the same file for every visitor and travels far, push it to the edge. If it’s computed per request and we just don’t want to recompute it, keep it in Redis next to the app.

Interview soundbite

A CDN caches static assets at edge locations near users to kill network latency, driven by Cache-Control headers — great for images, JS, CSS. Redis is an in-memory key-value store (with real data types and optional RDB/AOF persistence) used as the application cache and session store — sub-millisecond because it’s all in RAM. CDN for content at the edge, Redis for data at the app; big systems use both.

Our app is slow because too many people are using it. We have two ways to fix that. In simple language — vertical scaling means we buy a bigger machine, and horizontal scaling means we buy more machines. That’s the whole idea. Everything else is trade-offs.

Vertical scaling — scale UP

Vertical scaling (scale up) is throwing more power at a single box. More CPU cores, more RAM, faster disk. The app doesn’t change at all — it just runs on beefier hardware.

Why we love it: it’s dead simple. No code changes, no distributed-system headaches. One database, one server, one thing to reason about. For a while, this is the easiest win.

Why it hurts eventually:

• There’s a ceiling. A machine can only get so big. Once we max out the biggest instance our cloud offers, we’re stuck.
• Cost curves badly. The top-tier machines cost way more than double for double the power. The last bit of headroom is the most expensive.
• Single point of failure. One box means one thing to die. If it goes down, everything goes down. No redundancy.

Think of it like — upgrading from a hatchback to a truck. Great, until we need to move more than one truck’s worth of stuff.

Horizontal scaling — scale OUT

Horizontal scaling (scale out) is adding more machines and spreading the load across all of them. Instead of one giant server, we run five normal ones behind a load balancer.

Why it’s powerful:

• No real ceiling. Need more capacity? Add another box. This is how Google and Netflix run.
• Redundancy for free. One machine dies, the other four keep serving. No single point of failure.
• Cheaper hardware. Ten commodity boxes usually beat one exotic super-machine on price-per-request.

Why it’s harder: our app now has to cooperate across machines, and that’s where the real work is.

• The app must be stateless — no storing session data or uploaded files on one specific box, because the next request might land on a different one. State goes to a shared place (Redis, a database, object storage).
• We need a load balancer out front to distribute traffic (see the next note).
• Databases are the tricky part — scaling the app tier out is easy, but scaling the data out means replication and sharding (also coming up).

When each one fits

The honest answer most systems use: do both. Scale up first because it’s easy, then scale out once we hit the wall or need redundancy.

• Reach for vertical when we’re early, traffic is modest, or the workload is a single hard-to-split thing (a big in-memory cache, one heavy relational database). Bumping the instance size buys us months with zero engineering.
• Reach for horizontal when we need high availability (can’t afford one box dying), when traffic is spiky, or when we’ve simply outgrown the biggest machine money can buy.

One more thing interviewers like: horizontal scaling is what makes elasticity possible — auto-scaling groups add boxes during a traffic spike and remove them at 3am. We can’t auto-scale a single vertical machine up and down like that.

Interview soundbite

Vertical scaling means a bigger machine — simple, but it has a hard ceiling and stays a single point of failure. Horizontal scaling means more machines behind a load balancer — near-infinite headroom and built-in redundancy, but it forces our app to be stateless and pushes complexity into the data layer. In practice we scale up first because it’s free engineering-wise, then scale out for availability and elasticity once we hit the wall.

Once we scale out to many servers (previous note), something has to decide which server each request goes to. That something is a load balancer. In simple language — it’s a traffic cop sitting in front of our servers, spreading incoming requests across all of them so no single box gets hammered.

What it actually does

A load balancer accepts every incoming request and forwards it to one of the backend servers (the “pool” or “upstream”). Clients only ever talk to the load balancer’s address — they have no idea there are five servers behind it, or which one served them.

That indirection buys us three things at once:

• Distribution — spread load so no server melts while others sit idle.
• Availability — if a server dies, stop sending it traffic and route around it.
• Flexibility — add or remove servers without clients noticing (great for deploys and auto-scaling).

L4 vs L7

Load balancers work at one of two layers, and the difference is how much of the request they look at.

• L4 (transport layer) balances on TCP/UDP info only — source/destination IP and port. It doesn’t read the actual request content. It just forwards packets. Because it does so little inspection, it’s blazing fast and protocol-agnostic. Think of it like a mail sorter routing by envelope address without opening the letter.
• L7 (application layer) reads the actual HTTP request — the URL path, headers, cookies. So it can route /api/* to one pool and /images/* to another, terminate TLS, and do content-aware tricks. Smarter, but it does more work per request.

The only real difference is: L4 routes by address, L7 routes by content. Most modern web setups use L7 (NGINX, HAProxy, AWS ALB) because path/header routing is so useful.

Distribution algorithms

Once a request arrives, which server gets it? A few standard strategies:

• Round robin — hand requests out in a rotating order: 1, 2, 3, 1, 2, 3… Dead simple, assumes all servers are equal.
• Least connections — send the request to whichever server currently has the fewest active connections. Better when requests take uneven amounts of time.
• IP hash — hash the client’s IP and always send that client to the same server. Gives us stickiness for free.
• Weighted — give beefier servers a bigger share (weight 3 gets three times the traffic of weight 1). Handy for mixed hardware.

upstream app_servers {
    least_conn;                 # pick the least-busy backend
    server 10.0.0.1:8080 weight=3;
    server 10.0.0.2:8080;
    server 10.0.0.3:8080;
}

server {
    location / {
        proxy_pass http://app_servers;
    }
}

Health checks

The load balancer constantly pings each backend to ask “are you alive?” — usually a GET /health that should return 200 OK. If a server fails a few checks in a row, the LB marks it unhealthy and stops routing to it until it recovers. This is what makes horizontal scaling actually reliable: a dead box just quietly drops out of rotation instead of serving errors to users.

Sticky sessions

Normally any server can handle any request — that’s the goal (stateless servers). But sometimes a server holds per-user state in memory (an old-school session). Sticky sessions (session affinity) pin a given user to the same backend for their whole visit, usually via a cookie or IP hash.

It works, but it’s a bit of a smell. It fights the whole point of load balancing — if that one server dies, those users lose their session, and load can’t spread evenly. The cleaner fix is to make servers stateless and push session data into shared storage like Redis. Then any server can serve any request, and we don’t need stickiness at all.

Interview soundbite

A load balancer fronts a pool of servers and spreads requests across them, giving us distribution, availability, and easy scaling. L4 balances on IP/port without reading the request; L7 reads the HTTP content so it can route by path or header. Common algorithms are round robin, least connections, IP hash, and weighted. Health checks pull dead servers out of rotation automatically, and while sticky sessions pin a user to one backend, we’d rather keep servers stateless and store sessions in Redis so any server can handle any request.

Scaling the app tier is easy — just add more servers. Scaling the database is the hard part, and replication is usually the first tool we reach for. In simple language — replication means keeping live copies of our database on other servers, so we can read from many machines instead of pounding one.

Primary handles writes, replicas serve reads

The classic setup is one primary, many replicas (also called leader/follower or master/slave).

• The primary is the single source of truth. Every INSERT, UPDATE, and DELETE goes here.
• Each replica is a read-only copy. The primary streams its changes to them, and they apply those changes to stay in sync.
• Our app sends writes to the primary and spreads reads across the replicas.

Why this works so well: most apps read way more than they write — often 90%+ reads. Timelines, product pages, search results are all reads. By pushing all those reads onto replicas, we take a huge load off the primary, and we can add more replicas as read traffic grows.

Sync vs async replication

The big question: when the primary confirms a write, has it actually reached the replicas yet?

• Asynchronous — the primary commits and replies “done” immediately, then streams the change to replicas afterward. Fast writes, but replicas trail slightly behind. If the primary dies right after committing, a few just-written rows might not have made it out yet.
• Synchronous — the primary waits until at least one replica confirms it got the change before replying “done”. No data loss on failover, but every write is slower because it waits for a network round-trip.

Most systems run async by default (speed matters, and the tiny window is acceptable), and use sync only where losing a write is unacceptable — think payments. Think of it like — async is mailing a copy and moving on; sync is waiting for the recipient to sign for it before we do anything else.

Replication lag & read-your-writes

Because async replicas trail the primary, there’s a gap called replication lag — usually milliseconds, sometimes seconds under load. This causes a genuinely nasty bug: read-your-writes.

Picture this: a user updates their profile (write → primary), the page reloads and reads from a replica — but the replica hasn’t caught up yet, so the user sees their old profile and thinks the save failed.

// user just saved — this write hit the PRIMARY
await db.primary.query("UPDATE users SET bio = $1 WHERE id = $2", [bio, id]);

// immediate reload reads a REPLICA that may still be behind → stale bio 😱
const user = await db.replica.query("SELECT bio FROM users WHERE id = $1", [id]);

Common fixes: read from the primary for a short window right after a user’s own write, pin that user to the primary temporarily, or wait until the replica has caught up to the write’s position. The key interview point is just naming the problem — “async replication means reads can be stale, so read-your-writes needs care.”

Failover & promotion

Only the primary takes writes, so if it dies, writes stop entirely until we fix it. Failover is the process of promoting one of the replicas to become the new primary.

• Manual failover — a human (or runbook) picks a replica and promotes it. Safe, slow.
• Automatic failover — a tool (Patroni, RDS Multi-AZ, etc.) detects the dead primary, promotes the most up-to-date replica, and repoints the app. Fast, but needs care to avoid split-brain — two nodes both thinking they’re the primary and accepting conflicting writes.

After promotion, the app must be told the new primary’s address (usually via a floating DNS name or a proxy), and the old primary, once healed, rejoins as a replica.

Practical takeaway

Replication gives us read scaling and high availability, not write scaling — the single primary is still the write bottleneck (that’s what sharding is for, next note). Default to async replication, route reads to replicas but be deliberate about read-your-writes, and have a tested failover plan so a dead primary doesn’t mean a dead app.

Read replicas scale reads, but the single primary still takes every write. Once writes or data size outgrow one machine, we split the data itself. In simple language — partitioning is chopping a big table into smaller pieces, and sharding is putting those pieces on different servers. Sharding is partitioning that crosses machine boundaries.

Partitioning vs sharding

• Partitioning splits one table into chunks that still live on the same database server. It’s mostly for manageability and query speed — the engine can skip whole partitions it knows are irrelevant.
• Sharding spreads those chunks across multiple servers, so each server (a shard) holds only a slice of the data and handles only a slice of the traffic. This is the one that actually scales writes and storage horizontally.

The only real difference is the boundary: partitioning stays inside one box, sharding goes across boxes.

Horizontal vs vertical partitioning

Two directions to cut a table, and the names trip people up:

• Horizontal partitioning — split by rows. Users A–M in one piece, N–Z in another. Same columns, different rows. This is what “sharding” almost always means.
• Vertical partitioning — split by columns. Put hot, frequently-read columns (id, name) in one table and rarely-touched heavy ones (bio, avatar_blob) in another. Same rows, different columns.

Think of it like a spreadsheet: horizontal slices it with a knife across the rows, vertical slices it down between the columns.

The shard key

The shard key is the column we use to decide which shard a row lives on (e.g. user_id). This single choice is the most important — and most permanent — decision in the whole design. Pick well and load spreads evenly; pick badly and one shard drowns while others nap.

A good shard key has high cardinality (lots of distinct values) and even access (no single value that’s wildly more popular than the rest).

Strategies for mapping key → shard

• Range-based — shard by ranges of the key (users 1–1M on shard A, 1M–2M on shard B). Simple, and range queries stay on one shard. But it hotspots easily: the newest users often get all the action, so the newest shard burns while old ones idle.
• Hash-based — hash the shard key and route by the hash (hash(user_id) % N). Spreads load evenly and kills hotspots. The catch: range queries now hit every shard, and changing the number of shards reshuffles almost everything (consistent hashing softens this).
• Directory-based — keep a lookup table that maps each key to its shard. Maximum flexibility — we can move a specific key anywhere — but the directory becomes a component we must keep fast and highly available, or it’s a new bottleneck and single point of failure.

Hotspots & rebalancing pain

Two things make sharding genuinely hard:

• Hotspots — one shard gets disproportionate traffic. A celebrity user, a viral product, or a range key where all new writes pile onto the latest shard. The fix is a better shard key or splitting the hot shard, neither of which is fun in production.
• Rebalancing — adding a shard usually means moving data between shards while the system is live. With plain hash % N, bumping N remaps nearly every row. This is why teams reach for consistent hashing or pre-split into many virtual shards up front, so growing means moving whole virtual shards, not reshuffling everything.

Cross-shard queries

Here’s the tax we pay. Once data is split across shards, any query that isn’t scoped to one shard key becomes painful:

• JOINs across shards basically don’t work — the rows live on different servers. We denormalize or join in the app.
• Aggregations (COUNT, SUM, “top 10 overall”) must fan out to every shard and merge the results — scatter-gather, slow and chatty.
• Transactions spanning shards need distributed-transaction machinery (two-phase commit / sagas), which is a whole different level of complexity.

-- cheap: scoped to the shard key, lands on exactly one shard
SELECT * FROM orders WHERE user_id = 42;

-- expensive: no shard key, must hit EVERY shard and merge
SELECT COUNT(*) FROM orders WHERE status = 'pending';

Interview soundbite

Partitioning splits a table into pieces on one server; sharding spreads those pieces across servers to scale writes and storage. Horizontal partitioning splits rows, vertical splits columns. Everything hinges on the shard key — pick one with high cardinality and even access. Range strategy keeps ranges together but hotspots on new data; hash spreads evenly but breaks range queries and hurts on resharding; directory is flexible but adds a lookup dependency. The price of sharding is cross-shard queries — JOINs, global aggregations, and multi-shard transactions all get hard, so shard as late as we can.

Once our data lives on multiple machines (replicas, shards), the network between them can break — and then we have to make a hard choice. The CAP theorem is the rule that describes that choice. In simple language — when the network splits, a distributed system can stay consistent or stay available, but not both. It has to give one up.

The three letters

• C — Consistency — every read sees the most recent write. All nodes agree on the current value. (This is linearizability, a stronger promise than the “C” in database ACID.)
• A — Availability — every request gets a non-error response, even if it’s not the freshest data. The system stays up.
• P — Partition tolerance — the system keeps working even when the network between nodes drops or delays messages.

”Pick 2” is a bit of a lie

The famous line is “pick 2 of 3”, but that framing misleads people. In any real distributed system, network partitions WILL happen — cables get cut, nodes get slow, packets drop. So P is not optional. We can’t choose to opt out of the network being unreliable.

That means the real choice is only between C and A, and only during a partition. When the network is healthy, we get all three. The moment it splits, we must pick:

• Refuse requests to avoid serving stale data → we kept C, gave up A → CP.
• Keep answering with possibly-stale data → we kept A, gave up C → AP.

CP vs AP with a concrete example

Picture two data-center nodes, and the link between them dies. A user writes balance = $100 to node 1, then reads from node 2 (which never got the update).

• A CP system makes node 2 refuse the read (or error) rather than return a stale $50. Correct, but that user is temporarily locked out. Good for banking, inventory, locks, config — anywhere wrong data is worse than no data.
• An AP system lets node 2 answer with the old $50 and reconciles once the link heals. Available, but briefly wrong. Good for social feeds, product catalogs, shopping carts, DNS — where being up matters more than being perfectly fresh.

Strong vs eventual consistency

CAP’s “C” is all-or-nothing, but real systems live on a spectrum. Two labels worth knowing:

• Strong consistency — after a write completes, every subsequent read (from anywhere) sees it. Simple to reason about, but needs coordination, so it’s slower and less available. This is the CP flavor.
• Eventual consistency — replicas may disagree for a moment, but with no new writes they all converge to the same value eventually. Fast and available, but a read might be stale. This is the AP flavor.

There are useful middle grounds too — read-your-writes (we always see our own latest changes) and monotonic reads (we never see time go backwards). These make eventual consistency feel far less broken to users.

PACELC — the footnote CAP forgets

CAP only talks about behavior during a partition. But partitions are rare — what about the 99.9% of the time the network is fine? PACELC extends it:

If Partition → choose A or C (the CAP part). Else (normal operation) → choose Latency or Consistency.

The insight: even with no partition, keeping replicas perfectly consistent costs latency (waiting for confirmations). So systems like Cassandra are PA/EL — available under partition, low-latency otherwise — while a strongly-consistent store is PC/EC. It captures the everyday trade-off CAP ignores.

Interview soundbite

CAP says that during a network partition a distributed system must choose consistency or availability — and since partitions are unavoidable, P is a given, so the real choice is CP vs AP. CP systems refuse rather than serve stale data (banking, locks, etcd); AP systems stay up and reconcile later (Cassandra, DynamoDB, DNS). Strong consistency means every read sees the latest write; eventual means replicas converge over time. PACELC adds the missing half: even without a partition, we still trade latency against consistency.

This is the architecture question that shows up in every senior interview. In simple language — a monolith is our whole app as one deployable unit, and microservices split that same app into many small services that talk over the network. Neither is “better”; they trade one set of problems for another.

Monolith — one deployable

A monolith is a single codebase that builds into a single artifact and deploys as one thing. All the features — auth, billing, notifications — live together, call each other as plain in-process function calls, and usually share one database.

Why it’s great, especially early:

• Fast to build. One repo, one deploy, one place to run and debug. No network between modules.
• Simple transactions. A single database means real ACID transactions across features — no distributed-transaction gymnastics.
• Easy to reason about. Call a function, get a result. No serialization, no partial failures, no service discovery.

Where it hurts as it grows:

• Scaling is all-or-nothing. If only the image-processing part is hot, we still have to scale the entire app.
• One tech stack. The whole thing is stuck on one language and framework.
• Deploys get scary. A tiny change means redeploying everything, and one bad module can take the whole process down.

Microservices — many small services

Microservices slice the app into small, independently deployable services, each owning one business capability and (ideally) its own database. They communicate over the network — REST, gRPC, or async messages.

Why teams reach for it:

• Independent scaling. Scale just the checkout service during a sale, leave the rest alone.
• Independent deploys. Teams ship their own service without coordinating one giant release.
• Tech freedom & isolation. Each service can use the best stack for its job, and one crashing service doesn’t necessarily kill the others.

The price — and it’s steep:

• Distributed-system pain. The network is now inside our app. Calls fail halfway, latency stacks up, and we need retries, timeouts, and circuit breakers everywhere.
• No easy transactions. Data spread across service databases means no simple cross-service transaction — we’re into sagas and eventual consistency.
• Operational overhead. Service discovery, distributed tracing, dozens of pipelines, versioned APIs. We’re running a small fleet, not an app.

”Start with a monolith”

The most experienced answer we can give: start with a monolith. Martin Fowler calls it MonolithFirst. Microservices solve problems of scale and team size — problems we usually don’t have on day one. Splitting too early means paying the full distributed-system tax before we even know where the right service boundaries are, and early on those boundaries shift constantly.

The winning move is to build a well-structured monolith, watch where it actually strains, and carve out microservices along seams that have proven themselves — the part that needs its own scaling, or the module a separate team owns. Netflix and Amazon famously moved to microservices, but only after their monoliths were straining under real, known load.

The modular monolith — best of both

There’s a middle ground that’s quietly become the sensible default: the modular monolith. It’s still one deployable, so we keep simple ops and real transactions — but internally it’s split into strict modules with clear boundaries, each owning its own data and only talking to others through defined interfaces (not by reaching into each other’s tables).

Think of it like — microservices’ clean boundaries without the network in between. We get the discipline that makes a future split easy, and if a module ever truly needs to become its own service, the seam is already there. For most teams this is the sweet spot for a long, long time.

Interview soundbite

A monolith is one deployable — simple, fast to build, real transactions, but scales all-or-nothing. Microservices split it into small independently deployable services that scale and deploy on their own and isolate faults, at the cost of real distributed-system complexity: network failures, no easy cross-service transactions, and heavy ops. Best practice is to start with a monolith and extract services only along boundaries that have proven they need it. A modular monolith — one deploy, strict internal modules — gives us most of the benefits without the distributed-system tax.

When two services talk to each other, there are only two ways to do it. Either the caller waits for an answer, or it doesn’t. That’s the whole sync vs async story in one line.

In simple language — synchronous is a phone call (we wait on the line until the other person replies), and asynchronous is a text message (we send it and get on with our life).

Synchronous — the caller waits

Synchronous communication is request-response. Service A calls Service B, then blocks until B replies. A plain HTTP call is the classic example — we send the request, the thread sits there waiting, and only when the response comes back do we move on.

The nice part is it’s simple to reason about. We get an immediate answer, we know right away if it worked or failed, and the code reads top-to-bottom like a normal function call.

// synchronous — we wait for the result before the next line runs
const user = await http.get("https://users-service/users/42");
const cart = await http.get(`https://cart-service/carts/${user.id}`);
// nothing below runs until both replies come back

The catch is coupling. A and B have to be up at the same time. If B is slow, A is slow. If B is down, A’s request fails. We’ve chained their fates together.

Asynchronous — fire and forget

Asynchronous communication means the caller sends a message and moves on immediately, without waiting for a reply. It usually goes through a queue or an event in the middle — A drops a message, and B picks it up whenever it’s ready.

The point is A and B are now decoupled in time. B can be down for an hour; the messages just wait in the queue and get processed when B comes back.

// asynchronous — we hand off the work and return right away
await queue.publish("order.placed", { orderId: 42, total: 999 });
// this line runs instantly — we don't wait for anyone to process it

The trade-off is we don’t get an immediate answer. If we need the result, we have to get it back some other way (a callback, another event, polling). And “did it actually work?” becomes harder to answer.

Side by side

The trade-offs that actually matter

Three things decide the call:

• Coupling — sync couples A and B tightly (same time, same availability). Async decouples them; the queue is a buffer that absorbs outages on either side.
• Latency — sync gives us the answer now but makes us feel B’s slowness. Async has higher end-to-end latency (the message sits in a queue), but A’s own response time stays snappy.
• Resilience — this is async’s big win. A traffic spike or a crashed consumer doesn’t take A down; the backlog just grows and drains later. With sync, B’s failure ripples straight back to A.

When to use each

Reach for synchronous when we need the answer right now to continue — reading data for a page, validating a login, checking stock before showing “in stock”. If the caller can’t do its job without the reply, waiting is the honest choice.

Reach for asynchronous when the work can happen later and we just need it to eventually happen — sending a welcome email, generating a thumbnail, updating a search index, kicking off a report. Anything where “fire it off and trust it’ll get done” is fine.

Think of it like — sync for “I need this to keep going”, async for “please handle this, I’ve got other things to do”.

Interview soundbite

Synchronous is request-response where the caller blocks until it gets a reply (like a phone call) — simple, immediate, but tightly coupled: if the callee is slow or down, so are we. Asynchronous is fire-and-forget through a queue or event (like a text) — the caller returns instantly, giving loose coupling and resilience at the cost of no immediate answer and eventual consistency. Use sync when we need the result to continue, async when the work can happen later.

A message queue is a buffer that sits between whoever produces work and whoever does it. The producer drops a message in one end, the consumer picks it up from the other end. That’s it — a line at the deli counter, but for messages.

In simple language — it’s a to-do list that one service writes to and another service reads from, and neither has to be online at the same moment.

Why put a queue in the middle

Three big reasons, and they’re the answer to “why not just call the service directly?”.

• Decoupling — the producer doesn’t know or care who reads the message, or when. We can add, remove, or restart consumers without touching the producer.
• Load leveling (buffering) — if 10,000 orders land in one second but our worker can only handle 100/sec, the queue soaks up the spike. Work drains out at a steady pace instead of crushing the consumer. Think of it like a dam smoothing out a flood.
• Retries — if processing fails, the message isn’t lost. It stays in the queue (or comes back) so we can try again.

The producer-consumer pattern

The whole thing is just two roles talking through a buffer:

The key detail: with a plain queue, each message is delivered to exactly one consumer. Add more consumers and they share the load — that’s how we scale processing horizontally. (When we want every consumer to get every message, that’s pub/sub, a different pattern covered next.)

// producer — drop a job and move on
await channel.sendToQueue("emails", Buffer.from(JSON.stringify({ to: "a@b.com" })));

// consumer — pull jobs one at a time
channel.consume("emails", async (msg) => {
  await sendEmail(JSON.parse(msg.content.toString()));
  channel.ack(msg); // tell the broker: done, delete it
});

Acknowledgements — “I actually handled it”

An acknowledgement (ack) is the consumer telling the broker “I successfully processed this message, you can delete it now”. Until the ack arrives, the broker holds onto the message.

Why this matters: if the consumer crashes mid-processing before acking, the broker notices the connection dropped and re-delivers the message to another consumer. Nothing gets silently lost. If processing fails, we send a nack (negative ack) to requeue or reject it.

The trap: if we ack before doing the work, a crash means the message is gone forever. Rule of thumb — ack after the work is done, not before.

Delivery guarantees

This is a favourite interview topic. There are three levels:

• At-most-once — the message is delivered zero or one times. Fast, no retries, but we can lose messages. Fine for stuff like metrics where a dropped sample doesn’t hurt.
• At-least-once — the message is delivered one or more times. Nothing is lost, but a crash-and-redeliver can cause duplicates. This is the common default (RabbitMQ, SQS standard).
• Exactly-once — delivered precisely one time, no loss, no dupes. It’s the dream, but genuinely hard and expensive across a distributed system.

In practice most systems pick at-least-once and make the consumer idempotent — meaning “processing the same message twice has the same effect as once”. Stamp each message with an ID, remember which IDs we’ve handled, skip repeats. That gets us the effect of exactly-once without the pain.

Dead-letter queues

Sometimes a message just won’t process — bad data, a bug, a downstream that’s permanently gone. If we keep retrying, it loops forever and blocks the queue (“poison message”).

A dead-letter queue (DLQ) is a separate queue where these failures get parked after N failed attempts. Instead of retrying to infinity, the broker moves the message aside so the main queue keeps flowing. We inspect the DLQ later to debug or replay. Think of it like a lost-and-found bin for messages that couldn’t be delivered.

Where we see this

• RabbitMQ — a traditional message broker (AMQP). Producers publish to an exchange, which routes to queues; consumers ack messages. Great for classic task queues and routing.
• Amazon SQS — a fully managed queue. Standard queues are at-least-once with best-effort ordering; FIFO queues add strict ordering and exactly-once processing within a dedup window.

Interview soundbite

A message queue is a buffer between producer and consumer that gives us decoupling, load leveling, and retries. Each message goes to exactly one consumer, and consumers ack after processing so crashes trigger redelivery instead of loss. Delivery guarantees come in three flavours — at-most-once (can lose), at-least-once (can duplicate, the common default), exactly-once (hard) — and the pragmatic answer is at-least-once plus idempotent consumers. Messages that keep failing land in a dead-letter queue so they don’t block everything else.

Publish/subscribe (pub/sub) is a messaging pattern where a publisher sends one message and every interested subscriber gets a copy. The publisher has no idea who’s listening — it just announces “this happened” and walks away.

In simple language — a queue is a direct message to one worker; pub/sub is a group announcement that everyone subscribed hears.

Queue vs pub/sub — the one difference

This is the distinction interviewers love. Both are async messaging, but the fan-out is opposite:

The middle box in pub/sub is a topic — a named channel like order.placed or user.signed-up. Publishers send to the topic; subscribers register their interest in the topic. The topic is what lets N publishers and M subscribers connect without knowing each other.

// publisher — announce that something happened, don't care who listens
await sns.publish({ topic: "order.placed", message: { orderId: 42 } });

// three independent subscribers, each doing their own thing
onEvent("order.placed", sendConfirmationEmail);
onEvent("order.placed", decrementInventory);
onEvent("order.placed", updateAnalytics);

One order.placed event, three reactions — and the order service wrote none of that follow-up logic. That’s the magic.

Event-driven architecture

Event-driven architecture (EDA) takes pub/sub and makes it the backbone of the whole system. Instead of services calling each other directly (“hey inventory, decrement this”), a service just emits an event describing what happened (“an order was placed”). Other services react to events they care about.

The shift in thinking: nobody commands anyone. Services announce facts (“this occurred”) and interested parties respond on their own. The order service doesn’t orchestrate email + inventory + analytics — it just says “order placed” and those three services take it from there.

Think of it like a newsroom — a reporter publishes a story, and whoever’s subscribed (readers, aggregators, other papers) reacts however they want. The reporter doesn’t phone each reader.

The upside

• Loose coupling — publishers and subscribers don’t know each other exist. We add a new subscriber (say, a fraud-check service) without touching the publisher. Massive for evolving a system.
• Scalability — subscribers scale independently. The email service can have 2 instances and analytics 20, all fed by the same event stream.
• Extensibility — new features often mean just “subscribe to an existing event”. No changes rippling across services.

The downside (be honest in interviews)

• Harder to trace and debug — with a direct call, the flow is a straight line we can follow. With events, one publish fans out to who-knows-how-many reactions across services. Answering “what happens when an order is placed?” now means mapping every subscriber. We lean on correlation IDs and distributed tracing to follow a request.
• Eventual consistency — because reactions happen asynchronously, the system is briefly out of sync. Right after order.placed, inventory might not be decremented yet. Everything settles correctly given a moment, but “right now” the parts can disagree. We have to design for that instead of assuming instant consistency.
• No built-in “did it work?” — the publisher gets no confirmation that subscribers succeeded. Error handling moves into each subscriber (retries, DLQs).

Where we see this

Amazon SNS is the textbook pub/sub service — publish to a topic, and it fans out to every subscriber (often SQS queues, so each consumer group gets its own durable copy). This SNS → SQS fan-out combo is a super common pattern: one event, delivered to many queues, each drained by its own service.

Interview soundbite

Pub/sub is one-to-many messaging: a publisher sends an event to a named topic and every subscriber gets a copy — versus a queue, where a message goes to exactly one consumer. Event-driven architecture builds a whole system on this: services emit facts (“order placed”) instead of commanding each other, and interested services react on their own. The wins are loose coupling and independent scalability; the costs are harder tracing/debugging and eventual consistency, since reactions happen asynchronously.

Kafka is a distributed, append-only log. Not a queue that hands out messages and deletes them — a durable log that just keeps appending records to the end, like a never-ending append-only file that consumers read through at their own pace.

In simple language — Kafka is a giant shared logbook. Producers write new lines at the bottom, and each reader keeps their own bookmark for how far they’ve read.

Why a log and not a queue

This is the mental model everything else hangs off. A traditional queue deletes a message once it’s consumed — the broker tracks what’s been delivered. Kafka does the opposite: it keeps every record for a retention period, and each consumer tracks its own position. The broker doesn’t remember who read what; the consumer does.

That flip is the whole reason Kafka exists. Because records stick around and readers own their position, we get replay — a new consumer can start from the beginning and read all of history, or a broken consumer can rewind and re-process. A queue can’t do that; once a message is gone, it’s gone.

Core concepts

A handful of terms, and they all fit together:

• Broker — a single Kafka server. A cluster is several brokers sharing the load and replicating data for durability.
• Topic — a named stream of records, like payments or clicks. This is what producers write to and consumers read from.
• Partition — a topic is split into partitions, and each partition is its own ordered, append-only log. This is the unit of parallelism and ordering.
• Offset — a record’s sequential position within a partition (0, 1, 2, …). A consumer’s progress is just “I’m at offset N in partition P”. The bookmark.
• Producer — writes records to a topic. It picks which partition (often by hashing a key, so all records for one key land in the same partition).
• Consumer group — a set of consumers that split the partitions of a topic between them. Kafka guarantees each partition is read by exactly one consumer within a group, which is how work gets divided.

Why partitions matter

Partitions buy us two things that are usually in tension: ordering and parallelism.

• Ordering is guaranteed per partition, not per topic. Kafka promises records in one partition are read in the exact order they were written. Across the whole topic? No global order. So if we need related events ordered (all events for order-42), we give them the same key — Kafka hashes the key to a partition, so they all land in the same partition and stay ordered.
• Parallelism comes from having many partitions. Each partition can be processed by a different consumer at the same time. Want more throughput? Add partitions and consumers. The ceiling on parallelism is the partition count — more consumers than partitions and the extras just sit idle.

Think of it like — one partition is a single-file queue (perfectly ordered but slow), and splitting into many partitions is opening more checkout lanes (fast, but order is only kept within a lane).

Retention and replay

Kafka keeps records for a configured retention window — by time (say 7 days) or by size — regardless of whether anyone consumed them. Nothing is deleted just because it was read.

Because of that, and because consumers own their offsets, replay is trivial: reset the consumer group’s offset back to 0 (or to a timestamp) and re-read everything. This is huge for real work — reprocessing after a bug fix, backfilling a brand-new service with all historical events, or spinning up an analytics job that reads the full stream from the start.

# rewind a consumer group to the beginning and re-process the whole topic
kafka-consumer-groups --bootstrap-server localhost:9092 \
  --group analytics --topic orders --reset-offsets --to-earliest --execute

Queue vs log, one more time

The distinction worth nailing: in a queue, the broker owns delivery state and deletes consumed messages — one shot, then gone. In a log like Kafka, records are retained and each consumer group tracks its own offset independently — so five different services can read the same topic at their own speeds, and any of them can rewind. Same stream, many independent readers, full history.

Interview soundbite

Kafka is a distributed, append-only log rather than a queue. A topic is split into partitions, each an ordered log; a record’s position is its offset. Ordering is guaranteed per partition (use a key to keep related events together), and partitions give parallelism since each is read by exactly one consumer in a consumer group. The key difference from a queue: Kafka retains records for a retention window and consumers track their own offsets — so multiple consumer groups read the same stream independently, and any of them can rewind to replay history.

Observability is how well we can understand what’s happening inside our system just by looking at what it emits from the outside. In simple language — when production breaks at 2am, observability is the difference between “I can see exactly why” and “let me add some logs and redeploy and pray”.

The why is easy. A backend with a dozen services has too many moving parts to reason about in our head. We need the system to tell us what it’s doing. It does that through three kinds of signals, and interviewers love asking us to name all three.

The three pillars

Each pillar answers a different question. That’s the whole point — we need all three, not one.

Logs — the events

A log is a timestamped record of something that happened. “User 42 logged in”, “payment failed with code 402”. They’re the detail layer.

The one upgrade that matters: structured logging. Instead of a plain sentence, we emit a JSON object with fields. Why? Because we can then search and filter by field instead of grepping strings.

// Bad — a human sentence, painful to query
console.log(`payment failed for user ${userId}, amount ${amount}`);

// Good — structured, machine-queryable
logger.error({ event: "payment_failed", userId, amount, code: 402 });
// now we can filter: event=payment_failed AND code=402

Add a correlation id (more on that below) to every log line and suddenly we can pull every log for a single request across every service.

Metrics — the numbers

A metric is a number measured over time — request count, error rate, memory usage, queue depth. They’re cheap to store (just numbers + timestamps) so we keep them forever and graph them. This is what dashboards and alerts run on.

Two famous shortcuts for which metrics to track:

• RED — for request-driven services: Rate (requests/sec), Errors (failures/sec), Duration (latency). Think of it as the health of a thing that serves traffic.
• USE — for resources: Utilization, Saturation, Errors. The health of a thing that has capacity — CPU, disk, a connection pool.

Rule of thumb: RED for our API endpoints, USE for the machines and pools underneath them.

Traces — the request’s journey

A trace follows one single request as it hops across services. It’s made of spans — each span is one unit of work (an HTTP call, a DB query) with a start time, a duration, and a parent. Stitch the spans together and we get a waterfall showing exactly where the 800ms went.

The glue is a trace id (also called a correlation id). The first service generates it, then passes it along in a header to every downstream call. Every log and span tags itself with that id. That’s how “one request across ten services” becomes a single searchable story.

Think of it like a package tracking number — one id, and we can see every warehouse the parcel passed through.

Monitoring vs observability

People use these interchangeably but interviewers like the distinction.

• Monitoring = watching known problems. We decide in advance what to measure (“alert if error rate > 1%”) and watch those dashboards. Great for the failures we’ve already imagined.
• Observability = being able to ask new questions without shipping new code. When something novel breaks, rich logs/metrics/traces let us slice the data in ways we didn’t plan for.

In simple language — monitoring answers “is the thing I worried about happening?”, observability answers “why is this weird thing I’ve never seen happening?”.

A word on alerting

Metrics feed alerts, but a good alert has one property: it’s actionable and tied to user pain. Alerting on “CPU is 90%” wakes us up for nothing if users are fine. Alerting on symptoms — high error rate, high latency, SLO burn — wakes us up only when something actually hurts. The SRE mantra: page on symptoms, not causes.

Interview soundbite

Observability rests on three pillars: logs (discrete structured events — what happened), metrics (numbers over time — track RED for services, USE for resources), and traces (one request’s path across services, made of spans stitched by a shared trace/correlation id). Monitoring watches problems we already predicted; observability lets us investigate ones we didn’t. And we alert on user-facing symptoms, not raw resource numbers.

A health check is a tiny endpoint our service exposes so an orchestrator can ask “are you okay?” without guessing. In simple language — it’s the service answering a doctor’s “how are you feeling?” so the platform knows whether to keep sending it traffic or restart it.

The why: in a world of containers, something else (Kubernetes, a load balancer) is constantly deciding where to route requests and when to reboot a sick instance. It can only make good decisions if each instance honestly reports its state. There are two different questions it needs answered, which is why we have two probes.

Liveness vs readiness

They sound similar and get confused constantly. Here’s the clean split:

• Liveness = “are you alive, or should I kill and restart you?” A failed liveness probe means the process is wedged — deadlocked, stuck — and a restart is the fix.
• Readiness = “are you ready to take traffic right now?” A failed readiness probe means “I’m alive but busy/warming up — don’t send me requests yet”, and the platform just stops routing to us. No restart.

Why both? Because “broken forever, restart me” and “temporarily not ready, just hold traffic” need completely different responses. Restarting a service that’s merely warming up its cache would be a disaster loop. Removing a deadlocked service from rotation but never restarting it would leave it dead forever.

Think of it like a shop: liveness is “is anyone in the building?” (if not, unlock and send staff in), readiness is “is the shop actually open for customers?” (the lights are on but they’re still restocking shelves).

# Kubernetes — same container, two very different checks
livenessProbe:
  httpGet: { path: /healthz, port: 8080 }   # deadlocked? -> restart me
  periodSeconds: 10
readinessProbe:
  httpGet: { path: /readyz, port: 8080 }     # DB not connected yet? -> hold traffic
  periodSeconds: 5

A good /readyz actually checks dependencies — “can I reach the DB?” — while /healthz stays dumb and cheap so a slow DB doesn’t get our whole process killed.

How the platform uses them

The load balancer or orchestrator polls these on a schedule. On readiness fail, our instance is pulled out of the pool of endpoints receiving traffic — silently, no user sees an error. On liveness fail, our instance gets killed and a fresh one is started. This is the machinery that makes rolling deploys and self-healing work.

Graceful shutdown

Now the flip side. When we deploy a new version, the platform has to stop the old instances. If we just yank the plug, every request that was mid-flight gets dropped — users see 502s. Graceful shutdown is the polite exit that avoids that.

The trigger is SIGTERM — the platform sends this signal and gives us a grace period (Kubernetes default 30s) before it sends the un-ignorable SIGKILL. Our job is to use that window well.

The order matters. We fail readiness first so the platform stops routing to us, then close the listener, then drain. If we closed the listener before flipping readiness, requests could still be in transit and get refused.

// Node/Express — a minimal graceful shutdown handler
let ready = true;
app.get("/readyz", (_req, res) => res.status(ready ? 200 : 503).end());

process.on("SIGTERM", async () => {
  ready = false;                         // 1. stop new traffic
  server.close(async () => {             // 2+3. stop accepting, drain existing
    await db.end();                      // 4. close the DB pool
    process.exit(0);                     // 5. clean exit
  });
  // safety net: force-exit if draining hangs past the grace period
  setTimeout(() => process.exit(1), 25_000).unref();
});

That setTimeout safety net matters — if one request hangs forever, we still exit before SIGKILL yanks us harder.

Practical takeaway

Expose two probes: a dumb /healthz for liveness (restart me if wedged) and a dependency-aware /readyz for readiness (hold traffic if I’m not ready). On SIGTERM, fail readiness first, stop accepting new work, drain what’s in flight, close the DB pool, then exit — all inside the grace period, with a timeout as backstop. Do this and rolling deploys drop zero requests.

When our service calls another service, that call can be slow, fail, or hang forever. Timeouts, retries, and circuit breakers are the three patterns that stop one flaky dependency from dragging our whole service down with it. In simple language — they’re how we call something unreliable without betting our own uptime on it.

The why is a chain reaction. A downstream service gets slow → our requests to it pile up waiting → our threads/connections get exhausted → we go down → whatever calls us goes down. This is a cascading failure, and these three patterns are the standard defense.

Timeouts — never wait forever

A timeout is a hard limit on how long we’ll wait for a response before giving up. This is the most important one and the most often forgotten.

Why it matters: without a timeout, a hung downstream call holds our resources hostage indefinitely. One stuck dependency slowly consumes every connection in our pool, and now we’re down too — even though our own code is fine.

// Every outbound call gets a deadline. No exceptions.
const res = await fetch(url, { signal: AbortSignal.timeout(2000) }); // 2s cap

Rule of thumb: every network call — HTTP, DB query, cache lookup — gets a timeout. “Wait forever” is never the right default.

Retries — but carefully

A retry is trying again after a failure, on the bet that the failure was transient (a blip, a brief network hiccup). Sometimes that’s exactly right. But naive retries are dangerous, so two rules:

Rule 1 — only retry idempotent operations. An operation is idempotent if doing it twice has the same effect as doing it once (a GET, or a PUT that sets a value). Retrying a non-idempotent POST /charge might charge the customer twice. If we can’t guarantee idempotency, we don’t retry — or we add an idempotency key so the server dedupes.

Rule 2 — back off with exponential backoff + jitter. Don’t retry immediately, and don’t retry on a fixed interval. Wait longer each time (1s, 2s, 4s…) and add jitter (randomness) so a thousand clients don’t all retry in sync.

// exponential backoff with full jitter
const base = 100; // ms
const delay = Math.random() * (base * 2 ** attempt); // attempt = 0,1,2...
await sleep(delay);

The retry storm

Here’s the trap interviewers love. A downstream service is struggling under load. Every caller retries. Retries triple the traffic hitting an already-overloaded service. It falls over completely. This is a retry storm (or retry amplification) — retries turning a small blip into a full outage.

The jitter above spreads retries out. But the real fix for “the downstream is genuinely overloaded” is the next pattern.

Circuit breaker

A circuit breaker wraps a downstream call and stops calling it when it’s clearly failing — so we fail fast instead of piling on. Think of it exactly like the electrical breaker in our house: when there’s a fault, it trips and cuts the circuit rather than letting the wiring burn.

It’s a little state machine with three states.

• Closed — normal. Requests pass through. We count failures.
• Open — the failure threshold was crossed, so the breaker trips. Now every call fails instantly without touching the downstream. This gives the sick service room to recover instead of getting hammered.
• Half-open — after a cooldown, we cautiously let a few probe requests through. If they succeed, close the breaker (back to normal). If they fail, snap back open and wait again.

The magic is the open state: by refusing to call a failing service, we stop the retry storm and fail fast (users get an instant error or fallback instead of waiting for a timeout on every request).

Bulkheads, briefly

A bulkhead isolates resources so a failure in one area can’t drain everything — named after the sealed compartments in a ship’s hull that stop one flooded section from sinking the whole boat. In practice: give each downstream dependency its own connection pool / thread pool. If service X hangs and exhausts its pool, services Y and Z keep working because they have separate pools.

Interview soundbite

Three layers of defense against a cascading failure: timeouts (never wait forever — every call gets a deadline), retries (only for idempotent/transient failures, always with exponential backoff + jitter to avoid a retry storm), and the circuit breaker (a closed → open → half-open state machine that trips after repeated failures and fails fast so the sick dependency can recover). Add bulkheads — separate resource pools per dependency — so one hung service can’t drain the rest.

A distributed transaction is one logical unit of work — “place an order” — that spans multiple services, each with its own database. In simple language — it’s trying to keep several separate databases in agreement when there’s no single transaction wrapping them all. And the honest truth interviewers want us to say is: we can’t, not cleanly, so we use patterns that give us eventual consistency instead.

Why ACID doesn’t stretch across services

In a single database, a transaction is ACID — all-or-nothing, isolated, durable. We BEGIN, do three writes, COMMIT, and either all three land or none do. Beautiful.

The moment “place an order” touches the Order service’s DB, the Payment service’s DB, and the Inventory service’s DB, that guarantee is gone. There’s no shared BEGIN/COMMIT across three separate databases owned by three separate services. Each can commit or fail independently. So how do we keep them consistent?

Two-phase commit (and why we avoid it)

The textbook answer is 2PC (two-phase commit): a coordinator asks every participant “can you commit?” (prepare phase), and if all say yes, tells everyone “commit” (commit phase).

It technically works, but we avoid it in microservices because:

• It’s blocking. Everyone holds locks while waiting for the coordinator’s decision. Slow and low-throughput.
• The coordinator is a single point of failure. If it dies mid-commit, participants are stuck holding locks, unsure what to do.
• It couples services tightly and most modern datastores/message brokers don’t support it well.

So in practice, we reach for the Saga pattern.

The Saga pattern

A saga breaks the big distributed transaction into a sequence of local transactions, one per service. Each service does its own little ACID transaction and then triggers the next step. No global lock, no coordinator holding everyone hostage.

The catch: there’s no automatic rollback across the whole thing. So for every step, we define a compensating action — an explicit “undo” that reverses it. If step 4 fails, we run the compensations for steps 3, 2, 1 in reverse. We don’t roll back; we apologize and reverse.

Think of it like a chain of “do X / to undo X do Y” pairs.

There are two ways to coordinate a saga:

• Choreography — no central boss. Each service listens for events and emits its own. Order emits OrderCreated → Payment reacts, emits PaymentDone → Inventory reacts. Decentralized and loosely coupled, but the flow is spread across services and hard to follow (“who does what?” gets fuzzy).
• Orchestration — one orchestrator service holds the workflow and tells each service what to do next, step by step. Easier to reason about and to add compensation logic, but the orchestrator is a component we have to build and maintain.

Rule of thumb: choreography for simple 2–3 step flows, orchestration once the workflow gets complex enough that you want it written down in one place.

The dual-write problem

Here’s a subtle bug that sagas run straight into. A service often needs to do two things atomically: update its database AND publish an event (“payment done”) for the next step. That’s a dual write — two separate systems, no shared transaction.

What if we commit to the DB but crash before publishing the event? The DB says “paid”, but no one downstream ever hears about it. The saga stalls. Consistency broken. We cannot make “write to DB” and “write to the message broker” atomic — they’re two different systems.

The transactional Outbox pattern

The Outbox pattern fixes the dual-write problem with one clever move: instead of publishing the event directly, we write the event into an outbox table in the same database, in the same local transaction as the business data.

Now it’s a single local ACID transaction — the order update and the event row commit together or not at all. No dual write.

-- ONE transaction: business write + event write commit atomically
BEGIN;
  UPDATE orders SET status = 'paid' WHERE id = 42;
  INSERT INTO outbox (topic, payload)
    VALUES ('order.paid', '{"orderId":42}');
COMMIT;

Then a separate relay process (a poller, or Change Data Capture tailing the DB log) reads unpublished rows from the outbox table and publishes them to the message broker, marking each as sent. If the relay crashes, it just re-reads on restart — so events are delivered at least once. That means downstream consumers must be idempotent (safe to process the same event twice), which pairs perfectly with the idempotency habits from retries.

Interview soundbite

ACID transactions can’t span services, and 2PC is avoided because it’s blocking with a single-point-of-failure coordinator. Instead we use a saga: a sequence of local transactions each with a compensating action for undo, coordinated by choreography (event-driven, decentralized) or orchestration (a central workflow service). To publish events reliably we use the transactional Outbox — write the event to an outbox table in the same local transaction as the data, then a relay ships it to the broker at-least-once — which sidesteps the dual-write problem.

A container is just a normal process running on the host, wrapped in isolation so it thinks it has its own machine — its own filesystem, network, and process tree. Bundled with it are all the dependencies our app needs. In simple language — it’s our app plus everything it needs to run, sealed in a box that behaves the same on any machine.

The why is the oldest complaint in software: “works on my machine”. Different OS, different library versions, a missing system package — and the app that ran fine on our laptop face-plants in production. A container ships the app and its exact environment together, so “my machine” and “the server” become identical boxes.

Container vs VM

The classic interview question. Both isolate workloads, but at very different depths.

A virtual machine virtualizes hardware. Each VM runs a full guest OS (its own kernel) on top of a hypervisor. Heavy — gigabytes, and tens of seconds to boot.

A container virtualizes the operating system. All containers share the host’s kernel and are isolated by kernel features (namespaces for “what you can see”, cgroups for “how much you can use”). No guest OS per container. So they’re tiny — megabytes — and start in milliseconds.

Think of it like this: a VM is building a whole separate house next door; a container is a locked, private room inside the same house — sharing the plumbing (the kernel) but with its own door.

Trade-off in one line: VMs give stronger isolation (separate kernels), containers give far more speed and density. Most backend services want containers.

Image vs container

Two words people mix up constantly. The relationship is simple:

• An image is the blueprint — a read-only, packaged snapshot of our app + its dependencies. Static, stored, shareable.
• A container is a running instance of an image. Live, has its own writable layer on top.

Think classes vs objects: the image is the class, the container is the object we instantiate from it. One image → many identical containers.

Dockerfile layers & caching

We build an image from a Dockerfile — a recipe of instructions. The key insight for interviews: each instruction creates a layer, and Docker caches layers. On a rebuild, if an instruction and everything above it are unchanged, Docker reuses the cached layer instead of redoing the work.

This is why instruction order matters so much. Put things that rarely change first and things that change often last. Classic example — copy the dependency manifest and install deps before copying our source code, so a one-line code change doesn’t bust the (slow) dependency-install cache.

FROM node:20-alpine
WORKDIR /app

# deps first — this layer is cached until package.json changes
COPY package*.json ./
RUN npm ci

# source last — changing code only rebuilds from here down
COPY . .
CMD ["node", "index.js"]

If we’d copied COPY . . before npm ci, every code edit would invalidate the cache and reinstall every dependency. Slow builds, all self-inflicted.

Multi-stage builds, briefly

Sometimes we need heavy tooling to build (compilers, dev dependencies) but none of it to run. A multi-stage build uses one stage to build and a second, clean stage that copies only the finished artifact — so the final image stays tiny and has no build tools to attack.

# stage 1: build with the full toolchain
FROM node:20 AS build
WORKDIR /app
COPY . .
RUN npm ci && npm run build

# stage 2: ship only the output on a slim base
FROM nginx:alpine
COPY --from=build /app/dist /usr/share/nginx/html

Smaller image = faster deploys, less to download, smaller attack surface. Win, win, win.

Practical takeaway

A container is an isolated process sharing the host kernel, bundled with its dependencies — lighter and faster than a VM, which runs a whole guest OS. An image is the blueprint, a container is a running instance of it. Order Dockerfile instructions least-changing first so layer caching skips the slow steps, and reach for a multi-stage build to keep build tools out of the final image.

CI/CD is the automated pipeline that takes our code from a git push to running in production, with as little manual fiddling as possible. In simple language — it’s the assembly line that builds, tests, and ships our code so humans don’t do it by hand (and don’t forget a step at 5pm on a Friday).

The why: manual builds and deploys are slow, inconsistent, and error-prone. Automating them means changes land fast, in small batches, tested the same way every time — so bugs get caught early and a bad deploy is a quick rollback instead of a crisis.

CI vs CD — they’re two different things

The acronym smushes two ideas together. Interviewers like us to split them cleanly.

• CI (Continuous Integration) — developers merge their work into the shared main branch frequently (many times a day), and every merge automatically triggers a build + test run. The goal: catch integration problems now, while the change is small, instead of during a giant painful merge later.
• CD — has two flavors people conflate:
  • Continuous Delivery — every change that passes CI is automatically prepared and made deployable; a human clicks the button to actually release.
  • Continuous Deployment — one step further: passing changes go straight to production automatically, no human button.

The only difference between the two CDs is that last manual gate. Delivery = “always ready to ship, we choose when”. Deployment = “if it’s green, it’s live”.

A typical pipeline

A pipeline is a series of stages; if any stage fails, the whole thing stops and nothing ships. Each stage is a gate.

# a stripped-down GitHub Actions pipeline
on: [push]
jobs:
  ci:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci          # build
      - run: npm test        # test
      - run: npm audit       # scan
      - run: ./build.sh      # deploy: build image + push to registry

The build usually produces one immutable artifact (a Docker image) that’s promoted unchanged through staging to production — so what we tested is exactly what runs. No rebuilding per environment.

The 12-Factor App

The twelve-factor app is a set of principles for building services that are easy to deploy, scale, and run in the cloud. We won’t recite all twelve like a shopping list — that’s a red flag in an interview. Instead, here are the ones that actually shape backend design.

• Config in the environment. Anything that differs between environments (DB URLs, API keys, feature flags) lives in env vars, never hardcoded or committed. Why? One identical build runs everywhere; only the env changes. It also keeps secrets out of git.

  # same image, different env per environment
  DATABASE_URL=postgres://prod-db:5432/app
  LOG_LEVEL=info

• Stateless processes. Our app processes should keep no important state in local memory or disk between requests. Any state that must persist goes to a backing service (DB, Redis). Why? So we can kill, restart, and scale processes freely — and any instance can handle any request. Store sessions in Redis, not in-process. This is what makes horizontal scaling work.

• Backing services are attached resources. A database, cache, or queue is just a resource we reach by URL, swappable without a code change. Local Postgres and managed cloud Postgres should be interchangeable by flipping one env var — no code edits.

• Logs as event streams. The app doesn’t manage log files. It just writes to stdout as a stream, and the platform captures, routes, and stores it. Why? The app shouldn’t care where logs end up — that’s the environment’s job. (This is exactly why structured stdout logging pairs so well with containers.)

• Dev/prod parity. Keep development, staging, and production as similar as possible — same backing services, same OS, same config shape. Why? The bigger the gap, the more “worked in dev, broke in prod” surprises. Containers help enormously here.

There’s also disposability — start fast, shut down gracefully on SIGTERM — which ties straight into health checks and graceful shutdown.

Interview soundbite

CI is merging to main frequently with automatic build + test on every change; CD is either continuous delivery (always deployable, human clicks release) or continuous deployment (green means live, no human). A pipeline gates build → test → scan → deploy, promoting one immutable artifact through environments. The 12-factor rules that matter most for backend: config in env vars, stateless processes (state in backing services so we scale horizontally), attached backing services, logs to stdout as streams, and dev/prod parity.