A message queue is a buffer that sits between whoever produces work and whoever does it. The producer drops a message in one end, the consumer picks it up from the other end. That’s it — a line at the deli counter, but for messages.
In simple language — it’s a to-do list that one service writes to and another service reads from, and neither has to be online at the same moment.
Why put a queue in the middle
Three big reasons, and they’re the answer to “why not just call the service directly?”.
- Decoupling — the producer doesn’t know or care who reads the message, or when. We can add, remove, or restart consumers without touching the producer.
- Load leveling (buffering) — if 10,000 orders land in one second but our worker can only handle 100/sec, the queue soaks up the spike. Work drains out at a steady pace instead of crushing the consumer. Think of it like a dam smoothing out a flood.
- Retries — if processing fails, the message isn’t lost. It stays in the queue (or comes back) so we can try again.
The producer-consumer pattern
The whole thing is just two roles talking through a buffer:
The key detail: with a plain queue, each message is delivered to exactly one consumer. Add more consumers and they share the load — that’s how we scale processing horizontally. (When we want every consumer to get every message, that’s pub/sub, a different pattern covered next.)
// producer — drop a job and move on
await channel.sendToQueue("emails", Buffer.from(JSON.stringify({ to: "a@b.com" })));
// consumer — pull jobs one at a time
channel.consume("emails", async (msg) => {
await sendEmail(JSON.parse(msg.content.toString()));
channel.ack(msg); // tell the broker: done, delete it
});Acknowledgements — “I actually handled it”
An acknowledgement (ack) is the consumer telling the broker “I successfully processed this message, you can delete it now”. Until the ack arrives, the broker holds onto the message.
Why this matters: if the consumer crashes mid-processing before acking, the broker notices the connection dropped and re-delivers the message to another consumer. Nothing gets silently lost. If processing fails, we send a nack (negative ack) to requeue or reject it.
The trap: if we ack before doing the work, a crash means the message is gone forever. Rule of thumb — ack after the work is done, not before.
Delivery guarantees
This is a favourite interview topic. There are three levels:
- At-most-once — the message is delivered zero or one times. Fast, no retries, but we can lose messages. Fine for stuff like metrics where a dropped sample doesn’t hurt.
- At-least-once — the message is delivered one or more times. Nothing is lost, but a crash-and-redeliver can cause duplicates. This is the common default (RabbitMQ, SQS standard).
- Exactly-once — delivered precisely one time, no loss, no dupes. It’s the dream, but genuinely hard and expensive across a distributed system.
In practice most systems pick at-least-once and make the consumer idempotent — meaning “processing the same message twice has the same effect as once”. Stamp each message with an ID, remember which IDs we’ve handled, skip repeats. That gets us the effect of exactly-once without the pain.
Dead-letter queues
Sometimes a message just won’t process — bad data, a bug, a downstream that’s permanently gone. If we keep retrying, it loops forever and blocks the queue (“poison message”).
A dead-letter queue (DLQ) is a separate queue where these failures get parked after N failed attempts. Instead of retrying to infinity, the broker moves the message aside so the main queue keeps flowing. We inspect the DLQ later to debug or replay. Think of it like a lost-and-found bin for messages that couldn’t be delivered.
Where we see this
- RabbitMQ — a traditional message broker (AMQP). Producers publish to an exchange, which routes to queues; consumers ack messages. Great for classic task queues and routing.
- Amazon SQS — a fully managed queue. Standard queues are at-least-once with best-effort ordering; FIFO queues add strict ordering and exactly-once processing within a dedup window.
Interview soundbite
A message queue is a buffer between producer and consumer that gives us decoupling, load leveling, and retries. Each message goes to exactly one consumer, and consumers ack after processing so crashes trigger redelivery instead of loss. Delivery guarantees come in three flavours — at-most-once (can lose), at-least-once (can duplicate, the common default), exactly-once (hard) — and the pragmatic answer is at-least-once plus idempotent consumers. Messages that keep failing land in a dead-letter queue so they don’t block everything else.