Gyaan

Concurrency Patterns

advanced go concurrency

Once we know goroutines, channels, select, and sync, the next step is combining them into patterns. These are the building blocks that show up in real backend code — and in interviews.

Worker pool

A worker pool spins up a fixed number of goroutines that all pull from the same jobs channel and push to a results channel. We control concurrency by choosing how many workers to start.

In simple language, think of it like a checkout line with N cashiers. Customers (jobs) line up, and whichever cashier is free grabs the next one.

jobs chan →
worker 1 worker 2 worker 3
→ results chan 
func worker(jobs <-chan int, results chan<- int, wg *sync.WaitGroup) {
	defer wg.Done()
	for j := range jobs { // pulls until jobs is closed
		results <- j * 2 // do the work, send the result
	}
}

func main() {
	jobs := make(chan int, 100)
	results := make(chan int, 100)
	var wg sync.WaitGroup

	for w := 1; w <= 3; w++ { // start 3 workers
		wg.Add(1)
		go worker(jobs, results, &wg)
	}

	for j := 1; j <= 9; j++ {
		jobs <- j
	}
	close(jobs) // tell workers no more jobs are coming

	go func() { wg.Wait(); close(results) }() // close results once all workers done

	for r := range results {
		fmt.Println(r)
	}
}

Note the trick at the end: we close(jobs) so the workers’ range loops finish, then close results in a separate goroutine once the WaitGroup confirms everyone’s done. That lets the final range results terminate cleanly.

Fan-out / fan-in

Fan-out means several goroutines read from the same channel to spread work across cores. Fan-in means we merge several channels back into one. The worker pool above is already fan-out; fan-in is the merge step.

Fan-out
src →
w1 w2
Fan-in
c1 c2
→ merged 
func merge(cs ...<-chan int) <-chan int {
	out := make(chan int)
	var wg sync.WaitGroup
	for _, c := range cs {
		wg.Add(1)
		go func(ch <-chan int) {
			defer wg.Done()
			for v := range ch {
				out <- v // funnel every input channel into out
			}
		}(c)
	}
	go func() { wg.Wait(); close(out) }() // close out when all inputs drained
	return out
}

Pipeline

A pipeline is a chain of stages, each a goroutine, connected by channels. The output of one stage is the input of the next — like an assembly line.

func gen(nums ...int) <-chan int {
	out := make(chan int)
	go func() {
		for _, n := range nums {
			out <- n
		}
		close(out)
	}()
	return out
}

func square(in <-chan int) <-chan int {
	out := make(chan int)
	go func() {
		for n := range in {
			out <- n * n // transform each value
		}
		close(out)
	}()
	return out
}

// usage: for v := range square(gen(2, 3, 4)) { ... } // 4, 9, 16

Each stage takes a receive-only input channel and returns a receive-only output channel. Stages run concurrently, and closing flows downstream.

Rate limiting with a ticker

A time.Ticker fires on a channel at a fixed interval. We read from it before each action to throttle ourselves.

limiter := time.NewTicker(200 * time.Millisecond) // 5 ops/sec
defer limiter.Stop()

for req := range requests {
	<-limiter.C // wait for the next tick before proceeding
	go handle(req)
}

Done-channel cancellation

When we want to stop a pipeline early, we pass a done channel into each stage and select on it. Closing done signals everyone to quit. (In real code, context usually plays this role — same idea, standardized.)

func square(done <-chan struct{}, in <-chan int) <-chan int {
	out := make(chan int)
	go func() {
		defer close(out)
		for n := range in {
			select {
			case out <- n * n:
			case <-done: // bail out early if cancelled
				return
			}
		}
	}()
	return out
}

Interview soundbite

The core Go concurrency patterns are: worker pools (N goroutines draining a shared jobs channel, coordinated with a WaitGroup), fan-out/fan-in (spread work across goroutines then merge results), pipelines (chained stages connected by channels where closing flows downstream), rate limiting with a ticker, and done-channel cancellation. The recurring discipline is: close channels from the sender side and propagate cancellation downstream so nothing leaks.

References