Gyaan

Memory Management

advanced redis memory performance ops

Redis lives in RAM. When RAM runs out, bad things happen — either Redis stops accepting writes or the OS kills the process. Understanding memory management is the difference between a stable production deployment and 3 AM pages.

In simple language: you tell Redis how much memory it can use, what to do when full, and then watch for the keys that are eating disproportionate space or CPU. That’s the job.

maxmemory and eviction policies

maxmemory caps Redis’s memory usage. When hit, the configured policy decides what happens:

maxmemory 4gb
maxmemory-policy allkeys-lru

Policies fall into three families:

PolicyWhat it does
noevictionReject writes with OOM error (default). Reads still work.
allkeys-lruEvict least recently used across all keys
allkeys-lfuEvict least frequently used across all keys
volatile-lruLRU but only among keys with a TTL
volatile-lfuLFU but only among keys with a TTL
volatile-ttlEvict keys closest to expiring
allkeys-random / volatile-randomRandom eviction

For pure caches: allkeys-lru or allkeys-lfu. For mixed cache + persistent data: volatile-* so the data without TTL is never evicted.

Note: Redis’s LRU/LFU are approximate. It samples 5 keys (configurable via maxmemory-samples) and evicts the worst — way faster than tracking everything globally.

What happens at OOM

Memory hits maxmemory
noeviction
Writes → OOM error
Reads → fine
Risk: app errors flood
allkeys-lru/lfu
Old keys evicted on demand
Writes succeed
Risk: silent cache miss
No maxmemory set + RAM exhausted → OS OOM killer terminates redis-server.

If you don’t set maxmemory, Redis will happily consume everything until the Linux OOM killer steps in and SIGKILLs the process. Always set maxmemory in production.

Big keys: the silent killer

A “big key” is a single key holding a huge value or collection — multi-MB strings, lists with millions of items, hashes with tens of thousands of fields. They cause:

  • Slow DEL (O(N) free, blocks Redis). Use UNLINK instead — async free.
  • Slow migrations during cluster resharding.
  • Long blocking commands like HGETALL, SMEMBERS, LRANGE 0 -1.
  • Lopsided cluster shard load.

Find them:

# Sampling - safe in production
redis-cli --bigkeys

# Or per key
MEMORY USAGE user:42
DEBUG OBJECT user:42

Fix by sharding the value yourself: instead of one giant hash posts:all, use posts:bucket:0, posts:bucket:1 etc., or break by user/time.

Hot keys: the other silent killer

A “hot key” is one that gets disproportionately many requests. In single-instance Redis it just causes contention. In Cluster mode, the whole shard owning it becomes a bottleneck — and you can’t scale around it just by adding nodes.

Find them:

redis-cli --hotkeys     # requires LFU policy
MONITOR | head -1000    # CAUTION: huge performance hit, never long-term

Fixes:

  • Client-side cache the hot key (Redis 6+ supports server-assisted client caching).
  • Replicate reads to replicas if it’s read-heavy.
  • Shard the key if writes are heavy — split into counter:1, counter:2, etc., aggregate on read.

OBJECT ENCODING — how Redis stores stuff

Internally Redis picks compact encodings for small data structures, then switches to general ones when they grow.

SET small "hello"
OBJECT ENCODING small      # "embstr" - compact embedded string

SET big "long string > 44 chars..."
OBJECT ENCODING big        # "raw"

HSET user:1 name "M" age 30
OBJECT ENCODING user:1     # "listpack" (small hash)

# Add many fields...
OBJECT ENCODING user:1     # "hashtable" once it crosses thresholds

Thresholds are configurable:

hash-max-listpack-entries 128
hash-max-listpack-value 64
list-max-listpack-size -2
set-max-listpack-entries 128
zset-max-listpack-entries 128

Why care? Small hashes/lists/sets/zsets are dramatically more memory-efficient. A “hash of small hashes” pattern lets you store millions of items in a fraction of the RAM compared to flat keys.

Classic example: instead of user:1:name, user:1:age, …, store everything under user:1 as a hash. Or shard further: users:bucket:42 holds 100 user hashes — each lookup is HGET users:bucket:42 1.

Memory analysis tools

INFO memory                 # used_memory, peak, fragmentation ratio
MEMORY STATS                # detailed breakdown
MEMORY USAGE key            # bytes for one key
MEMORY DOCTOR               # heuristic advice
redis-cli --memkeys         # sample biggest keys by memory

mem_fragmentation_ratio > 1.5 suggests fragmentation. activedefrag yes enables online defrag (with CPU cost).

Production checklist

  • Always set maxmemory and a sane eviction policy.
  • Run --bigkeys regularly; alert on any single key over a threshold.
  • Monitor hot keys; design schema to avoid concentrating writes.
  • Use UNLINK for large deletes, never DEL on suspect keys.
  • Tune hash-max-listpack-* and friends for your data shape.
  • Watch fragmentation ratio; enable active defrag if needed.

Quick recap

  • maxmemory + eviction policy = OOM protection.
  • LRU/LFU are sampled approximations.
  • Big keys block; hot keys bottleneck shards.
  • OBJECT ENCODING reveals internal layout. Small structures use compact encodings.
  • UNLINK > DEL for large values.

References