Wide-Column Stores

Wide-column stores look like tables on the surface, but they work very differently from relational databases. Each row can have a different set of columns, and the data is organized for fast writes and horizontal scaling.

The name “wide-column” comes from the fact that rows can have a huge number of columns — potentially millions. But we shouldn’t confuse them with traditional relational columns. Think of each row as a sorted map of column names to values.

The Big Names

Apache Cassandra — massively scalable, eventually consistent, peer-to-peer architecture (no single point of failure). Used by Netflix, Discord, Apple.

Apache HBase — built on top of Hadoop HDFS. Strongly consistent, modeled after Google’s Bigtable paper. Used for random, real-time read/write access to big data.

Google Bigtable — the original, managed service on Google Cloud. Powers Google Search, Gmail, Maps, etc.

Core Concepts

Column Families

A column family is like a table, but each row can have different columns. Columns within a family are stored together on disk, which makes reading a family’s data efficient.

// Column Family: user_profile
Row Key     | name           | email                  | age  | city
------------|----------------|------------------------|------|--------
user:001    | "Manish"       | "manish@example.com"   | 25   | "Mumbai"
user:002    | "Priya"        | "priya@example.com"    |      | "Delhi"
user:003    | "Rahul"        |                        | 30   |

Notice how user:002 has no age and user:003 has no email or city. That’s fine — there’s no schema enforcing that every row must have every column.

Partition Key + Clustering Key

The partition key determines which node stores the data. All rows with the same partition key live on the same node.

The clustering key determines the sort order within a partition.

This is the most important design decision in Cassandra. We design our keys based on how we query the data.

-- Cassandra CQL (looks like SQL but isn't)
CREATE TABLE sensor_readings (
    sensor_id TEXT,           -- partition key
    timestamp TIMESTAMP,      -- clustering key (sorted)
    temperature DOUBLE,
    humidity DOUBLE,
    PRIMARY KEY (sensor_id, timestamp)
);

-- This is fast: reads all data for one sensor, sorted by time
SELECT * FROM sensor_readings
WHERE sensor_id = 'sensor-42'
AND timestamp > '2024-01-01';

-- This is SLOW or impossible: scanning across all sensors
-- SELECT * FROM sensor_readings WHERE temperature > 30;
-- Cassandra can't efficiently do this without scanning every partition

Write-Optimized

Wide-column stores are designed for insane write throughput. Here’s how:

Every write is an append. Cassandra doesn’t update data in place. It writes to an in-memory table (memtable), then flushes to disk as immutable sorted files (SSTables). Old versions are cleaned up later during compaction.

This means writes are always fast — there’s no “find the row and update it” step. We just append.

-- Inserting data (CQL)
INSERT INTO sensor_readings (sensor_id, timestamp, temperature, humidity)
VALUES ('sensor-42', '2024-03-15 10:30:00', 25.5, 65.0);

-- "Updating" is just inserting with the same key — newer timestamp wins
INSERT INTO sensor_readings (sensor_id, timestamp, temperature, humidity)
VALUES ('sensor-42', '2024-03-15 10:30:00', 26.0, 64.0);

Eventual Consistency

By default, Cassandra is eventually consistent. When we write data, it goes to one node first and then replicates to other nodes. For a brief moment, different nodes might have different versions.

We can tune this with consistency levels:

-- Write to a quorum of nodes (majority must acknowledge)
CONSISTENCY QUORUM;
INSERT INTO sensor_readings (...) VALUES (...);

-- ONE: fastest, least consistent
-- QUORUM: good balance
-- ALL: slowest, most consistent (all replicas must respond)

Use Cases

Wide-column stores excel at:

• Time-series data — sensor readings, metrics, stock prices (partition by device/symbol, cluster by time)
• IoT data — millions of devices writing data continuously
• Logging at scale — billions of log entries that need fast writes
• User activity feeds — partition by user, cluster by timestamp
• Messaging systems — Discord uses Cassandra to store billions of messages

They struggle with:

• Ad-hoc queries (we must know our access patterns upfront)
• Multi-partition queries (aggregating across all sensors)
• Strong consistency requirements (possible but costs performance)
• Small-scale applications (the overhead isn’t worth it for small data)

In simple language, wide-column stores are built for massive write throughput and horizontal scaling. We design the data model around our query patterns, not around the data relationships. If we’re writing millions of events per second and reading them back by a known key, this is the right tool. For anything that needs flexible querying or strong consistency, stick with a relational database.