Wide-column, graph and time-series stores
Three specialised shapes: tables read one partition at a time, data that is all relationships, and measurements over time.
6 minute lesson
Three shapes for three kinds of question
- Wide-columnHuge tables read by one keymessages of a chat, events of a device
- GraphThe relationships are the datawho knows whom, what depends on what
- Time-seriesMeasurements over timemetrics, prices, sensor readings
Each gives up generality to be very good at one kind of question. None of them replaces a relational database for everything else.
Rows that live together
A wide-column store spreads a table over many machines by a partition key. All rows with the same key sit together on one machine, sorted by a second column. A query names the key and reads that one partition.
- Partition keychat 42
- One partitionits messages by time
A chat application always asks one question: the latest messages of a given chat. It stores them in a wide-column store.
How should the table be keyed?
- Partition by chat id, sorted by the time sent
- Partition by time sent, sorted by the chat id
- Partition by message id, in no special order
- Partition by the sender, sorted by message id
Show the answer
- Partition by chat id, sorted by the time sent
The query names a chat and wants its newest messages. With the chat id as partition key all messages of one chat sit together on one machine, and sorted by time the newest are read from one end. Any other key scatters a chat's messages over the cluster, and reading them means asking many machines.
In a wide-column store the partition key is what the query names, and the sort order is how it reads.
When the links are the data
A graph database stores nodes and the edges between them. A question is a walk: start at Ana, follow "friend" edges twice, and you have the friends of her friends. In a relational database each step is a join over the whole friendship table.
- Anastart
- 1 hopher friends
- 2 hopstheir friends
Points on a time line
A time-series database stores points: a time, a value and labels that say what was measured. New points are only ever appended. Queries ask for a range of time and a summary of it: the average CPU load of the last hour, per server.
- Appends only. Nobody edits the temperature of last Tuesday.
- Recent data matters most. Old points are thinned out or dropped after a set time.
- Every combination of labels is a series of its own. A label with millions of values, such as a user id, creates millions of series.
Ten thousand machines report their temperature every ten seconds. Dashboards show averages per hour, and nothing older than 90 days is needed.
Which kind of store fits?
- A time-series store
- A graph database
- A document store
- A key-value store
Show the answer
- A time-series store
The data is a stream of timestamped values that is only appended. The questions are ranges of time and averages over them, and old data expires. That is exactly the workload a time-series database is built around: cheap appends, fast summaries and automatic removal of old points.
Timestamped values, appended, summarised by range and expired by age: a time-series database.
What kind of question is a graph database built for, and why is that question awkward in tables?
Show the answer
Questions that follow relationships over several steps: friends of friends, everything that depends on a part, the shortest path between two people. A graph database walks from a node along its edges. In tables every step is a join, and the cost grows with each additional step.
Recap
- Wide-column: rows grouped by a partition key and sorted within it. Design one table per query.
- Graph: nodes and edges, for questions that follow relationships over several steps.
- Time-series: appended points with a time, a value and labels, queried by ranges and summaries.