Key-value and document stores
Stores that give up joins: what they can look up, what they cannot, and how to shape data for them.
6 minute lesson
A dictionary the size of a data centre
SET session:9f2c "user 42, cart 3 items"
GET session:9f2cA key-value store does two things: store a value under a key, and fetch it by that key. It does not look inside the value. Because every key stands alone, lookups are very fast, and the keys are easy to spread over many machines.
A key-value store holds a million user profiles, each under a key such as user:42. Somebody wants all users who live in Berlin.
How can the store answer that?
- Only by reading every value, or from a list you kept
- With a query that filters the profiles by their city
- By joining the profiles with a table of the cities
- By asking for the key that is named after the city
Show the answer
- Only by reading every value, or from a list you kept
The store knows keys. What is inside a value is none of its business, so it cannot filter by city. Either the application reads all million profiles and looks, or it has maintained a second entry all along, such as a key city:berlin that lists the user ids. Every extra way of finding data is something the application builds and keeps correct itself.
A key-value store finds by key only. Any other lookup you build and maintain yourself.
Values the database can read
{
"_id": "order_981",
"customer": { "id": 7, "name": "Ana" },
"items": [
{ "sku": "A12", "qty": 2 },
{ "sku": "B07", "qty": 1 }
]
}A document store also stores a value under a key, but the value is a document, usually JSON, and the database understands it. It can search and index fields inside. Documents in one collection need not have the same fields.
Store together what is read together
There are no joins, so the model follows the screens: what one read needs goes into one document. The order above embeds its items.
Embed
inside the document
- Reading
- One fetch gets it all
- Changing shared data
- Every copy must change
- Fits
- Parts that belong to one whole
Reference
by id
- Reading
- A second fetch
- Changing shared data
- One place
- Fits
- Things shared by many
A blog shows each post together with its comments, of which there are rarely more than twenty. Comments are never shown anywhere else.
How should a document store hold them?
- The comments embedded in the document of their post
- Each comment as a document that refers to its post
- All comments of the blog together in one document
- Each comment copied into its author's document
Show the answer
- The comments embedded in the document of their post
The post and its comments are always read together, the comments belong to exactly one post, and their number stays small. Embedded, one fetch returns the whole page. Separate documents would cost a second fetch on every view and gain nothing, because no screen needs a comment without its post.
Embed what is read together, belongs to one parent and stays small.
Which shape for which data
- Key-valueLookups by one keysessions, carts, counters, flags
- DocumentSelf-contained objectscatalogue entries, profiles, articles
- RelationalConnected data and open questionsorders, payments, anything with rules
What can a document store do that a key-value store cannot, and what does it still lack compared with a relational database?
Show the answer
It understands its values: documents with fields, which it can search and index, so data can be found by more than its key. Documents in a collection may differ in their fields. What it lacks is joins and the rules between tables. Data that belongs together is embedded in one document, and data shared by many documents is either copied or fetched separately.
Recap
- A key-value store finds a value by its key and by nothing else. It is fast and easy to spread out.
- A document store understands its values and can search their fields. The shape may vary.
- Without joins, model for the reads: embed what belongs to one whole, reference what is shared.