Metadata in the database, bytes through a CDN
The usual shape of file handling in an application: a database row that describes each file, an object that holds it, and a CDN that delivers it.
5 minute lesson
The file and the facts about it
A blob is a file as the storage sees it: a run of bytes under a key. Your application needs more. Who owns it? What is it called? Is the upload finished? An object store cannot answer such questions for millions of files. A database can.
- Databaseone row per fileowner, name, size, type, status and the key
- Object storagethe bytesfound by the key in the row
A row for every file
CREATE TABLE photo (
id uuid PRIMARY KEY,
owner_id uuid NOT NULL,
object_key text NOT NULL,
size_bytes bigint,
content_type text,
status text NOT NULL
);
The row is written first, with the status pending. When the upload has arrived, the status becomes ready. Rows that stay pending point at uploads that never finished, and a clean-up job can remove them.
An app stores 80 million photos in a bucket. A screen has to show one user's photos, newest first.
Where does the list come from?
- From database rows that hold each photo's key
- From a listing of every object in the bucket
- From the CDN's record of what it has cached
- From tags that are read from each object
Show the answer
- From database rows that hold each photo's key
By owner, newest first, is a query. A database answers it from an index in milliseconds. The bucket can only list keys by prefix, and the CDN only knows what somebody happened to request. The rows give the keys, and the keys give the photos.
Questions about files are answered by the database. The object store only delivers bytes by key.
Let a CDN hand out the bytes
- First
A user in Sydney asks for a photo. The CDN location near them does not have it yet, so it fetches the object from the bucket and keeps a copy.
- Userasks
- CDNhas no copy
- Bucketsends it
- Next
The next users nearby get the copy from the CDN. The request never reaches the bucket.
- Usersask
- CDNanswers itself
- Result
The bytes travel a short distance, and the bucket serves each object a few times instead of millions of times.
- Usersfaster
- Bucketfew requests
A new version gets a new key
A CDN keeps its copy for as long as it was told to. If you replace an object under the same key, users keep getting the old copy until it expires.
- Same keyavatars/881.jpgthe old picture stays in caches
- New keyavatars/881-v7.jpga new address with no old copies
A user uploads a new profile picture. It replaces the object avatars/881.jpg, which a CDN caches for a day. Other users still see the old picture.
What prevents this in future?
- Storing each new picture under a new key
- Uploading each new picture a second time
- Keeping the pictures in a second bucket
- Making the pictures smaller than before
Show the answer
- Storing each new picture under a new key
The CDN has a copy under that address and was told it is good for a day. It has no reason to ask again. A new key is a new address that no cache has seen, so the first request fetches the new picture. The database row points at the current key, and the old object can be deleted later.
Never change an object that caches hold. Store the new version under a new key.
An application stores user files. What goes into the database, and what goes into object storage?
Show the answer
The bytes go into object storage, under a key. The database gets one row per file with the facts the application asks about: the owner, the name, the size, the type, a status and the object's key. The application searches and checks permissions in the database, and then fetches or links the object by its key. A status such as pending and ready shows which uploads really arrived.
Recap
- Keep a database row for every file: the facts and the key. Keep the bytes in object storage.
- Put a CDN in front of the bucket. It serves copies near the users and takes the load off the bucket.
- Give each new version a new key, so that no cache holds an old copy of it.