Block, file and object storage
Three ways to keep bytes, and why images, videos and backups end up in the third.
5 minute lesson
Three ways to keep bytes
- BlockA disk for one machinefast, and changed a few bytes at a time: databases
- FileA folder tree for several machinesshared directories: legacy applications, home folders
- ObjectWhole files by key over HTTPhuge, cheap and reachable from anywhere: media, backups
Block storage is what a database runs on. Object storage is where everything goes that is large and stored as a whole.
A key and its bytes
PUT /photos/2026/cat.jpg store the whole object
GET /photos/2026/cat.jpg fetch it
DELETE /photos/2026/cat.jpg remove itAn object store has a simple model. A bucket holds objects. Each object is a key, the bytes and a little metadata. The key may look like a path, but there are no real folders. You store, fetch and delete whole objects over HTTP, and the store grows as far as you care to pay for.
A database server changes small pieces of its data files thousands of times a second, and needs each change on disk within a millisecond.
Which kind of storage does it run on?
- Block storage: a fast disk of its own
- Object storage: whole files over HTTP
- File storage: a folder that many share
- No storage: it keeps all data in memory
Show the answer
- Block storage: a fast disk of its own
A database rewrites a few kilobytes here and there, constantly, and waits for each write. That needs a disk that it can address block by block with very low delay. Object storage would rewrite the whole file for every change, over HTTP. A shared folder adds a network file protocol between the database and its data.
Many small in-place changes with low delay need block storage. Databases live on it.
What it is good and bad at
Good at
write once and read often
- Size
- Objects of any size and any number
- Reach
- Any client with a URL
- Cost
- The cheapest storage per gigabyte
Bad at
small and frequent changes
- Editing
- A change rewrites the whole object
- Speed
- Slower per request than a local disk
- Searching
- Only by key or key prefix
A 2 GB log file is stored as one object. An application wants to append a single line to it.
What does that take?
- Uploading a whole new object in its place
- Sending the one line to the end of the file
- Opening the object and writing at its end
- Locking the bucket while the line is added
Show the answer
- Uploading a whole new object in its place
An object is replaced, not edited. There is no call that changes some bytes in the middle or adds to the end of an ordinary object. To append one line, the application uploads all 2 GB again with the line added. That is why logs are written to object storage as many small objects, one per hour or per batch.
Objects are written whole. Data that grows is stored as many objects, not as one that is appended to.
How does an object store organise data, and what kind of data suits it?
Show the answer
Buckets hold objects, and each object is a key, its bytes and some metadata. Objects are stored, fetched and deleted as a whole over HTTP, with no folders and no editing in place. That suits data which is large, written once and read many times: images, videos, backups, logs and the files of a website. It does not suit data that changes a little at a time.
Recap
- Block storage is a disk for one machine, file storage a shared folder tree, object storage whole files by key.
- An object is a key, its bytes and metadata in a bucket, reached over HTTP.
- Object storage suits large data that is written once. Changing a part means rewriting the object.