Block, File and Object Storage: Which You Actually Need

One of the most common and most expensive mistakes in infrastructure is choosing a storage type by habit, by whatever the last array did, or by whatever a vendor happened to lead with. The three types are not interchangeable. They sit at different points on the trade off between speed, sharing and scale, and the right one is decided by the workload, not the logo on the array.

So before any conversation about Dell, NetApp, Pure or anyone else, the first question is which of these three you actually need, and for which data. Get that right and the vendor choice becomes far easier. Get it wrong and no amount of premium hardware will rescue the design.

Start with the workload, not the protocol

Block, file and object are all just ways of presenting the same underlying capacity to whatever is consuming it. What separates them is the unit they hand out and the path the data takes. That unit decides the latency, the way data is shared, and how far the system can scale. So the honest way to choose is to start from the workload and let it pull you toward the right protocol, rather than starting from a protocol you already own and bending the workload to fit.

Block storage: the database and virtual machine workhorse

Block storage presents raw volumes to a host, which then formats them with its own file system and treats them as if they were local disks. There is very little between the application and the media, which is exactly why block delivers the lowest and most predictable latency of the three. This is the storage that databases, virtual machine datastores and any latency sensitive transactional system want underneath them.

Block is carried over a storage network, historically Fibre Channel and increasingly Ethernet using iSCSI or NVMe over Fabrics. That network, the SAN, is what people are really talking about when they say SAN storage. The strength of block is performance and control. The cost is that a block volume is typically owned by one host or cluster at a time, so block is not how you share files between many users.

File storage: shared access for people and applications

File storage presents a shared file system over the network that many clients can mount and use at the same time. It speaks the protocols people already know, NFS in the Linux and virtualisation world and SMB in the Windows world. The storage itself manages the directory tree, the permissions and the locking, so clients simply read and write files as if the share were local.

This is the natural home for anything genuinely shared: user home directories, departmental shares, application data that several servers read at once, media and engineering working files, and many virtualisation datastores presented over NFS. File storage trades a little of the raw performance of block for the enormous convenience of shared, concurrent access with familiar permissions. Network attached storage, or NAS, is simply the common name for storage that serves files this way.

Object storage: scale, durability and cheap capacity

Object storage takes a completely different shape. Instead of volumes or a file tree, it holds data as objects in a flat namespace, each with a unique identifier and its own metadata, reached over HTTP using the S3 protocol that has become the de facto standard. There is no traditional file system in the path, which is what lets object storage scale to billions of objects and petabytes of capacity on commodity hardware, with durability built in by spreading copies or erasure coded fragments across many nodes.

The trade off is latency. Object storage is built for throughput, scale and cost, not for fast random reads and writes. You would not put a transactional database on it. But for the right data it is unbeatable on cost per terabyte and on resilience. The classic fits are backup and archive targets, media and content libraries, log, sensor and analytics data, and cloud native applications that were written to speak S3 from the start.

Where people get it wrong

Most storage regret traces back to one of a few familiar mismatches. Each is easy to avoid once you have seen it.

• Object for transactional work. Object storage is cheap and durable, which tempts teams to push latency sensitive applications onto it. The applications then crawl, because object was never built for fast random access. The saving evaporates into a performance problem.
• File for absolutely everything. A NAS is convenient, so it quietly becomes the home for databases and virtual machines that would have been faster and more predictable on block. It works until it does not, usually under load at the worst possible moment.
• Block where file would be simpler. Carving up block volumes and layering a clustered file system on top to share data between hosts, when a file share would have done the same job with a fraction of the complexity.
• Buying the protocol you already own. Extending the incumbent platform because it is there, rather than because it fits the new workload. This is how estates drift into the wrong shape one project at a time.

How modern arrays blur the lines

To complicate the neat picture, many modern platforms are unified. They present block and file from the same system, and a growing number add object too, so a single array can serve a database over block, a department share over SMB and a backup target over S3 at once. That is genuinely useful. It simplifies operations, consolidates the estate and reduces the number of things to manage and support.

But unified hardware does not repeal the rules. A single array that can speak object does not make object suitable for your transactional database. The protocol still has to match the workload. The right way to use a unified platform is to choose each protocol on its workload first, and only then ask whether one platform can serve several of them well. Consolidation is a benefit you collect after the design is right, not a substitute for getting it right.

How to decide, in practice

You do not need a long study to get this right. For each significant workload or data set, walk three questions.

• How latency sensitive is it? Transactional and performance critical data points to block. Tolerant, throughput oriented data opens up file and object.
• How is it shared? Owned by one host or cluster suits block. Read and written by many clients at once suits file. Reached by applications over HTTP at scale suits object.
• How big does it get, and how cheap must it be? Modest and performance led stays on block or file. Very large, growing and cost sensitive moves toward object.

Run every workload through those three and the estate sorts itself into a sensible shape. Primary databases and virtual machines on block, shared and collaborative data on file, bulk unstructured and backup and archive on object. Then, and only then, you look at which platforms can serve those tiers well, and how few of them you can get away with.

Not sure which storage your workloads actually need?

Send us your workload mix or your current situation and we will give you an independent, vendor neutral view: what belongs on block, file and object, where the estate has drifted out of shape, and how to consolidate it sensibly. We have designed and run all three at scale.

Name *

Company *

Work email *

Phone (optional)

Details

Send us your situation

Prefer email? Reach us directly at hello@c4cgroup.co.uk.

Frequently asked questions