Storage Wars: File vs Block vs Object Storage

Data storage was simple once upon a time. Your structured data was neatly tucked in a database while employees would save individual files to the local file systems on their desktops. To say times have changed would be an understatement. In our post How Much Does File Storage Really Cost?, one section highlights unstructured data and how its exponential growth has overtaken the digital universe. That massive influx of documents, videos, emails, and web content has IT shops scrambling for a better way to store their informational assets. The technology gods have given us a few options to work with, but choosing isn't as easy as one might assume.

When it comes to architectures, there are basically three levels of storage to choose from: file, block, and object based storage systems. Let’s overview the basics, differences, and best use cases for each storage system level at our disposal.

Simple and Straightforward Storage at the File Level

File-based storage is the old school approach to storage. And like most older things, it's about as simple as it gets. You give files a name, tag them with metadata, then organize them in folders under directories and sub-directories. The standard naming convention makes them easy enough to organize while storage technologies such as NAS allow for convenient sharing at the local level. Many companies demand a centralized, easily accessible way to store files and folders. File level storage can deliver these perks at a cost that is typically affordable on a small business budget. With file level storage, you have a hierarchical system that excels at handling relatively small amounts of data. However, any IT admin with experience managing this sort of architecture can probably attest to its shortcomings. Sure, you can technically create and store an unlimited number of files. But simply finding those files is a real chore over time. The more files you accumulate, the bigger the headache. Flipping through dozens of folders. Scrolling through hundreds of files all to find that one you need. It's not fun. File Storage Use Cases Despite what it lacks, file-level storage makes sense for a wide variety of scenarios, including: File sharing: If you just need a place to store and share files in the office, the simplicity of file-level storage is where it's at. Local archiving: The ability to seamlessly accommodate scalability with a scale-out NAS solution makes file-level storage a cost effective option for archiving files in a small data center environment. Data protection: Combined with easy deployment, support for standard protocols, native replication, and various drive technologies makes file-level storage a viable data protection solution.

Performing at the Next Level with Block Storage

IT pros will deploy block storage most commonly in  SAN architectures. It is the natural alternative to file-based storage. A block is a raw storage volume filled with files that have been split into chunks of data of equal size. A server-based operating system manages these volumes and can use them as individual hard drives. That means they perform a number of functions native to individual OS platforms. For example, organizations often use supported third-party tools to manage and backup the data placed in block storage architectures. One of the most interesting things about block storage is how it handles metadata. Unlike in file-based architectures, there are no additional details associated with a block outside of its address. Instead, the controlling operating system determines how storage management. The system allocates storage for different applications, decides where data goes in the block, and makes provisions with tools that are not available to other types of storage. Under ideal conditions, this granular control helps create the high-performance environment block storage is so well known for. Block Storage Use Cases The unique ability to create volumes that essentially act as hard drives makes block storage useful for a wide range of applications, including: Databases: Block storage is common in databases and other mission-critical applications that demand consistently high performance. Email servers: Block storage is the defacto standard for Microsoft's popular email server Exchange, which doesn't support file or network-based storage systems. RAID: Block storage can create an ideal foundation for RAID arrays designed to bolster data protection and performance by combining multiple disks as independent volumes. Virtual machines: Virtualization software vendors such as VMware use block storage as file systems for the guest operating systems packaged inside virtual machine disk images.

Object Based Storage: A Complex Solution for Complex Storage Needs

As the name suggests, object-based storage stores data in isolated containers known as objects. You can give a single object a unique identifier and store it in a flat memory model. This is important for two reasons. You can retrieve an object from storage by simply presenting its unique ID, thus making information much easier to find in a large pool of data. The data could be physically stored on a local server, or a remote server thousands of miles away in the cloud. Thanks to the flat address structure, the object can be retrieved in the same fashion. Object based storage also lends much greater flexibility to metadata. For instance, you can customize metadata so it pairs objects with specific applications. You can also determine that application’s level of importance, move objects to different areas of storage, and even delete objects when you no longer need them. The sky is the limit and these endless customization possibilities are exclusive to object storage systems. You can argue that file and block-level systems offer better performance. But flexibility makes object storage systems far more superior in terms of being easy to manage. Keeping pace with explosive data growth is one of the biggest storage challenges IT teams face have in front of them. File and block-based architectures can both be expanded, but reliability will surely be tested as storage needs climb into scores of terabytes and petabytes. Scalability is where object-based storage does its most impressive work. Scaling out an object architecture is as simple as adding additional nodes to the storage cluster. Every server has its physical limitations. But thanks to location transparency and remarkable metadata flexibility, this type of storage can be scaled without the capacity limits that plague traditional systems.

Companies like Exablox have paved the way in building highly scalable systems while reducing costs and complexity for storage systems, so there's an option for every situation.

RESTful APIs Provide Easy Access

Access is another important factor in understanding what makes object based storage tick. While files and blocks are generally available to an operating system, object-level platforms typically rely on REST APIs for access. API applications such as Amazon Simple Storage Services and OpenStack Swift round out flexibility. They make objects accessible via HTTP and facilitate management functions related to authentication, permissions, and file properties. From uploads to deletion, these tools help optimize object storage system performance while streamlining administration at the highest level. The thing about REST APIs is they are not accessible to all storage systems and applications. This may be due to licensing issues that accompany proprietary software. Or you may be lacking programming skills needed to write the calls or functions into the application. On the bright side, there are alternatives in the form of conversion tools that allow you to access objects with common file protocols. These utilities make object based storage function like a traditional storage architecture without overhauling the file system or application.

Object Based Storage Use Cases

It's not an entirely new concept. But it would seem that object storage was tailor-made to tackle the humongous wave of data flowing through the modern day enterprise. Some ideal uses include: Big data: Object storage has the ability to accommodate unstructured data with relative ease. This makes it a perfect fit for the big data needs of organizations in finance, healthcare, and beyond. Web apps: You can normally access object storage through an API. This is why it's naturally suited for API-driven web applications with high-volume storage needs. Backup archives: Object storage has native support for large data sets and near infinite scaling capabilities. This is why it is primed for the massive amounts of data that typically accompany archived backups. All signs indicate that the digital universe is trending towards an IT environment more suited for object-based storage. With that said, block and even file-level solutions still have their place in today's storage landscape. Identifying your individual needs is the first step in determining which type of storage best fits your organization.