Time for Flash to Come Out of the Ivory Tower

The time of year has come when students and their parents anxiously await and receive results from the college of their choice.  In academia, the ivory tower is an exalted place. The thought of your college student getting admitted into a prestigious Ivy League University is an exciting one, even more so when the acceptance letter finally arrives.

In the world of technology, all-flash storage is seen as being in the “ivory tower” and still the purview of high-end primary applications and larger companies. Unfortunately, this means flash storage has been limited to a fairly small set of use cases with a small footprint in the middle market and small market. Many enterprise flash array vendors still stick to the old business model of charging the premium price for their flash storage medium  (SATA / SAS SSDs, or NVMe) to build out their margins. After all, the same model has worked for the vendors  for many years with hard drives. However, the truth is that the Intellectual Property (IP) for building enterprise storage capabilities is in software, not in the flash medium itself. It is time to value the components of a storage system appropriately for customers. Shouldn’t customers be able to buy flash drives at the same retail prices as storage vendors themselves when those vendors are not adding any value to the standard off the shelf flash drive? Most of the innovation in flash media is done by the drive manufacturers, be it with NVMe, SATA/SAS, or m.2 form factors. The other factor to consider is the remarkable growth in SSD capacities every quarter. Customers who pay a premium for their flash drives find that those drives are now superseded by the next generation of drives in a quarter. Most storage vendors ride this cycle of innovation to provide better performance or lower cost, but don’t pass on the benefits to customers. Some hyper-converged vendors don’t charge a premium for the flash in servers customers already have in their data center. However, this model comes at the cost of adding overhead to the compute and memory requirements on the server. We’ve yet to see a solution that offers a disaggregated compute and storage server with enterprise storage capabilities while allowing customers the flexibility to purchase the flash drives of their choosing. The typical reason vendors are not able to offer the “bring your own drive” business model is not bottom line alone; it’s also because their product technically cannot support such a business model. To support this kind of a model, the technology should be able to recognize newly added drives and ingest the new capacity on the fly, rebalance the underlying data for optimal performance and support mix of different drives sizes in the same enclosure.  The StorageCraft OneBlox all-flash scale-out appliance utilizes the object storage architecture and the platform features to enable adding drives and nodes to the cluster at run time. Some of the key design points in enabling this is the following:

• Fine grained objects with a proportional distributed hash space across the cluster allow for both mix and match of drives to co-exist as well as provide robust performance by relaying out the objects across the new set of drives.
• Auto discovery of drives that are added and ingesting the new capacity in the file system immediately without a long format phase with no interruption to the name space is another key enabler
• Data structures that self-encapsulate flash drives of varying capacities, which obviates the need to change the memory footprint when adding drive capacity
• Parity bits are stored as part of the OneBlox storage data format to handle heterogeneous flash medium some of which could have higher rate of uncorrectable bit errors (MLC and TLC NAND flash, for example).

Without the above technologies, it is not possible to offer an enterprise-class storage solution that provides customers the flexibility of adding their own flash drives of varying capacities over time. And without this ability to dynamically provision additional flash drives at drive vendor pricing, enterprise class all-flash storage will remain the privy of the few.