Storage Follows Computing

Storage trends tend to follow computing. As computing becomes increasingly virtualized, storage must become much simpler to use and with features that work much better with virtualization.

In 2009, virtualization crossed an important threshold. According to IDC, this was the first year that new virtual server deployments exceeded physical server deployments. For every physical server sold, more than one new virtual machine (VM) was created. This trend is accelerating; average VM-to-server consolidation ratios range from 6:1 to as high as 10:1. Virtualization has rapidly become the de facto approach to server deployment. But virtual and cloud-based infrastructures consolidate workloads, which in turn quickly overwhelm conventional disk-based architectures.

In traditional physical environments, read/write access is typically less than half the overall workload on a shared storage system. Consequently, high-performance file servers are tuned to accommodate a high proportion of very small meta-data operations.

However, in a virtual environment, read/write access completely dominates the workload for shared storage. In addition, these read/write operations are nearly all completely random. This puts massive stress on a conventional hard-disk storage system.

The storage system is the most expensive component to purchase and the most costly to maintain; storage is also the source of most management and performance problems. New storage networking models are attempting to address this bottleneck.

Storage is nearly always the slowest component in a virtualized environment, with typical latencies in the tens of milliseconds. But a properly implemented NAND flash-based storage system can dramatically improve storage networking latency, typically moving the bottleneck back to the host server.

Solid-state NAND flash is more than just a fast disk replacement. Flash eliminates a key mechanical barrier to scaling. Flash-based products will not only be faster, but eventually much simpler and provide better functionality than today’s disk-based systems.

While still far from being competitively priced with high-capacity serial-attached SCSI/Serial ATA (SAS/SATA) hard disk drives, IDC reported flash storage prices have dropped from about $21 per GB five years ago to less than $2 per GB for multilevel cell (MLC) flash today.

Relentless improvements in CPUs and networking are increasing the pressure on storage performance. But other than steady capacity growth, traditional shared storage is famously resistant to technological change.

Big data, virtualized applications and business-critical applications are beginning to fragment outside the classic network architecture. Legacy-storage systems cannot address these different requirements without expensive bolt-on additions.

From Fusion-io’s initial public offering in 2011 to a host of venture-backed startups that have launched over the past year, there is no doubt that the big players who have owned the storage world for the last two-plus decades will have some fresh competition, or acquisition targets, in the not-so-distant future.