eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Enterprise storage might look like a problem thats been solved in principle, requiring only continued refinement of proven technologies combined with the careful management thats needed to minimize costly data glut. IT system builders must look ahead, however, to the dead end on the path of hard disk evolution and start to think now about other pathways that can be explored as storage volumes inexorably grow.
Intelligent choice among storage technologies involves a careful balance among four distinct, often-contentious characteristics: total capacity, access speed (for both random and sequential access, depending on the application), physical density and storage media lifetime.
Improvement in any one of these attributes will often come at the expense of others. Emerging alternatives to magnetic disk have complementary strengths and weaknesses. Every enterprise needs its own carefully tailored portfolio, meeting not only volume requirements but also operational and even regulatory needs.
A study by the University of California at Berkeley in 2000 estimated the worlds production of new bits to store as being on the order of 250MB per person per year and projected that this would almost double every year. The same groups follow-up study in 2003 confirmed that per-capita data production two years later had more than tripled to at least 800MB per year, with the volume of data on the Web alone comprising 17 times the volume of the Library of Congress print collection.
To estimate future storage needs, by simple extrapolation of the exponential trend, is cause for amusement or despair. Sometime before 2100, one would conclude, the entire crust of Earth will be representing data at a (purely theoretical, so far) density of one bit per atom. Parsimonious data retention policies, as well as aggressive rich-media compression algorithms, will have to be thrown at this problem, along with every possible improvement in storage technology.
The search for greater real-world storage density, meanwhile, is rapidly taking engineers into the realm of nanoscale devices—devices whose dimensions are on the order of nanometers, or millionths of a millimeter. IBM, for example, is already fabricating electromechanical storage systems whose probes are only 10 nm in diameter. A human hair is about 6,000 times as coarse. Even in current hard disk assemblies, component dimensions and tolerances that are on the same order of size as individual atoms are already becoming the rule.
Enterprise IT buyers are most concerned, at least directly, with finances rather than physics. For the past two decades, dollar-denominated measures of storage performance have looked pretty good. Specific figures vary hugely, depending on the level of device packaging that one uses for purposes of estimating costs, but the downward trend of storage cost “is clearly exponential,” said Steve Gilheany, senior systems engineer at document management consultancy ArchiveBuilders.com, in Manhattan Beach, Calif.
Gilheanys year-2000 analysis, combining historical trends and projected technology developments, projected the price of desktop-grade magnetic data storage at about 2 cents per gigabyte in 2010. Mainframe-grade devices would sell for approximately 12 times that cost, he estimated, with intermediate grades of equipment at varying levels of speed and RAID protection sprinkled along the continuum in between.
If anything, Gilheany noted in a discussion this month with eWEEK Labs, the rate of price decline had already been accelerating during the most recent years from which his cost-trend data was taken. It now appears to us that the recent explosion of consumer-device demand has steepened the decline in magnetic storage costs still further. Gilheany, in 2000, projected a cost in 2004 of 77 cents per gigabyte, but a 250GB drive is already available as of this writing for as little as 55 cents per gigabyte.
At the risk of ruining a good party, however, the scientists have some truly bad news. Barring the development of downsized atoms, or the multivendor approval of a new set of physical laws, the shrinkage of magnetic storage has a hard stop at around 60GB per square inch. Attempting to store bits more densely, theorists believe, will be futile—random reorientation of magnetic domains will bury data in the resulting noise.
Its not yet possible to declare the question closed, because tuning the magnetic properties of materials can sometimes seem more like alchemy than like conventional materials science. The fabrication techniques available to storage-device designers include the construction of multilayer films that are literally just a few atoms thick, allowing intricate combinations of materials with sometimes surprising results.
Past experience does suggest, however, that a stable magnetic storage medium may also be a sluggish one: If a material tenaciously maintains its magnetic state despite the forces of random disturbance, it may also be hard to change that state on purpose—slowing the materials response in data-rewriting operations.
Page Two
This, therefore, raises the question of maintaining a balance between storage volume and storage bandwidth. Even if one could store, hypothetically, a terabyte of data on one magnetic platter the size of a quarter, the resulting device would require a blistering data transfer rate of 776M bps (pushing the next-generation limits of the IEEE 1394b FireWire standard) just to back it up in a leisurely 3 hours.
By 2010, even consumers might sneer at this proposition. In what one might imagine as a video version of Apple Computer Inc.s iPod, for example, the storage device just described would hold only about 100 hours of MPEG2-compressed high-definition TV content. Before anyone dismisses such a product concept as fanciful, note that desktop hardware vendors are already paving the way. Sony Corp., for example, announced this month a desktop system with four 250GB hard drives: two for normal applications and two more dedicated to TV program recording with a capacity of 19 hours per day for seven days on six concurrent channels.
Clearly, new frontiers must be found.
If bits per unit area are disks fundamental physical limit, it seems logical to explore the storage of data in a volume instead of on a surface. Thats the essential attraction of holographic storage, which creates a distinct optical interference pattern at the intersection of two interacting laser beams.
With suitable hardware, data can be read or written simultaneously at multiple locations within a single medium. Without the inertia of a magnetic drive heads mechanical actuators, the agility of the massless laser beams enables high performance during sustained random-access operations.
Moreover, the nature of the holographic interference process enables new modes of associative retrieval, effectively measuring the similarity—at many points throughout a volume of an optical storage medium—to a given reference pattern. The best-match value can then be retrieved, without prior knowledge of its location and without need for time-consuming, item-by-item examination.
This doesnt merely change the economics of storage; it also suggests fundamentally different approaches to application development models in business intelligence, homeland security and other domains of considerable interest to enterprise and government IT planners.
After decades of anticipation, holographic technologies are condensing from vapor to reality: InPhase Technologies Inc. plans to deliver its pioneering Tapestry 200-R holographic storage units next year, with each providing 200GB of recordable (but not yet rewritable) storage at a rate of 20MB per second and a removable-media cost of 25 cents per gigabyte—about twice the current cost of recordable DVD.
The road map of this Lucent Technologies Inc. venture forecasts the offering of rewritable media by 2008, with data rates of 40MB per second. Planned capacity expansion, going hand in hand with improved data throughput, could yield a 1.6TB recordable device operating at 120MB per second by 2010.
Marking a much greater departure from electromagnetic storage traditions is IBMs microelectromechanical Millipede technology—in effect, a fantastically scaled-down computer punch card.
A Millipede drive moves a brushlike apparatus, supporting a grid of several thousand submicroscopic probes in two dimensions across a film of thermoplastic material on a silicon foundation thats only a few millimeters square. Each of the 10-nm probe tips remains within a neighborhood about 100 microns square, minimizing power-consuming movement of the probe grid. Each probes neighborhood can store thousands of rewritable bits by forming tiny pits in the plastic, or reforming its flat surface, using the electrically wired probes local application of heat and pressure.
Each probe tip can achieve a rewritable data rate of well over a megabit per second, with a storage medium lifetime of at least 100,000 cycles at any given storage location. By using thousands of independently addressable probe tips in each device, Millipede offers highly scalable parallel data access. Sources close to the project have suggested that a 4,096-tip probe may already be functioning in a demonstration device, with overall data transfer rates on their way toward a target of 800G bps.
Next page: Data transfer rate vs. power consumption.
Page Three
Thanks to the intrinsic stability of a mechanical recording medium, IBM has already achieved a Millipede storage density of 125GB per square inch—20 times the current density of magnetic devices and double the predicted theoretical limit of magnetic media.
As a mechanical system, Millipede also offers product designers a direct trade-off between data transfer rate and power consumption. IBM has said that a device based on this technology, with data capacity on the order of 40GB to 80GB and other characteristics competitive with current flash memory units, could be ready to come to market in an SD (Secure Digital)-compatible form factor by 2006 if an overall product road map can be satisfactorily defined.
Enterprise-scale devices have yet to be discussed but seem to eWEEK Labs to be a logical extrapolation of the concept—high density, long lifetime and low power consumption for low-data-rate applications such as offline archival are a compelling set of characteristics.
Familiarity should not breed contempt for the venerable technologies of magnetic tape and solid-state memory. These have long held down opposite ends of the storage spectrum: tape with low cost but with low data throughput to match, solid-state memory with far superior speed but at enormous cost compared with other bulk-storage options.
Storage industry trade associations agree that, by 2006, hard disk storage will cost only 10 times as much per unit of capacity as tape—a significant narrowing of the fortyfold cost advantage that tape had in 1998. This gap will probably not close much further thereafter, however, based on current technology road maps. The hard drive road map, furthermore, will be nearing a probable dead end in terms of further density improvement, while tape manufacturers are not as dangerously close to their magnetic density limits.
Those with tape experience will know, unfortunately, that mechanical rather than magnetic phenomena are more important to long-term tape performance. Both tape manufacturers and tape-drive builders are exploring the use of microscopic monitoring of the physical condition of tape edges, during both initial production and storage operations, to provide improved manufacturing quality and to minimize data loss by warning of mechanical deterioration during use.
As for solid-state storage devices, its hard to argue with 250 times less latency than a hard disk—except that it comes at a cost of about 1,000 times as much per unit of capacity, making solid-state storage an alluring but generally impractical option.
However, whats working quite well in bandwidth-intensive applications is the use of midsize, solid-state units—typically 16GB to 64GB in size, sometimes configured in arrays—to serve as cache units where many processes access the same data or where theres a high rate of sustained random access.
Sevenfold acceleration of SQL queries, to cite one typical result, can yield good returns on judiciously targeted solid-state storage investments. No single silver bullet, but rather a well-aimed spread, is what it will take for enterprise IT builders to hit their storage targets.
Technology Editor Peter Coffee can be reached at peter_coffee@ziffdavis.com.
Be sure to add our eWEEK.com storage news feed to your RSS newsreader or My Yahoo page