Do We Really Need Bigger Hard Disks?

By David Morgenstern  |  Posted 2006-10-17

Do We Really Need Bigger Hard Disks?

Creating bigger—that is, higher-capacity—hard disks is usually the story of the industry, particularly in this year commemorating the invention of rotating storage. And with demand reportedly growing to hold collections of compliance data in the enterprise—not to mention the consumer segment—the search for bigger drives is seen as the natural order of the technology.

Despite the common wisdom, however, some suggest that this single-minded pursuit may mean the storage industry is ignoring some pressing issues. And for some market segments, especially in the enterprise, more capacity may not necessarily be better or even necessary.

At the recent Diskcon conference of IDEMA (the International Disk Drive Equipment and Materials Association), in Santa Clara, Calif., almost every presentation began with recollections of the ancient days of hard drive manufacturing when the memory capacity that now fits in an iPod used to fill a bunker.

Often, the speakers continued the history lesson with a comparison of hard drive advances to those of some other invention, usually automotive.

For example, in his "kickoff" speech, Seagate Technology Chief Technology Officer Mark Kryder said that if a 1956-vintage standard car had undergone the same rate of "progress" as a hard disk, "We ought to be squeezing 146,800 people into that automobile today; the price should have dropped to $15; and have a top speed of almost 1 million miles per hour."

Following the wave of such stories (including his own), Dan Frost of the San Francisco Chronicle blogged a clever response from a reader, which then made its way around a number of storage lists. Heres bit of the post:

"If my car was like my hard drive, I would need to keep an exact copy of everything that I carry in the car because sooner or later the car is going to lock itself, and I will never get into it again. If I decide to go to the trouble of getting into the car, I will have to take it to a specialized mechanic who will probably charge as much as the car cost, with absolutely no guarantee of salvaging anything," the reported author, Dave Hector, observed.

His final shot was: "You get the idea. I love my car and I trust it. I love my computer, but I dont ever, ever, ever trust it."

While this comment is certainly entertaining, it is also educational. It points to a fundamental disconnect between the makers of storage and computing systems and their customers.

IBM invented hard disk storage 50 years ago, in the form of the 350 Disk Storage Unit that went along with the RAMAC Computer. So what has the worlds largest technology company done for data retention lately? Click here to read more.

In the storage industry, almost all R&D is focused on raising the level of areal density in a drive, or the number of bits of data that can be packed on a disk platter. This is the way that we find ever higher-capacity hard drives in our notebooks and desktops, or smaller drives in portable music players that have the same capacity as larger drives from a few years ago. All makers of storage, including hard disk drives, flash and optical, are driving towards higher capacities.

At Diskcon, all presentations contained a mention of the continuing progress toward higher densities. There were many graphs showing how this and that technology would let a company keep up with the storage growth curve. Almost all questions raised were about the rate of increasing capacity, its sustainability and what technology will be employed to improve capacity. And whether a profit can be made in this manufacturing cycle.

Click here to read more about the technologies hard drive makers will employ to meet expected capacity goals.

But customers might have a different view of the priorities for progress in the storage arena and what benchmarks would constitute progress.

Items on a customers list might include: What have the manufacturers done to improve security on disk mechanisms? What can disk makers do to increase performance, especially seek performance, which hasnt kept up with capacity in any way? And then theres the subject of reliability.

The response from the storage execs appears to be mostly finger-pointing to different parts of the system. Or worse, just a shrug. The answers must come from someone else, they seem to say, since they are too busy working on the capacity problem.

Sure, hard disks are more reliable than they were a dozen years ago, but how much have they improved recently? Drives sure seem to fail at the worst times.

Some presentations at the conference suggested that storage makers (or their OEMs) may not fully understand that consumers may demand more robust storage than business users and may be more sensitive to failure. (The hard disk in my DVR is starting to fail, even though its spec sheet and its low annualized fail rate would suggest that the drive should work for years more. My wife is having some trouble understanding how statistics lose their meaning when faced with the performance of an individual drive.)

Next Page: The downside to bigger drives.

The Downside to Bigger


On the enterprise front, are drive makers addressing technologies that could help drives destined for use in servers, desktops and aftermarket NAS (network-attached storage) to survive mishandling? Drives aimed at laptops and portable applications have improved, but I get the willies sometimes when I see users treating larger drives like floppies.

At the same time, we face a fundamental question: Do we always need more capacity? Is more capacity needed for every application, on every server and on every client system?

For example, the entry point for a 2.5-inch notebook drive is 40GB nowadays. A good example is Seagates MobileMax family of drives, which spans 40GB to 80GB. But that entry point will climb over time.

As drive makers push capacity, that low point will rise and the manufacturers will discontinue production of the lower-capacity drives. For the same cost, we gain capacity.

But do we even need 40GB for the enterprise client? When we get down to it, the average client machine in the enterprise could get along with a 32GB drive or even smaller one, depending on the system installation and applications.

The full Ultimate Vista install takes about 11GB of disk space and it comes with many items not needed for an enterprise workflow. So even when you add in space needed for memory paging, search indexes and other system tasks, and then applications, the total must still be well under 20GB.

Meanwhile, the accumulation of user data comes to perhaps 3GB per year, some analysts suggest.

So, whats the problem, you might ask? In this minimum calculation, customers gain an extra 10 to 20GB "for free" with the low-capacity drives. With a bigger drive, we get more room and that must be better, right?

Yet, additional capacity can be a nuisance to IT managers when calculating backups. The more data thats stored on client machines, the more needs to be backed up.

In addition, the bigger the hard drive on the client system, the more that users will take advantage of that extra capacity. Extra space encourages users to store unnecessary local copies of files or worse, large collections of video, audio and image data, which may fall outside the enterprises purpose.

Dan Renouard, a senior research analyst at Robert W. Baird and Co., of Milwaukee, admitted in a Diskcon Q&A session that many enterprise CIOs want to keep and control everything on the network. In their eyes, the worst place for data is on a local machine.

Flash companies such as Samsung Electronics are pushing this lower-capacity concept for flash-based drives that aim to replace hard disks. The pitch is that flash may cost more for less capacity but it offers greater reliability for mobile apps, as well as potential performance benefits for Windows Vista. They hope to woo IT managers with the benefits of a thinner workflow with less capacity.

The hard disk makers at Diskcon scoffed at this notion. Of course, everyone wants more capacity for the same cost.

Should hard drives be outfitted with flash memory? Microsoft says flash is a good thing for Windows Vista performance. Click here to read more about Vistas ReadyBoost and Intels Robson flash technologies.

The divergent tracks of reliability and capacity are also felt in server-side storage.

For a change of pace, instead of looking at the high-capacity points, lets look at some of the low points. In Seagates enterprise lineup, the low-capacity point is 73GB for its 10,000 rpm and 15,000 rpm Cheetah series, and 10,000 rpm Savvio drives. These drives are storage-server-specific, using either a SAS (serial attached SCSI), Ultra 320 SCSI or Fibre Channel interface. Of course, the lines come in higher capacities: 300GB and 146GB for the 3.5-inch Cheetah and 146GB for the 2.5-inch Savvio.

But if capacity were the be-all and end-all of the storage equation, then why would anyone want a 73GB drive? And you can get a SATA (Serial ATA) drive holding 750GB!

One of the advantages of lower capacity is that you put more of your data at risk if a single disk fails in an RAID Level 5 array. A higher-capacity disk will take longer to rebuild, leaving your data at risk for a longer period of time. (And theres a performance advantage on these drives, but lets not go there.)

Now, to address this reliability issue, storage vendors pitch RAID Level 6 systems instead of Level 5. (RAID 6 provides two sets of parity data, meaning that it can recover when two drives fail in an array, or if a drive goes south while the set is rebuilding from an earlier failure.)

The reliability calculations from storage vendors suggest that a RAID 6 array should have a "mean time before data loss" of some 86,695 years. Despite this immense figure, at almost every storage conference and storage user group meeting Ive attended this year, someone has predicted that somewhere in the world a RAID Level 6 array will fail sometime in the next 12 months, leading a company to lose important data or productivity. And for a standard RAID Level 5 array, the risk is even greater.

With the industry bent on increasing capacity and cost, its no wonder that storage manufacturers have little energy for improvements in other areas. (We didnt get to security at Diskcon, but I attended a session on the subject at a recent developers conference. The outlook is troubling and I will report on the developments in an upcoming column.)

Of course, a major disincentive for manufacturers is that these alternate solutions would add to the cost of a drive, which is an almost impossible sell to system makers.

So, if we want something other than ever-bigger drives, IT customers will need to decide whether its worth a bit extra to have a drive thats more reliable, and may hold less.

Check out eWEEK.coms for the latest news, reviews and analysis on enterprise and small business storage hardware and software.

Rocket Fuel