Data Warehousings Big Sleep

Dormant data—information that never has and never will be accessed—can give a data warehouse a heart attack. Could your systems be at risk?

The cost of storage may be falling, but are you reaping the benefits? Probably not, according to Bill Inmon, creator of the data warehouse concept. "Were using storage at a rate that far surpasses the rate at which prices are dropping," he says.

Corporate data warehouses, which collect far more detail than standard databases and retain information historically, account for a significant proportion of storage costs. But the usefulness of a warehouse doesnt necessarily increase as you stuff more data into it. Rather, many data warehouses suffer from an abundance of "dormant data"—data that has never been accessed and probably never will be.

Such excess "clogs up the efficiency of data for all users," says Inmon. "If you get enough cholesterol in your bloodstream, you have a heart attack. Its the same with dormant data." Superfluous data can also aggravate the complexity of storage systems, which are becoming increasingly hard to manage.

Besides performance, theres cost to consider. Disk-storage vendors like to advertise falling prices, but "the ratio of disk to nondisk prices has been fairly constant for almost 10 years," says Inmon.

Once you add the complexity of managing live data, the cost of keeping data on disks far outweighs the cost of keeping it on "near line" media, such as DVDs or tapes. Inmon has found that for every dollar a company pays to run a disk-based storage system, it would pay only 13.4 cents for a near-line setup of similar size.

Before you count your chickens, however, you need to know how much dormant data you actually have.

For a calculation tool click here