Why Arent You Archiving?

Updated: Opinion: Despite the many benefits of a "lean and mean" database, the database archiving market is not growing as much as it should.

Database archiving, to me, is a no-brainer.

I have held this opinion since 2002 when I first came across some vendors in this space and after speaking with organizations that had implemented a database archiving solution.

Indeed, I had also predicted the database or structured data archive market should reach $240 million by the end of this year.

That means that a paltry 1,200 organizations would have invested a relatively small amount (about $150,000-$200,000) and perhaps a month of their time to implement a database archive solution.

Database archiving is simply the process of removing inactive or low value rows of data from a production database table and placing them in another table either within the same database instance or a different instance.

/zimages/1/28571.gifDBAs are predicted to thrive and multiply. Click here to read more.

The process is complicated somewhat by the business rules and application logic that you would want to employ to evaluate if a row was a candidate for archive.

For this reason the market-leading vendors started offering application-specific solutions for Oracle eBusiness Suite and PeopleSoft, among others. Of course, custom database archiving was always an available option.

Now admittedly, my projection was hardly scientific. I simply asked the leading vendors (Princeton Softech, Applimation, OuterBay, Solix) to tell me their average deal size, number of customers and assumed that 100 percent CAGR through 2006 was easily achievable.

I did expect it to level out in the later years, but that strong growth would remain for many years to come. It is halfway through 2005 and my sense is that the database archive market has failed to achieve the type of growth I had predicted.

I recently asked a manager of a Fortune 300 company if they do database archiving as a best practice. He told me they didnt feel like there was a pressing need. So I responded with some specific what-if value propositions that I have seen in some shops:

  • Perhaps as much as 80 percent of the data in your production database could be inactive.
  • What if users that are budget sensitive realized that the production data is replicated an average of seven times?
  • What if you could achieve a 1:1 correlation in performance of your most expensive queries (e.g., 30 percent reduction in DB size = 30 percent improvement in query performance)?
  • What if users understood the time savings assuming the database needed to be fully recovered if the database was 50 percent smaller?
  • What if users understood the relationship between personnel costs and the size of the database? The larger it is, the harder it is to maintain linear performance.

Next Page: Why has database archiving been neglected?