Database Archiving Market Evolves with Exploding Data Needs

Analysts say the small database archiving market will see changes as the demands of information-as-a-service vendors and the explosion of corporate data come to a head.

The database archiving market is set to change as the needs of users usher in an era of partnerships and a new emphasis on the technology by database vendors, industry watchers said.

Currently, the database archiving market remains relatively tiny, with a small subset of players such as Solix, Neon and Applimation. But as HP's acquisition of OuterBay in 2006 and IBM's acquisition of Princeton Softech in September demonstrate, the market for database archiving tools is growing. An IDC report estimated the sector grew 13.6 percent between the third quarter of 2006 and the third quarter of 2007, driven by e-discovery, regulatory compliance and overall storage optimization needs.

Brian Babineau, an analyst with Enterprise Strategy Group, sees an opportunity for application, database and information service vendors to partner with providers of archiving tools to deal with the explosion of data organizations are facing.

"The privacy regulations are making subsetting more important in test and development environments," Babineau said. "The approximate average retention period for database records is roughly 6-10 years. Over half of electronic discovery events involve database information. With all these drivers, just over a third of the market is using database archiving solutions. It is only a matter of time before organizations realize the risks of creating too many copies of the information and decide that archiving as well as subsetting is a much better approach."

The number of primary databases is going up 25 percent annually, Enterprise Strategy Group estimates, with the number of secondary databases rising as well.

Large databases can drag application performance and it is here that database archiving vendors make their pitch. When databases become less manageable, customers look to archive, and while many organizations develop custom archiving tools, integration with the database, storage software and seamless integration with the application tend to make things more complex.

Eventually, database vendors are going to step up their own archiving tools, predicted Carolyn DiCenzo, an analyst with Gartner.

"I think [the market] actually goes away; I think it becomes part of the database infrastructure," she said. "It just becomes one of the things that database vendors need to provide for their users in order to meet this requirement. I really don't see any benefit for having it heterogeneous and people would rather get this from the database vendor than from a third party," she said.

"But where you'd see vendors get involved, it would be vendors such as SAP, or the application side of Oracle...where it sees customers hitting the wall, performance or recoverywise," she continued. "So they would team with a database archiving solution to offer users a way to segment and remove older data from the archive. The vendors wouldn't do it themselves, they would just extend the eco-structure and either certify or co-market with an archiving solution."

Officials at Neon and Princeton Softech agreed that IAAS (information as a service) vendors will look to integrate with archiving software providers to support the integration of structured, unstructured and semi-structured data in both active and inactive repositories.

"This move toward IAAS should also make it easier for companies to include data archiving as part of the application and information life cycle, making the practice of archiving even more prevalent," said Robin Reddick, vice president of marketing and communications at Neon. "As we have talked with customers about archiving, several have recognized the opportunity to use an archive data store to populate the databases used for business intelligence/business analytics. By using an archive, they are able to have a centralized data store from which to extract data."

Click here to read more about the "new" ILM.

IBM's Information On Demand strategy, which included the acquisition of FileNet to handle unstructured data management and Princeton Softech for structured data, has been built partly with that idea in mind. Many of IBM's solutions address data in both active and inactive repositories, noted Al Smith, executive vice president of engineering for IBM Information Management. Optim, for example, addresses data through its full life cycle-production, near line, archived, offline tape, worm and ultimate disposal-and has the ability to present data that has been moved to inactive stores back to the application, he said.

"Also you will find part of the Cognos acquisition strategy is addressing the unique needs of warehouse and BI use of data that is no longer transactional," he said. "It is another step in the full Enterprise Data Management structure."

Check out's Database Center for the latest database news, reviews and analysis.