Real-Time Data Analysis Due in DB2, Yukon

By Matthew Hicks  |  Posted 2002-07-01 Print this article Print

IBM and Microsoft are upgrading their respective database systems to perform real-time analysis of incoming data, giving users the ability to react more quickly to changes in customers' habits and market conditions.

IBM and Microsoft Corp. are upgrading their respective database systems to perform real-time analysis of incoming data, giving users the ability to react more quickly to changes in customers habits and market conditions.

IBM is working on integrating into its DB2 Universal Database features for real-time analysis that it gained in its acquisition of Informix Corp. last year. Microsoft plans to add to its next release of SQL Server, code-named Yukon, the ability to do OLAP (online analytical processing) and data mining on real-time streams of data.

"[Companies] big problem is that theyre so backlogged with the data they have already," said Darrell Starnes, deputy CIO at Houston-based, a division of Global Sports Interactive Inc. "The hardest thing they have to do is getting filtered, valid, important data that they can actually make business decisions on in real time."

IBMs key tools for real-time analysis are Informix Time Series Real-Time Loader and Informix NAG DataBlade. Combined, the products allow for the IDS (Informix Dynamic Server) database to perform immediate analysis on data feeds by processing it in the memory of the database server, rather than waiting until the data reaches storage disks, said Paul Taylor, a Distinguished Engineer at IBMs Silicon Valley Laboratory here.

This "in-memory" feature allows for fast processing of incoming data, which Taylor said has reached rates of more than 25,000 rows per second. In-memory analysis should start appearing in DB2 next year, he said. Last year, IBM through its Data Joiner federation technology tied DB2 to IDS for companies using both databases.

"Once we finish integrating the technology, the idea would be to just have DB2, and that would simplify the management because there would be one less server," said Taylor.

IBM will focus its real-time analysis efforts initially on Wall Street users, but researchers said the functionality can apply across many industries.

Microsoft plans to add the ability to perform OLAP and data mining functions in real-time ETL (extract, transform and load) data streams flowing into a data warehouse or a production database, according to Bill Baker, general manager for SQL Server business intelligence, in Redmond, Wash.

SQL Servers existing ETL capabilities can hook up to a variety of data sources, such as real-time data feeds, Baker said. The SQL Server upgrade, slated for a beta release late this year or early next year, adds the analytical capabilities.

Separately, data warehousing specialist Teradata, a division of NCR Corp., in Dayton, Ohio, also plans to push more features for what it calls "active data warehousing" into the next release of its namesake database, planned for this fall, but officials declined to offer details.

DB2 user Bank of Montreal, which has implemented a data warehouse for business intelligence, is looking to update customer information more frequently. Today, the bank refreshes data monthly, said Ted Medes, director of database management services.

Medes envisions moving more toward real-time data, but that will depend more on justifying the business need than on technology issues.

"Its really [about] serving the customer in a more holistic way," said Medes, in Toronto. "So that if you did something with the bank—you purchased a product, or you deposited some money—the bank is able to respond to you in a helpful way soon thereafter. Thats what you expect."

Related stories:
  • Review: DB2 7.2 Smoothes Data Integration Bumps
  • IBM Adds Hybrid OLAP in DB2
  • Oracle, IBM DB2 Gain Intelligence
  • IBM Gets SMART About Managing DB2
    Matthew Hicks As an online reporter for, Matt Hicks covers the fast-changing developments in Internet technologies. His coverage includes the growing field of Web conferencing software and services. With eight years as a business and technology journalist, Matt has gained insight into the market strategies of IT vendors as well as the needs of enterprise IT managers. He joined Ziff Davis in 1999 as a staff writer for the former Strategies section of eWEEK, where he wrote in-depth features about corporate strategies for e-business and enterprise software. In 2002, he moved to the News department at the magazine as a senior writer specializing in coverage of database software and enterprise networking. Later that year Matt started a yearlong fellowship in Washington, DC, after being awarded an American Political Science Association Congressional Fellowship for Journalist. As a fellow, he spent nine months working on policy issues, including technology policy, in for a Member of the U.S. House of Representatives. He rejoined Ziff Davis in August 2003 as a reporter dedicated to online coverage for Along with Web conferencing, he follows search engines, Web browsers, speech technology and the Internet domain-naming system.

    Submit a Comment

    Loading Comments...
    Manage your Newsletters: Login   Register My Newsletters

    Rocket Fuel