Active Warehousing

Ive recently used this space to warn of the perils of building real-time systems that dont feed a real-time decision-making process. To look at the input side of that equation, and at the strengths that a system must have to make real-time decision making a plausible goal, Id like to share some things I heard from Stephen Brobst, CTO of the Teradata division of NCR, when he spoke at this months Chicago conference titled "Creating the Real-Time Enterprise."

Brobst took what could be another mere buzzword, "active data warehousing," and explained it as a process of evolution in how an enterprise uses data. Without the "active" adjective, a data warehouse can still be a valuable resource for strategic identification of trends and opportunities; with the active emphasis, though, a data warehouse becomes much more of an operational tool and represents much more of an opportunity to change the way the

enterprise makes tactical decisions that dramatically increase its value to all its partners.

I found Brobsts five-stage model persuasive: It fits my own experience of how organizations grow in their use of data. His first stage, reporting, relies on batch-oriented tools with little in the way of ad hoc query capability. This is the approach that first-wave personal computing brought to enterprise desktops with tools like Ashton-Tates dBase II, starting around 20 years ago.

I remember how startled people were by the low cost and rapid development cycles of those early tools and the speed with which they replaced costly minicomputer-based systems—but I also remember the upfront effort involved in defining the range of queries that these systems would support and the high cost of redefining queries as users learned what these systems could provide.

In the second stage of data use, analysis, people want to ask new questions when the old ones give unexpected answers. Spreadsheets spread, so to speak, across enterprise desktops to enable this kind of dynamic process. Unfortunately, in the process, spreadsheets do more than just illuminate our thinking: They cast shadows that can warp our perceptions. Studies have found that people who use spreadsheets become overly confident of their ability to predict, and even control, the numbers that they crunch with such facility.

Analytics, therefore, should be constructed with the same rigor that administrators apply to raw data. The conclusions that come from statistical methods, for example, need to be viewed with professional skepticism by people who know the difference between a high degree of fit and a useful level of significance. Plotting best-fit regression lines is much easier than it ought to be, considering the potential of those lines to point in misleading directions.

When analytic tools are properly used, though, the organization is ready to enter Brobsts third stage of data use: prediction. This is an important step, but the resulting forecasts are still an offline, strategic device that doesnt really change the way business gets done.

This is just the foundation that must be in place before the enterprise can move into the fourth and fifth stages. I dont especially care for Brobsts name of "operationalizing" for stage four. It sounds like psychobabble, but I cant come up with anything better to describe the goal of this stage—to move the organization from asking "What will happen?" to "What is happening?"

It might seem as if describing the present would be easier than predicting the future, but different types of information move at different speeds: Producing an accurate snapshot in time is like making an elaborate dinner and having all the different dishes ready to serve at the same moment. Its harder than it looks.

When we can actually say that we know whats happening right now, were ready to build event-triggered systems that enable better moment-to-moment decisions—not just annual plans or quarterly goals. When quality information is delivered to people who can appreciate its impact, in a way that helps them see opportunities before they grow stale, then enterprise systems are able to address the question, "What do I want to happen?" Thats what deserves the label of "active" warehousing—and its a beginning, not an end, to a real-time transformation. ´

Peter Coffee can be reached at peter_coffee@ziffdavis.com.