In the hands of a government preoccupied with security, data mining technologies raise the specter of intrusiveness to Orwellian proportions. But developers of such technology, who have grown weary of seeing it held out as some kind of bogeyman, say data mining technology, like many others, was designed for good.
Researchers from academia and industry gathered here recently to discuss the future of the technology at the annual Knowledge Discovery and Data Mining Conference sponsored by the Association of Computing Machinery. Facing a recent spate of bad publicity regarding government data mining initiatives, researchers are eager to champion the technologys potential as a strategic tool for businesses.
A group of doctorate-level panelists floated ideas for future commercial applications in the fields of security and fraud detection, e-commerce, and bioinformatics. But panelists offered few details on when or how these applications would become available. Nonetheless, some see it as a growth industry in light of recent advances in storage technologies, which have created new ways to warehouse vast amounts of data cheaply.
Also a boon to the technology is Microsoft Corp.s recent decision to add data mining capability to SQL Server. Other companies—and not just developers—are focusing instead on honing the technology for business use.
General Motors Corp., for example, is working on ways to turn data mining into a strategic enterprise tool, Ramasamy Uthurusamy, a researcher at GM, said at the conference last week. Citing the recent reported success of Harrahs Entertainment Inc. in examining customer data to increase return business through new customer incentives, Uthurusamy said some businesses are already availing themselves of the technology.
Despite the optimism, the developers at the conference acknowledged numerous challenges—both technological and cultural—that must be addressed before data mining becomes a widespread and useful enterprise tool.
One of the main technological obstacles is managing mined data and keeping sight of the purpose for collecting it. Usama Fayyad, chairman and co-founder of Revenue Science Inc., said he makes a “mess” whenever working on a data mining project, leaving “a trail of droppings thats of biblical proportions.” Within two to three days of initiating a project, it becomes easy to lose sight of the purpose and goals, Fayyad said.
Another major obstacle, in Fayyads view, is the disconnect between the way data is represented in data stores and the way mining technologies work. Revenue Science, which changed its name in June from DigiMine Inc., is based in Bellevue, Wash.
The publics concern about protecting privacy remains a key obstacle, and researchers increasingly see it as a problem that they must address themselves.