Law Enforcement Agencies Learn the Value of Data Mining

The Internet works against — and for — law enforcement and intelligence agencies. On the one hand, it opens new channels of communication for terrorists and gives them, authorities argue, a leg up in the war against their targets. On the other hand, it has populated authorities databases with reams of information, giving them exponentially more data than theyve ever had — and could ever use.

"Our signal intelligence communities are awash in data," says Frank Cilluffo, deputy director of the Center for Strategic and International Studies Global Organized Crime Project. "Processing the data, even if you have it, is difficult. Seeing how that information makes sense or doesnt make sense is difficult. . . . Its putting usable information on those systems, and linking what may not seem linkable in real-time, that is the challenge."

What intelligence agencies need, expert say, is better data mining technology — and lots more of it.

"What they really need to figure out is a clear vision of how you can come up with smart programs that make sense of data automatically," says Inderpal Bhandari, founder and CEO of data mining company Virtual Gold. "That is the piece they are missing. There are new techniques they need to be aware of in the data mining space. They need to pull that together, and somebody has to decide how to use the techniques." After the attacks, Bhandari started writing a White Paper about the problem, which he will send to intelligence agencies and other experts in the field.

One thing the agencies should be doing is using technology that visualizes their data for them, says Patrick Hogan, Cedar Groups vice president of content value management technology.

"These guys are struggling," Hogan says. "They have huge amounts of information. Just the e-mail tips alone [since the Sept. 11 attacks] would be enormous. They have to be processing that information, and we are saying you can visualize that data and extract the key correlations — all sorts of weird and wonderful pieces of the jigsaw puzzle — and that releases valuable labor for those guys to go out into the real world and solve problems."

Michael Welge, director of the automated learning group of the University of Illinois at Urbana-Champaigns National Center for Supercomputing Applications, says intelligence agencies problems are unique, but not necessarily uniquely difficult.

There are no off-the-shelf commercial tools that can meet their data mining challenges, Welge says, calling on the government to work more closely with researchers and industry to develop the right tools.

Shortly after the bombings, in fact, government representatives did call the University of Illinois engineering department, asking for any expertise it could offer in the war against terrorism, Welge says. He is drawing up a data mining proposal now to deliver to the government soon, he says.

Intelligence agencies have long been aware that they were falling behind on the technological front. The Central Intelligence Agency in 1999 paid for the launch of its own nonprofit venture capital firm, called In-Q-Tel, aimed at giving seed money to companies that could develop technologies to solve CIA-specific problems.

Since 1999, In-Q-Tel has funded about 20 companies. An In-Q-Tel representative says the company is not talking publicly about its projects.

But, the CSIS Cilluffo says, "I assure you, a lot of what they are looking at is this: How do we make our data mining smarter? "