eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
WASHINGTON, D.C. —”Times have changed,” warned security consultant Mischel Kwon, former director of US-CERT. “We can’t afford to wait for an antivirus warning or to have malware trip a firewall.”
Kwon was kicking off a high-level seminar of government cyber-security experts gathered by FedInsider, a management publication for federal government executives. The panel Kwon was chairing included speakers from agencies ranging from the Department of Homeland Security (DHS) to the North Atlantic Treaty Organization (NATO), all of whom were addressing the role of big data in providing warnings of cyber attacks as they were about to happen.
But dealing with the data that describes the attack environment is a daunting task, pointed out J. R. Reagan of Deloitte & Touche. “We are amassing huge amounts of event data,” Reagan said, noting that the volume of data “is beginning to outstrip the human ability to see patterns.”
He noted that one of the most significant advances in cyber-security is the ability to produce visualizations of that data that make it possible to see patterns that wouldn’t be visible otherwise.
“We can see pictures about sixty thousand times faster than we can read text,” Reagan explained. “Now we can see the point of attack.” He said that by producing the right visualization of the event data, security officers can see patterns in the events leading up to the attack that they never would have been able to see otherwise, and as a result can see the attack as it starts. He said that by the time a security event actually happens, it’s too late.
Reagan, who is also on the faculty of Johns Hopkins University, in Baltimore, said that event data from sensors throughout an enterprise contribute to the data that can be used in the cyber-security analytics used to understand an attack when it’s just starting.
But to be able to produce visualizations that are useful requires a huge amount of data. “It can be a billion events a day,” Kwon said, explaining that this could mean as much as 24 terabytes of event data daily. But all of that data can be subjected to an analytical process that reveals patterns in nearly real time, which is important. “Patterns grow and change quickly,” Kwon said.
Visualizing all of that data isn’t easy, said Curtis Levinson, U.S. cyber-defense adviser to NATO. “What are true events and what is background noise?” he wondered. Levinson said that collecting event data is complicated by the fact that it has to be shared, and to be shared it must be cleaned of all personally identifiable information. Only then, he said, can researchers produce really useful visualizations.
Big Data Analysis Can Detect Cyber-Attacks Before It’s Too Late
But just collecting the data, even if it’s well visualized, isn’t enough. To use those visualizations to thwart an attack before it’s too late also requires sharing the data with people and organizations with the means to take remedial action. Such sharing is difficult, said Stephen Dennis, innovation director of the Homeland Security Advanced Research Projects Agency, which is part of DHS.
“We’re on the precipice of being able to really see information,” Dennis said. He added that despite all of that data, it’s still not exactly clear whether what they’re dealing with is really big data or something else. “There’s no definition of big data,” he noted.
The bad news is that there’s so much data, and as a result handling it is tough. While storage costs continue to drop and the processing power required to manipulate that data has grown substantially, it’s still hard to do. But the good news is that as more data accumulates, it becomes easier to use it to create the visualizations necessary to actually see a threat while there’s still time to do something about it besides pick up the pieces.
Unfortunately, visualization of attack profiles and the attack environment are just beginning to be developed. But that doesn’t mean security organizations can simply wait. As Kwon explained, security operations need to be looking beyond firewall alerts and antivirus warnings. “We’ve been looking at the wrong things,” Kwon said. “We must embrace our data. We have to look at how we take that data, how we use it to make our compliance team into a functional team.”
But Kwon also said that security operations need to do more than look at visualizations. They must also make sure that what they’re doing is having the right effect by making sure that incident numbers are going down and that penetration attempts are unsuccessful.
Researchers have started to look beyond just the government or specific industries to produce visualizations that are effective at seeing attacks before they become incidents. Reagan pointed to efforts by the financial services industry to produce visualizations of truly vast quantities of data. He also said that the gaming industry is an excellent example of how to do things. “The gaming industry in Las Vegas is good,” he said. “They know how to protect their money. What can we learn from that industry?”
There’s still a lot to learn about how security officers can use big data to protect the data that others want to steal, but the process has moved beyond just looking for malware into the world of predictive analytics, and that can go a long way in protecting against attacks of all types.