The process of developing security metrics can be a head scratcher. With the volume of data enterprises create, how can organizations determine what’s worth measuring?
I put the question to Marcus Sachs, executive director of National Security and Cyber Policy for Verizon, who sat on a panel at the recent RSA security conference in San Francisco and shared some thoughts on the matter with eWEEK.
“All businesses should know how to do cost-benefit analysis, and developing a security metrics tracking mechanism is no different,” he said. “You have to ask the ‘so what’ question about everything you are considering, as in ‘so what if I measure this – will it lead to a cost savings or other benefit?’ However, some data – while appearing to be without value when collected – might prove to be very valuable later if you have to do an analysis of events leading up to a breach or data loss. Often this is a trial-and-error approach where you learn over time what is useful to collect and what isn’t.”
Every business is different, he said, so there is no perfect solution for handling all this.
The place to start is with any of the various regulations businesses have come to know over the years in the name of compliance, such as SOX and HIPAA. Beyond compliance reporting, businesses need to examine what he called their “business continuity plans” as well as other strategic planning that includes risk assessment. Those assessments, he said, will contain items that should be considered for tracking as part of a security metrics program.
“The best (metrics) are those that can inform company leaders about strategic decision making,” he said. “For example, knowing how well trained my system administrators are can help inform my personnel department in terms of resource allocation for additional training. Knowing how much fuel I have on hand for my backup generators can help inform my logistics group on whether or not to invest in larger fuel storage tanks or some other power backup plan. Knowing how many diverse circuits I have and the reliability of each circuit can help my telecommunications engineers with decisions on whether to consolidate circuits or to purchase additional ones.”
“Security is holistic – you have to look at all aspects, not just the pure cyber areas like the number of rules in my firewall or how many signatures are loaded in my intrusion detection engines,” he added.
At a minimum, the types of metrics being used should be reviewed annually to determine whether the information being gathered is useful or not, he advised. Breaches are not just the result of dumb luck, but of an organization’s ability to make good decisions and execute on those decisions.
“One does not have to read too much management theory to discover that measurement is a necessary factor in effective management programs…focusing on operating the essential elements of a security program reduces risk more effectively than chasing the next ‘silver bullet’ technology,” Sachs said. “So in order to measure the right things, security metrics programs should focus on excellence in the fundamentals, as well-managed programs don’t seem to have significant failure rates, in terms of data breaches at least.”