Jason Brvenik, chief technology officer at NSS Labs, spends his time testing a lot of different security technologies in an effort to evaluate vendor claims and product efficacy. The most recent set of technologies tested by NSS Labs are next-generation firewalls (NGFWs), with test results published on July 17.
Among the products tested by NSS Labs were NGFWs from Barracuda, Check Point, Cisco, Forcepoint, Fortinet, Palo Alto Networks, SonicWall, Sophos, Versa Networks and WatchGuard.
In a video interview with eWEEK, Brvenik explains how NSS Labs tests security technologies and why his firm is moving to a continuous evaluation model to help provide the most accurate results.
There are multiple techniques and tools for testing cyber-security technologies, including the popular open-source Metasploit penetration testing framework. Brvenik said NSS Labs goes above and beyond what Metasploit does, analyzing protocols and looking for ambiguities in specifications.
Brvenik added that NSS Labs has its own Baitnet test harness that is a core element of the evaluation process. Baitnet is an automation framework for replaying attacks in parallel. Compliance conformance is not, however, something that Brvenik is overly concerned about.
“It’s about effectiveness at the end of the day. We look to assess how well a technology meets an enterprise’s needs and compliance is irrelevant, especially in our space. Our entire industry exists because compliance fails,” he said. “Whether or not you comply with any given standard really comes down to how well the technology identifies an attack, identifies evasive behavior, identities the presence of an adversary or prevents that action from occurring.”
Watch the full video interview with Jason Brvenik above.
Sean Michael Kerner is a senior editor at eWEEK and InternetNews.com. Follow him on Twitter @TechJournalist.