How to Evaluate Products Based on Appropriate Usage
Email
Print
Information security products have evolved rapidly over the last decade. However, according to NSS Labs CTO Christian Stankevitz, the science of evaluating products has virtually stood still during that same time period, creating a knowledge gap that has made it difficult for information security buyers to determine whether or not a product meets specific security and/or compliance needs. By applying a Use Case-based methodology, information security professionals can more clearly identify detailed protection requirements for a given environment.
Use Cases in Product Testing Use Cases are a concept from software and system engineering that is employed to capture the functional requirements of a system. Software developers identify Use Cases to set the parameters for how a product will behave under various conditions. This method of understanding and defining requirements is an effective method for transforming a need into formal logic (lines of code), and lines of code that then become a functional product.- Accurately identify protection capabilities of a specific product.
- Ensure product protection capabilities match the security requirements for certain types of environments.
- Provide guidance on technology buying that more accurately matches customer needs with product capabilities.
- Recommend product usage based upon rational testing methodologies and empirical data.
- Avoid over or under spending for solutions.
- Functional protection requirements. What implications are there for security coverage for each class of assets? What threats should the solution detect?
- What compensating controls or existing solutions are already deployed?
- Specific compliance requirements. Example: Are there specific compliance mandates that apply, such as PCI DSS?
- Performance: how much traffic must be handled?
- Administration and reporting: What specific information will be needed/required in logs and reports? Who is allowed access to specific types of data?
- Once the requirements are understood, comparing Use Case testing results enables quick identification of products that meet functional and performance objectives.
- Datacenter â Focus on server protection for complex internal applications. In testing such offerings, uptime is mission critical in these production networks and high availability/failover is required. False positives are also unacceptable, and products are often maintained by experienced professionals who custom-tailor policies to prevent false positives.
- Network Perimeter â Focus on client and server protection from internet-based attacks, includes simple DMZs with web, mail, DNS. When evaluating these products, HA/failover is preferable, but always not required. False positive rates are important, but not critical.
- Retail Storefront â Focus on client protection. Uptime is important when testing these offerings, but not mission critical 24x7x365. Network devices need to provide backup network access (via dial-up, ISDN, DSL line) and high availability for operations such as larger retail stores. False positives rates are less important.
- E-Commerce Site â Focus on server protection for complex DMZs that include application servers & database servers that require protection from sophisticated Internet based attacks. E-Commerce sites are tested as "production networks," therefore low false positive rates are required as false positives mean lost business.
