How to Evaluate Products Based on Appropriate Usage

Use Cases are a concept from software and system engineering that is employed to capture the functional requirements of a system. Software developers identify Use Cases to set the parameters for how a product will behave under various conditions. This method of understanding and defining requirements is an effective method for transforming a need into formal logic (lines of code), and lines of code that then become a functional product.

Applying Use Case-based methodology for testing products based upon appropriate usage provides a clear picture of which technologies are effective against a particular type of attack. Thus, products can be evaluated by comparing their capabilities against specific deployment scenarios and protection requirements.

Measuring products based upon appropriate usage has a number of benefits:

• Accurately identify protection capabilities of a specific product.
• Ensure product protection capabilities match the security requirements for certain types of environments.
• Provide guidance on technology buying that more accurately matches customer needs with product capabilities.
• Recommend product usage based upon rational testing methodologies and empirical data.
• Avoid over or under spending for solutions.

Use Cases in Product Evaluation

For information security products, Use Cases identify what content should be allowed to pass and the types of attacks a given product is designed to protect against. When evaluating such solutions, Use Cases should take into consideration:

• Functional protection requirements. What implications are there for security coverage for each class of assets? What threats should the solution detect?
• What compensating controls or existing solutions are already deployed?
• Specific compliance requirements. Example: Are there specific compliance mandates that apply, such as PCI DSS?
• Performance: how much traffic must be handled?
• Administration and reporting: What specific information will be needed/required in logs and reports? Who is allowed access to specific types of data?
• Once the requirements are understood, comparing Use Case testing results enables quick identification of products that meet functional and performance objectives.

Environmental Usage

With Use Case testing, several environmental factors should be considered when evaluating products:

• Datacenter – Focus on server protection for complex internal applications. In testing such offerings, uptime is mission critical in these production networks and high availability/failover is required. False positives are also unacceptable, and products are often maintained by experienced professionals who custom-tailor policies to prevent false positives.
• Network Perimeter – Focus on client and server protection from internet-based attacks, includes simple DMZs with web, mail, DNS. When evaluating these products, HA/failover is preferable, but always not required. False positive rates are important, but not critical.
• Retail Storefront – Focus on client protection. Uptime is important when testing these offerings, but not mission critical 24x7x365. Network devices need to provide backup network access (via dial-up, ISDN, DSL line) and high availability for operations such as larger retail stores. False positives rates are less important.
• E-Commerce Site – Focus on server protection for complex DMZs that include application servers & database servers that require protection from sophisticated Internet based attacks. E-Commerce sites are tested as "production networks," therefore low false positive rates are required as false positives mean lost business.

The Future of Use Cases in Security

A number of trends are driving increased adoption of Use Cases; including software development best practices, network and systems design, and even compliance auditing against standards like the PCI DSS.

Stakeholders in the information security product marketplace, such as product managers, IT and compliance professionals, product evaluators, and auditors, can all benefit from speaking the same language. Use Cases offer an effective vehicle for unifying the discourse of product needs and capabilities.

As buyers in the marketplace continue to identify new and evolving requirements (whether for compliance or security), the thoughtful application of Use Case methodology will not only make their jobs easier, but provide clearer guidance for product vendors seeking to communicate the benefits of their offerings.