Hacker Challenges Are an Extremely Poor Testing Technique
I've said it before, and I'll undoubtedly say it again: The foundation of strong application and operating-system security is testing, testing and more testing.Ive said it before, and Ill undoubtedly say it again: The foundation of strong application and operating-system security is testing, testing and more testing. Most software security problems arise from design and coding errors, rather than a lack of some security-specific feature. The only way to catch those errors is to put the software in question under a microscope, checking for buffer overflows, unexpected responses to input and unpredictable interactions with other software, as well as manually auditing source code. As Ive also said before, that kind of testing is not popular among software development companies. Effective security regimes are slow, expensive and resource-intensive. More importantly, they add very little to the bottom line; security testing simply isnt "sexy" from a marketing perspective. Potential customers have very little basis for distinguishing between well-tested, poorly tested and untested software until well after the fact.
Public awareness of security issues is on the rise, however, and vendors are starting to feel the pressure to demonstrate that they take those issues seriously. In response, they have increasingly begun to focus on an unfortunate type of pseudo testthe "hacker challenge." By making a test platform with their product available to the public and granting permission for anyone to attempt an attack, vendors can theoretically enlist the hacker community to help dig out vulnerabilities.