Lets Get Fuzzing
Opinion: When systems get as complex as Windows, the only way to keep up with the security flaws is with aggressive fuzz testing.Ive said it before and Ill say it again: You just cant do enough testing of software systems. And as complexity of software systems rises, more testing is necessary. Do you think, for example, that a Boeing 747 is a complicated piece of machinery? I suspect it pales in comparison to Windows Vista for complexity. The recent ANI episode said a lot about the problem of complexity and testing. In fact, Microsoft does test aggressively and more than ever, and yet the ANI failure was more than anything else a failure of its testing. There are many different kinds of testing you can do, but when it comes to the bizarre attacks of the ANI sort, nothing beats fuzzing. Microsoft knows this, and it has been an enthusiastic fuzzer for years.
Are you a security idiot, doing the wrong thing all the time? Click here to find out.
It turns out none of the .ANI fuzz templates had a second "anih" record. This is now addressed, and we are continually enhancing our fuzzing tools to make sure they add manipulations that duplicate arbitrary object elements better.Oopsie! This is especially sloppy in light of the fact that a previous severe vulnerability in the same format also involved multiple anih records. Making the decision that something is not worth fuzzing is getting increasingly dangerous. Read the April 27 Symantec Security Response blog for an idea of how bizarre the attacks are getting. The malware writers smell blood in the water in the form of sloppy parsing and sloppy testing. You cant be too careful. And fuzzing is a two-way weapon. Dont think that the people who found and exploited the ANI zero-day found it by flipping bits in a file by hand. They have their own fuzzers too, as do the vulnerability research companies like eEye and Determina (who found the ANI bug). This is why, with as much fuzzing as it already does, Microsoft needs to do more and more. Heres a good candidate: Microsoft recently announced Silverlight, a Flash-like browser plug-in for multimedia content. Someones got to fuzz the hell out of this thing, and not only on IE, but on Firefox and Safari, both of which are supported. As Joanna Rutkowska likes to say, computer systems these days are too complex to have real confidence in. The ANI failure was basically the result of sloppy testing, but even if Microsoft had tested well and found it, there are a large, unknown number of similar issues in a system as complex as Windows (or any other modern operating system). Until a better approach comes along, we need to have more testing and much more aggressive testing. Security Center Editor Larry Seltzer has worked in and written about the computer industry since 1983.