eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
Testing is a consistently underappreciated part of software development. Comprehensive testing takes time away from development and delays product releases, both anathemas to developers. And yet when a program turns out to be low-quality, you have to know that on the inside many people are whispering that the testing guys didnt do their job.
Ive been in the computer-testing business professionally since 1987, so I speak from experience when I say that the only thing better than a well-designed test is a well-designed test that can be extensively automated. A new class of testing tools may vastly increase the amount of automation possible in some important classes of testing.
“Fuzzing” is a testing concept that has been in use for many years, mostly with in-house testing departments. The basic concept is to send unexpected input into a program to see if it reacts properly. The granddaddy of fuzzers is Fuzz from the University of Wisconsin-Madison.
I remember the first public fuzzer I heard of: It was an HTML fuzzer that created strange tags—some malformed, some just bizarre—and fed them into browsers. It found lots of errors and exploitable vulnerabilities in Internet Explorer and even more in Firefox.
And yet this fuzzer, like most of the public efforts so far, was a naive one, with a high degree of randomness in the input. An intelligent fuzzer isnt random; its actually quite deliberate, and it probably avoids genuinely malformed input.
This is what Beyond Security, a vendor of security tools and services, is working on. The fuzzing tool it is working on, called beSTORM, focuses on network-enabled applications and models the protocols used to communicate with them in a graph using a language they developed similar to BNF (Backus-Naur Form) notation. (A “graph” in computer science isnt bars and lines, its a complex data structure with nodes and edges between them.)
Check out eWEEK.coms for the latest news, reviews and analysis about productivity and business solutions.
beSTORM exercises the protocol with specific emphasis given on technically legal but functionally erroneous and stressful cases. A simple example would be where the protocol calls for a length value followed by a stream of data. What happens if you send x for the length byte, but then send x*2 bytes? Or x/2 bytes? What if the application is expecting a file name and you send it characters not valid in one? What if you do illogical things with protocol sequence numbers?
Next page: More fuzzing logic.
More fuzzing logic
So why not do illegal things with the protocol, too? Why not just send a zillion FF bytes? First, proxy servers will usually block out transmissions that are illegal at the protocol level, so application handlers dont usually have to deal with them. Second, there are an infinite number of possible attacks of this type, and they dont usually result in anything of interest, so its a black hole from the testing standpoint.
Even while staying legal, there are test cases with a theoretically infinite series of possibilities. Consider an HTML <A> tag with multiple hrefs in it: <A href=”sdfsdf” href=”sdfsdf” href=”sdfsdf”…> For purposes of practicality, you can cap the number of these at some very high value.
beSTORM is designed to be used in a development environment in concert with a debugger, in order to elicit errors in the application being tested, such as subtle overflows. This is a difficult task to do in a user environment, another problem with much testing. A large beta test, for example, is a good thing to do, but its not likely to find abusive error cases of the type beSTORM is designed to catch.
Thoroughly exercising a protocol can result in a very large number of test cases and a long test period, but the automation can minimize the impact. Beyond Security claims that beSTORM has optimizations that allow it to focus first on cases that are more likely to produce results. And even better, by dividing up the test cases into sections, the tests can be made parallel, and you can often get linear scalability just by adding systems to the test. This is one way to mitigate the testers dilemma I described above.
Fuzzing is not just about network applications. There are many classes of applications for which it is useful; consider database applications, where the database programming protocol could be fuzzed. My personal favorite is window messaging, the protocol on Windows and other GUIs wherein windows send messages to one another. Especially in Microsoft Windows, this has been the subject of much vulnerability research in the last few years, and attacks on them (called “shatter” attacks) have potential for privilege escalation.
A next-generation fuzzer could be an important tool, and could result in a period of rapid vulnerability exposure. In the longer term, as testing like this becomes routine, such errors could become rare, which is a delightful scenario. Today they are all too common. Fuzzing isnt the end game in testing, since it addresses only certain classes of errors, but the more testing we do, the better security generally will be.
Editors Note: This story was updated to correct the spelling of the beSTORM product name. We also changed the reference to the original fuzzing tool to Fuzz.
Security Center Editor Larry Seltzer has worked in and written about the computer industry since 1983.
More from Larry Seltzer