Most security vulnerabilities are software bugs—in the strict sense of the word. And most of these bugs would be considered innocuous, perhaps in an environment where people arent trying to break the program. But then, we come to the Internet.
The most famous class of such bugs is the buffer overflow, by now the kind of term that makes it into your local paper when another Windows flaw makes the news. But in recent years a new type of vulnerability is being exploited more frequently: Integer manipulation bugs.
Not all integer manipulation bugs are integer overflows—some of them are underflows. Still, the class of error is usually referred to generically as overflows.
The basic problem is that integers in computers have a finite range. For instance, the rage of a signed 16-bit integer is -32767 to 32767.
What happens if arithmetic moves the value out of that range? The number could easily turn out to be massively larger or smaller than the expectation of the programs logic. Another example is a number that turns out to be negative instead of positive, changing the result of an “if (a<b)” comparison from what it was originally designed to be.
And then there are errors relating to the effects of integer promotion. When operations are made on integers of different sizes, say a short and a long, the smaller one is promoted temporarily to the larger size, and the result is potentially truncated back to the smaller size.
So what can go wrong just because a number is not what it should be? Some of those numbers are used for important stuff.
Consider this example (lifted without permission from Michael Howards excellent article on MSDN: Reviewing Code For Integer Manipulation Vulnerabilities):
bool func(size_t cbSize) { if (cbSize < 1024) { // we never deal with a string trailing null char *buf = new char[cbSize-1]; memset(buf,0,cbSize-1); // do stuff delete [] buf; return true; } else { return false; } }
For the non-C coders out there, this function tests to see whether the variable cbSize is less than 1024, allocates it and then zeros out a buffer of cbSize-1 bytes, and then deletes it.
The program carefully checks to make sure the number isnt too big, but what if its too small?
We can see that size_t is unsigned, so it cant become negative, but it can get to be zero. When you subtract 1 from it, the value 0 wraps around to the 32-bit value 0xFFFFFFFF.
Oops—you just allocated a 4GB block of memory, and unhappily, HAL wont be opening the pod bay door.
Next Page: How Integer Errors are Exploited
How Integer Errors are
Exploited”>
The code above shows how integer errors are not directly exploitable. Attackers need to look for consequences of the integer error, in this case a buffer overflow or underflow, that themselves are exploitable. Its the same principle when the results of a comparison are reversed because one value is very large instead of very small.
Who, even just a few years ago, would have thought that a program could be remotely exploitable because of how it added two numbers together? But some of the most serious recent vulnerabilities are in this class.
Consider the Integer overflow in the Sun RPC XDR library. This nasty vulnerability had multiple exploitable buffer overflows and, even though it was originally Sun code, it made its way in to libc and glibc, and just about every other *NIX out in the world.
A bug in Microsofts Javascript implementation, JScript, is another great example.
As explained in the CVE (common vulnerability and exposure) entry on the bug: an “Integer overflow in JsArrayFunctionHeapSort function used by Windows Script Engine for JScript (JScript.dll) on various Windows operating system allows remote attackers to execute arbitrary code via a malicious Web page or HTML e-mail that uses a large array index value that enables a heap-based buffer overflow attack.”
This is one of those promotion issues. Jscript needs to create a buffer to store some data in order to sort it. When it calculates the size of the buffer to allocate, Jscript multiples two numbers and can, in some circumstances, come up with a value greater than 32 bits. It then uses a 32-bit value to store this number, chopping off the high bits, and creating a buffer far smaller than it should be. No error is generated, so the program then goes on to write more data than there is room for in the buffer.
There are many other examples. An integer overflow in the Apache Web server led to buffer busting and reports of a worm. There are similar reports on FreeBSD and OpenSSH.
Its hard enough for developers to write good software for users without having to consider that people will attack a program to find bizarre behaviors. In the long term, problems such as over-and underflows will be prevented to some extent by virtual machines.
Yet until the day that programmers become perfect, or that no code is released unless its been tested for years, these vulnerabilities will always be with us. Its just too darn hard to avoid them.
Security Center Editor Larry Seltzer has worked in and written about the computer industry since 1983.
More from Larry Seltzer
