The Doomsday Machines of Malicious Software

Nothing unusual running on your system? Don't bet on it. A small piece of code called a rootkit may be lurking in the recesses of your system.

Every week I write about evil people and the evil software they write. But nothing scares me like rootkits.

Ever hear of a rootkit? Its a surreptitious program--not necessarily malicious (but if not, then whats the point?)--that goes to great lengths to conceal its presence on your system. A rootkit intercepts file system and other calls to prevent you from seeing it in directory listings, or lists of running programs. And once it has accomplished this, it can go about its business, whatever that may be. Rootkits have been around on UNIX for many years. They started showing up for Windows about a year ago, and theres every reason to think that Windows will be where the action is from now on.

With so much going for them, of course this weeks BlackHat USA 2003 Briefings & Training has a two-day course on"Aspects of Offensive Root-kit Technology." The course is filled up. Gulp!

(Anyone else think the Department of Homeland Security is keeping track of who takes these courses? Let me know what you think of that.)

Before we all go running in circles while screaming, remember that for a rootkit to run, it needs to find its way onto your system and then be executed. It also needs a fairly high level of privilege, which presumes a fairly serious breach of security.

If youre not already being infected by viruses and Trojan horses all the time, you probably already have the sort of measures in place that would block most attempts to place a rootkit on your systems. But its also true that some vulnerabilities can go undetected for a while, and once a rootkit is installed, with sufficient rights on a trusted system, it can become a vector to compromise anything else on the network.

It also occurs to me, despite my recent skeptical treatment of heuristic virus scanners, that rootkit installation is a natural target for heuristic analysis.

Surely, the operating system or some other trusted component could track processes that hook the appropriate calls and interrupts, and then flag them. I have to think that an operating system on the lookout for such behavior would be much harder for a rootkit to slip past. In the meantime its possible for conventional pattern-based antivirus programs to detect known rootkits, but only known ones.

At the same time, everyone should realize that it is possible to detect a rootkit, but its just not easy. Discovery will require that you to suspect its presence and have a decent instinct for forensic examination of your system.

For instance, a rootkit may hide its presence in Windowss Task Manager, but it will still consume memory. Perhaps you might notice that theres less memory available than there should be. Another symptom might be a machine operating slower than one with a supposedly identical configuration.

On the other hand, the rootkit could be installed as a service and utilize Windowss practice of glomming many services together in single process blocks. But once you have a strong enough suspicion, you can boot into the Windows Recovery Console or to boot CD-based tools to examine the system without running the infected installation.

Then again, you cant just block all software that looks and acts like a rootkit. Antivirus software, for example, is likely to hook most of the same system calls as rootkits. The answer is a trust relationship: Probably no such application or driver should be allowed to run without a digital signature from a trusted source, and the user should be explicitly warned in detail what is happening.

Like a scene out of Invasion of the Body Snatchers, rootkits inspire paranoia and fear, and yet there is something there. Have they taken over your computer? Things may not be entirely what they seem.

Security Supersite Editor Larry Seltzer has worked in and written about the computer industry since 1983.