Stripping Away Malware's Armor

Security analysts use stealth to win the "arms race" against malware.

Getting around the defense mechanisms that malware writers erect isn't easy. From packers to run-time obfuscators, software-armoring techniques create problems for reverse engineers and security analysts.

Security pros Danny Quist and Val Smith, of Offensive Computing, have a solution-Saffron, a stealthy debugging platform they are developing. Having first presented the tool at the 2007 Black Hat conference in Las Vegas, the two demonstrated an updated version Feb. 17 at ShmooCon 4 in Washington, D.C.

The pair was able to successfully defeat two well-known packers, TeLock and Vmprotect, as well as a custom packer presented to them afterward by two members of the audience.

"Danny and I do a lot of malware analysis, and manual analysis can take hours," said Smith, chief technology officer of Offensive Computing. "We really want to be able to analyze most of the files rapidly, but to do that you have to be able to deal with the fact that most malware is obfuscated, it's encrypted, coded, compressed, so that you can't understand what's going on."

Security risks are very real in virtual worlds. Read more here.

Saffron uses page faults to assist in the unpacking process. A page fault is an interrupt that occurs when a program requests data that is not currently in real memory. Memory management uses page faults to enforce a separation of memory from kernel space and user space, Quist said.

"Basically you don't want anything running in user mode to modify kernel memory," he said. "This is implemented with something called a supervisor bit. That's something that lets the hardware know that a currently running program should not access certain memory.

"We change this method slightly," Quist continued. "What we do is to mark all of the program's memory with this supervisor bit so that it always creates a page fault. Once we get this page fault we check to see if it's our program. If it is, we log it and allow it through. When the OS gets a page fault you can see whether it was trying to read, write or execute. So we mark all the writes to memory, then watch for an execute in that memory. If we see an execute, then that's our trigger for a candidate OEP [original entry point] and we dump the memory."

Saffron uses a hybrid memory scanning and tagging mechanism to identify memory areas, as well as the dynamic run-time tracing capabilities of Intel's PIN tool to monitor the flow of executables. What makes it faster than other unpacking utilities, Quist said, is that it's an automatable process and it runs entirely on hardware.

"Since the CPU tells us when this occurs, we can largely take a hands-off approach," he said. "Other methods involve emulating a running CPU or setting debuggers, which makes for slow unpacking."

The tool relies on stealth, hiding from the malware to avoid being counteracted, Smith said.

"Basically, it's an arms race," Quist said.