Apple Places Encrypted Binaries in Mac OS X

A computer researcher has made public information that Apple Computer has encrypted at the binary level critical parts of its Mac OS X operating system.

These "Apple-protected binaries" can serve to protect the OS from being pirated and also to make it "nontrivial" to run Mac OS X on non-Apple hardware, said Amit Singh, a member of Googles technical staff in Mountain View, Calif., and the author of "Mac OS X Internals: A Systems Approach." Singh has also given lectures on Mac OS X to the National Security Agency and at Apples main campus in Cupertino, Calif.

According to Singh, the parts of Mac OS X that are protected include the Finder and Dock applications, as well as parts of Rosetta (Mac OS Xs application for running Power PC applications on an Intel-based Mac) and services that manage the user interface.

Singh noted that his list was not exhaustive.

Resource Library:

Much of Mac OS X is open source, including Darwin, an entirely functional, open-source operating system based on FreeBSD 5.0 and the Mach 3.0 microkernel, and the basis for Mac OS X.

The Apple-protected binaries signal their protected status by setting a special bit in the header, Singh said. When any binary is called upon by the system, the kernel checks to see if it is Apple-protected; if it is, the kernel unencrypts the code through an "unprotect" operation.

This operation, Singh noted, includes a "dsmos_page_transform" command, in which "dsmos" stands for "Dont Steal Mac OS X". He also found a "Dont Steal Mac OS X.kext" kernel extension in the operating system.

"A lot of times, encrypted binaries are used as piracy protection," said Bruce Schneier, founder and chief technology officer of Mountain View, Calif.-based Counterpace Internet Security. "Its a common technique," he said.

"But more often, and probably what its used for here," he added, "is as anti-reverse engineering."

Click here to read about an exploit for an unpatched vulnerability in the Apple Airport driver that ships with some PowerBook and iMac computers.

Schneier noted that encrypted binaries can affect application performance due to the extra decoding step before they can be executed.

However, he said, "As computers grow faster, theres more processing power to do stuff like this.

"The devils in the details," he said.

Speaking to concerns about privacy, Schneier said, "Theres nothing sinister here."

"This is a method for Apple to protect its code," he said, adding that for people who still want to try to get Mac OS X running on commodity PC hardware, "you can get around it, but not easily."

Apple representatives were not available to comment.

Check out eWEEK.coms for the latest news, reviews and analysis on Apple in the enterprise.