A Matter of Trust

A secure OS has plenty to offer-if you can use it.

For IT administrators, one of the toughest and most thankless jobs is securing a server that will run vital enterprise applications. No matter what operating system the server is running, effectively hardening the server requires long hours and painstaking effort.

Even if you do the best job possible, however, it may be for naught if the enterprise applications running on the server arent secure. If these applications have bugs that make them susceptible to buffer overflows or scripting attacks, a hacker or worm will be able to bypass all of your hardening work to gain full access to the server system.

For a few years, trusted operating systems have been the only practical defense against the vulnerabilities of bad applications. Simply put, a trusted operating system is based on the precept that no user or application has the right to do anything,including the root user. Essentially, a trusted OS trusts nothing.

In a trusted OS, the system administrator must give specific rights to each user and application, neither of which can go beyond these rights. Even if a hacker breaks through an application, it does the hacker no good, because he or she cant do anything beyond what the application itself is allowed to do.

The strength of a trusted operating system was proved a couple of years ago during eWEEKs OpenHack 3 tests, when we pitted thousands of security experts and hackers worldwide against servers protected by Argus Systems PitBull trusted OS for Linux.

Facing the highest number of attacks of any of the four OpenHacks, the servers remained unscathed despite the fact that many attackers were able to compromise user accounts and even the root account.

Generally, the only way to beat a trusted operating system is to crack the kernel of the OS itself, which is a very difficult task, though not an impossible one, as Argus found when PitBull was defeated in a subsequent (non-eWEEK) test.

With these facts on the table, youre probably wondering why anyone would not use a trusted server operating system. Well, there are several hurdles to deploying a trusted OS. First, they are very complex to configure. Its a constant run of trial and error to configure the permissions so that your applications run properly. Second, making subsequent changes can be a nightmare, often requiring starting from scratch.

These hurdles arent insurmountable and, given the benefits of a trusted OS, companies needing to protect vital enterprise applications and data will find deploying one well worth the effort. However, it would be great if the benefits of a trusted operating system were more easily available to all. A couple of recent announcements might help.

First, Sun announced a set of new security features that will be included in Solaris 10 when it ships at the end of the year. Catching my eye in this list was the fact that Solaris 10 will include process rights management. This is a key feature of trusted OSes, such as Suns own military-grade Trusted Solaris, which makes it possible to limit the rights of every user and application.

Suns announcement was followed by Red Hats statement that Red Hat Enterprise Linux 4.0, which comes out next year, will include support for SE Linux, a secure version of Linux developed by the National Security Agency. While not a pure trusted OS, SE Linux implements many trusted OS concepts.

Of course, companies can already get Trusted Solaris, and experienced Linux administrators can already deploy SE Linux into their Red Hat systems. But these announcements should lower the bar to using these high-level security capabilities.

Whats needed now is something that helps administrators properly deploy these security measures, perhaps taking them step by step through the process, or letting them choose default security templates as starting points. Even other operating systems such as Windows Server 2003, which has capabilities to lock down the rights of processes and accounts, could benefit from these kinds of user aids.

The powerful security protections in trusted OSes and their kin are proven to protect vital systems—but only if administrators can deploy them.

Labs Director Jim Rapozas e-mail address is jim_rapoza@ziffdavis.com.