Wield the Shield: How Trustworthy Is Your OS?

So-called trusted operating systems have been used for years to lock down sensitive data, but with security attacks rising, enterprises are looking to fortify their defenses. Red Hat, Novell and Sun are introducing trusted features. eWEEK Labs examines th

Trusted operating systems have been used for some time to lock down the most sensitive of information in the most sensitive of organizations. But with security concerns rising and changing by the hour, its now a matter of trust for any organization looking to tighten its computing ship.

Several vendors, including Red Hat, Sun Microsystems and Novell, are responding by adding and/or improving trusted elements in their operating system offerings.

Operating systems are designed to do what theyre told, and we tell them what to do by running applications. However, whether through bugs or malicious exploits, applications can tell an operating system to do things that no one intended or wanted it to do—at least no one authorized to do so.

Todays mainstream operating systems are trusting—they trust that the applications running on them are doing what users intend the apps to be doing. These operating systems havent been designed to limit applications from doing more than theyre intended to do.

This can be bad enough when youre talking about individual users, whose privileges can be limited enough to ensure that they cant modify system files.

But many applications, including many server apps, require some root-level rights to do their jobs in the first place. Once subverted, one of these applications can be wrangled into causing all manner of mischief (and into covering its tracks, to boot).

Enter trusted operating systems.

Rather than trusting the apps they host, trusted operating systems include functionality thats intended to restrict the damage an exploited application can wreak by limiting it to only those capabilities and rights it requires to get its job done.

While trusted operating systems have a lot to offer, theyre famously tricky to manage—with tight application control comes plenty of room for incompatibility.

As a result, trusted operating systems have tended to occupy a slender niche and, in turn, have lacked the full attention of operating system vendors and of the software and hardware vendors whose certifications and support are critical when working with products that are challenging to manage.

However, things have been changing during the past few years, as makers of general-purpose operating systems have been pushed to include trusted functionality in their mainstream products.

Most notably, Sun, Red Hat and Novell are each shipping enterprise-class operating systems with built-in trusted functionality available out of the box—specifically, provisions for finer-grained access controls.

Accordingly, enterprises have more opportunity than ever to defend themselves from software exploits and bugs by relying on operating systems that view the applications they host with a healthy dose of skepticism.

eWEEK Labs has put the application lockdown options from Sun, Red Hat and Novell through their paces, installing and evaluating this lockdown functionality in the context of securing the Apache/MySQL/PHP-based Mediawiki Web application in hopes of providing a starting point for eWEEK readers own evaluation of these technologies.

Sun and Solaris 10

Solaris 10 ships with a bolstered access-control scheme, called Process Rights Management, that extends the standard Unix permissions under which processes have to run as root with a set of privileges that administrators can parcel out to processes as need be.

While Solaris 10 makes it possible to get rid of the superuser account, it doesnt do so by default (to preserve backward compatibility).

However, rather than break privileged rights into all or nothing, in Solaris 10, the privileged rights are broken into all or basic—with basic being a handful of specific privileges that together represent the rights that a regular Unix user would possess.

Unlike with a standard Unix or Linux system, however, administrators can use Solaris 10s tools to add rights to a process beyond the basic set, as well as remove rights from the superusers "all" set.

There are four ways to add and drop processes in Solaris.

First, there is the ppriv command-line utility, with which you can view or modify the privilege set of a particular process. This command is handy for debugging system privileges, and by running "ppriv -lv" you get a nice list of the 50 or so permissions covered under Solaris rights scheme.

Another route to fine-tuning the rights with which applications run is to use Solaris new Service Management Framework. Following a tutorial on Suns site, eWEEK Labs configured an Apache installation from The Apache Software Foundation to start as a non-root user. We then added the rights that Apache required to our non-root-users basic privilege set.

You can also adjust the rights with which users and applications run in Solaris 10 by creating roles with the needed range of rights. Roles in Solaris are akin to user accounts, but they lack log-in rights; users must be logged in to assume a role.

Finally, Solaris 10 offers APIs for building privilege-aware applications—those that can assume and drop the rights they require from the set of privileges theyre accorded by the system administrator.

/zimages/1/28571.gifWhen will Microsoft be trusted? Click here to read more.

With the variety of methods available for using it, and with the provisions throughout Solaris 10 for maintaining backward compatibility, Process Rights Management definitely takes some getting used to.

In our testing so far, weve found Solaris Containers feature to be a much simpler-to-approach means of process confinement, although you can use Containers and Process Rights Management together. While Red Hat and Novell are each working on providing similar functionality through Xen virtualization, the Solaris approach is much leaner, more manageable and, most important, more mature.

The trusted functionality in Solaris 10 is descended from Suns Trusted Solaris, the companys stand-alone trusted operating system.

Marking the completion of the merging of the Solaris and Trusted Solaris products, Sun plans to release this summer Solaris Trusted Extensions, a set of add-on components for Solaris 10 that will enable organizations running Trusted Solaris to move to Solaris 10.

These extensions will add to Solaris the multilevel security functionality of Trusted Solaris, along with desktop integration that will allow organizations to, for example, keep classified data displayed in one window from being pasted into an unclassified document in another window.

Next Page: Red Hat and SELinux.