eWEEK Labs fourth OpenHack online security test provided a number of lessons on how to protect corporate data. It also offered further proof that when it comes to security, the devil truly is in the details.
OpenHack is an interactive security evaluation, where a heterogeneous deployment of corporate IT products is evaluated for its ability to withstand the rigors of the Web. Whereas past OpenHack tests have focused on firewalls, IDSes (intrusion detection systems) and trusted operating systems, OpenHack 4 focused on Web application security.
At the heart of this years test, which started Oct. 22 and ended Nov. 8, was a Web application we use in production here at the Labs. We asked Microsoft Corp. and Oracle Corp. to harden the application and then deploy it on their choice of operating system, Web server, application server and database platforms. The hardened applications and the servers upon which they ran were hosted online at www.openhack. com, and a challenge was issued to aspiring hackers the world over to try to complete any of five hacking challenges.
What happened during the course of the test drives home the importance of code review (and review and review again) and a multilayered defense strategy. We also saw clearly the benefit of structured Web programming frameworks that automate some of the work required to write secure code (click here to see how the site fared).
Indeed, its the little things that really matter in security: Each of the two successful cross-site scripting attacks was made possible by a single mistake on a single line of code in the test application.
Its important to reiterate that the OpenHack application is specific to eWEEK and to this test. We discovered no security problems (or any significant operational problems) in any of the other components tested, including Microsofts and Oracles respective Web servers, application servers and databases; in the host Windows 2000 Advanced Server and Red Hat Inc. Red Hat Linux Advanced Server 2.1 operating systems; or in the OpenBSD firewalls or Extreme Networks Inc.s switches we used.
During the 18 days the test was open, only one of the five security challenges we posed was successfully accomplished.
Poteet describes how a functionality bug in the application tipped him off to a possible weakness in the code that he was later able to exploit. He was also able to perform a second cross-site scripting attack by manipulating a URL field to take advantage of the way the application checked unusual data types.
In both cases, Poteet was able to discover exactly where several undiscovered bugs lay in the application simply by careful observation combined with his own experience. He was also able to do this without access to source code—one more reminder that keeping source code secret is very thin protection at best.
: OpenHack Wrap”>
None of the other four, more serious test challenges—discovery of application source code, Web page defacement, theft of credit card data and ability to issue arbitrary SQL commands to the database—was accomplished.
Oracle and Microsoft both had hardening carte blanche, as long as they stuck with production software and kept the basic structure of their applications similar to the original.
Clearly, the hardening done by both companies was good enough to get their respective installations through the OpenHack gauntlet largely unscathed, but some readers questioned how realistic it would be for IT security staff and developers to harden an application to the degree that Microsoft and Oracle hardened the OpenHack application.
As one reader wrote, “First, how many companies are going to be able to afford to have a staff of Microsoft and Oracle people come to their site and lock it down? And how many companies have a staff of senior Microsoft and Oracle programmers writing the software? … I would like to see a real test. That is, two systems are shipped in from the manufacturer; the systems are set up; and a new Microsoft Certified Systems Engineer or Oracle Certified Database Administrator has four hours to lock it down. Then open it up to the public. This is a real test. This is the real world.”
As much as possible, we hope to test technology rather than people, but the human resources available will always play the main role in whether an organizations IT systems are secure.
We dont consider the steps taken by either vendor unusual or beyond the reach of a careful administrator, but one thing we have learned from past OpenHack tests is that nearly any system can be made extremely secure while still retaining adequate functionality when the right person is at the keyboard.
We felt that it took relatively few steps to harden to acceptable levels the operating systems and Web servers on the Unix systems used in the test. Individual developer and security staff experience will, of course, vary widely, given individual backgrounds.
In general, Microsofts ASP (Active Server Pages) .Net Framework and Visual Studio .Net tools provided significant development time and application security benefits over the JSP (JavaServer Pages) platform Oracle used. However, Microsoft also took security precautions (such as extensive use of IP Security and virtual private networks) that Oracle did not, something that made the Microsoft setup more complex and tricky to manage right from the start.
Both companies took advantage of standard tools to make system hardening simpler. Windows 2000s security policy tools made it easy to apply a wide variety of security changes to the operating system once a policy file had been written. Use of standardized security policies and hardening scripts is also a very effective security methodology. (Oracle used the free Bastille Linux hardening scripts on its servers.)
Both Microsoft and Oracle have written detailed white papers describing their precise hardening processes. These are posted at OpenHack 2002 Downloads, along with several other OpenHack resources.
As weve seen before in our OpenHack tests, IT security is a global concern. The first two OpenHacks were brought down by the same hacker, Lluis Mora, who hailed from Gibraltar. The top 20 attackers of OpenHack 4, as detected by IntruVert Networks Inc.s IntruShield 2600 IDS monitoring the site, were from eight different countries across four continents (click here to see map).
While running, the site weathered almost 53,000 attacks. The most common attacks detected by the IDS were primarily standard Web server attacks based on URL encoding and Web directory tree escape attacks using directory traversal attempts. Other common attacks included low-level TCP/IP pokes and probes of various kinds. (Click here to see top 20 attack types.)
Most of the Web server attacks were just background radiation from all the Web server worms now eternally circulating, looking for easy prey. Any Web server kept up-to-date on patches is safe from these attacks, and our OpenBSD-based firewalls effortlessly rebuffed the TCP/IP-level attacks.
: OpenHack Wrap”>
Much more dangerous were the approximately 240,000 malformed or illegal HTTP request attacks received by the OpenHack Web servers. Since this test was all about Web application security, manipulating HTTP requests and the parameters to those requests was the modus operandi of the more serious hackers (and is exactly the method Poteet used in his attacks).
Accepting user-supplied parameters, taking actions based on that data and then storing the data in a corporate database is an inherently dangerous activity; organizations need to use all the standard tools at their disposal to help filter out dangerous input before it reaches their applications.
Both Oracle and Microsoft used a multilayered approach to scrubbing HTTP traffic.
Oracle added specific entries in its Apache Web server httpd.conf file to immediately block URL requests containing suspect characters before the request could reach the application server. Microsoft used its freely downloadable URLScan traffic filter to scan and clean HTTP traffic before it reached the ASP engine. Both approaches were very effective, blocking many tens of thousands of dangerous requests.
Oracle and Microsoft handled two other challenges—authenticating users and validating parameters—very differently, highlighting more fundamental differences between the Java and .Net development platforms.
The Oracle application was based quite closely on eWEEK Labs reference application, which we had originally written in JSP and Java. This application used its own logic to authenticate users, check access control rights to pages and validate parameters.
In fact, a significant portion of code on each page is devoted to security, with hundreds of such checks. However, missed checks in two places discovered by Poteets trial-and-error probing resulted in the successful cross-site scripting attacks. Both bugs were in our original code as well and had been there for some time, demonstrating how difficult it can be to find stealthy errors in existing code.
Its Internet Information Services Web server cant run JSP, so Microsoft had to rewrite the OpenHack application from scratch (although Microsofts application was functionally identical to the original one).
Microsoft wrote the application using ASP .Net, which has pre-written APIs that make user authentication and parameter validation simple. For example, parameter validation is done declaratively using the visual Web form builder in Visual Studio .Net, and all the code to match user input to a validation pattern regular expression is generated automatically. This approach helps to ensure that no security checks are missed and frees developer time for back-end business logic.
Oracle staff recommended that Java developers take advantage of various third-party Java frameworks for user management and forms handling to get the benefits of using a more structured development process.
In particular, they advised using The Apache Software Foundations Struts project, at jakarta.apache.org/ struts. Struts provides a declarative forms language that automates field and form state management as well as built-in data validation features, and has emerged as a leading structured Web application development tool for Java.
We hope that the slings and arrows aimed at our OpenHack site will help enterprise security managers better gauge the threat to their own systems and take the appropriate steps to mitigate it.