Tools Offer Near-Real Online Tests

Web site testing, even for companies with abundant resources, has been something of a dark art because it is far easier and less expensive to test the components of networks and infrastructures than it is to test entire sites.

However, as cracks and crashes get more complex and online customers demand better service, Web site managers, especially at large e-commerce sites, should look at real-world types of tests because they can provide a more accurate appraisal of infrastructure performance. A wide variety of vendors have tools that can simulate Web loads, but these solutions in general are confined to the test lab.

Now, a new generation of tools aims to offer near-real tests of entire Web systems. eWeek Labs recently got some hands-on time with outsourced online testing tools from Keynote Systems Inc., and we were impressed with the ease of use and capability of the tools.

We evaluated Keynotes Transaction Perspective, a trend and reporting tool; Test Perspective, a load-testing tool; and Wireless Perspective, a test for wireless networks. All three leverage Keynotes growing worldwide infrastructure.

Keynotes strongest selling point is its ability to give IT managers a Web-based platform on which to run their tests through the Internet, without forcing them to set up testing points at multiple locations.

Currently, Keynote has load generators on the East and West coasts and in Europe. Using these generators, IT managers can send traffic through various backbones to see how well their sites perform. In addition to the load generators, Keynote also has numerous automated agents around the world that hit Web sites and track performance.

New Perspectives

During tests of Transaction Perspective, Test Perspective and Wireless Perspective, we had access to 102 agents running in a wide variety of locations and on many ISPs systems. We found Transaction Perspective to be the most useful of the three for day-to-day management of Web sites because of its trending and alerting capabilities.

Using Transaction Perspective, we were able to track the performance of the eweek.com Web site over the course of several days and could find out what times our site had the most latency. The ability to perform trending analysis was extremely useful and made it easy to pinpoint problem areas.

Transaction Perspective comes with diagnosis tools, which allowed us to run standard commands such as nslookup and ping to see how well our site was performing in cities around the world, including Beijing and Dublin, Ireland, as well as across the United States.

Using Transaction Perspectives instant-diagnosis option, we could see exactly how long it took test agents to download all the elements on the Web site. Instant diagnosis breaks down each transaction and shows the response time for acquiring data while also showing if a transaction failed or required a retransmission.

Transaction Perspectives alarm feature automatically sends e-mail to an IT manager if site availability drops below satisfactory levels.

The user interface for Transaction Perspective was easy to navigate, and the help pages and documentation were more than adequate.

Test Perspective allows IT managers to run load testing using Keynotes infrastructure. Using Keynotes Transaction Recorder software, we easily created test scripts by simply pointing and clicking our way through the target Web page. We could also do simple tasks such as create users accounts and add items to shopping carts.

Once the scripts were created, we used Test Perspectives interface to schedule the test and have it carried out by our simulated users.

In Test Perspective 4.0, load is ramped up and ramped down in a manner similar to most benchmarks. The 5.0 release, which is slated to arrive near the end of this month, will give users the option of a new Arrival Rate methodology, which will stagger the addition of client load, thus making the benchmark process more realistic than the linear increase of load we see in nearly all benchmarks.

IT managers who need professional help to guide them through their testing can also turn to Keynotes Professional Services department for assisted testing.

Wireless Perspective is designed to allow IT managers to see how well wireless clients can access their sites. In place of standard Web clients, Wireless Perspective uses wireless clients in Seattle and New York that attempt to connect to client Web sites through various major wireless carriers.

In tests, Wireless Perspective enabled us to see how latency changed during the day. We could set up alert thresholds to ensure that the proper person will be contacted in the event of a prolonged outage.

Senior Analyst Henry Baltazar can be reached at henry_baltazar@ziffdavis.com.