Segue Software Inc. has significantly improved its SilkPerformer V Web application testing and load simulation tool by streamlining test setup procedures and incorporating several innovative analysis utilities.
IT managers should consider SilkPerformer V to help with regression testing of complex Web sites that allow numerous user interactions, such as e-storefronts or online information clearinghouses. In eWeek Labs tests, the products new TrueLog capability allowed us to track down problems with user interactions by recording the screens as they appeared to end users while also providing us with information about where the probable breakdown occurred.
SilkPerformer V started shipping last month and is competitively priced, although at $18,000 for a six-month, 5,000-virtual-user license, it might give potential buyers pause. The number of licensed users “decrements” as tests are run, so, for example, a load test with 100 virtual users could be run 50 times (which should be more than enough to determine how an application will perform). Other license options, including one for perpetual use, are also available.
Would-be customers should check the list of predefined performance modules to make sure the products they use are supported. Competitors such as Mercury Interactive Corp.s $3,495 Astra LoadTest 5.0 provide a long list of prebuilt monitors, whereas SilkPerformer V offers modules only for AIX system information, IBM WebSphere and SilverStream Software Inc. applications, and a few others.
Perhaps the most impressive new feature in SilkPerformer V is True-Log on Error. During tests, we were able to find some (glaring) Web application problems on our BenchTen test site. SilkPerformer V saved the problem screens, such as unavailable page errors. This feature neatly saves only the error, not the entire session, making it a timesaving, resource-friendly way to diagnose Web application problems.
Reality Check
A common complaint about load-generation and simulation tools such as SilkPerformer is that test loads are very flat. Segue answered that concern with All Day Workload, which enables SilkPerformer V to create more-realistic tests. We handily created a daily rush of virtual users logging on from 5 to 8 a.m. to check the news at BenchTen, then simulated a percentage of those users going to the BenchTen store site to purchase widgets. All Day Workload allowed us to vary the number of virtual users across a long period of time, resulting in an interesting diagnosis of how well the site worked.
We suggest adding a random burst option to make tests even more realistic. Nearly every site has periodic, unpredicted interest, whether because of market fluctuations or breaking news. Although we could schedule a burst of interest in the test Web site every day at the same time, the random appearance of a traffic spike would have proved useful too.
The SilkPerformer V base-line wizard enabled us to set up workloads that provided more-realistic tests of our Web site and helped us to determine the threshold response times for which we should test. In the case of our Bench- Ten site, it took several seconds to get a log-on screen but only a few seconds more to get from a table of contents to a particular news story. These measurements made it much easier for us to interpret the performance of the Web site when it was under a test load of several hundred simultaneous users.
Users of previous versions of Silk- Performer will see improvements in its reports, which previously were text-only. We found the new HTML reports much easier to read and analyze.
Senior Analyst Cameron Sturdevant is at cameron_sturdevant@ziffdavis.com.
SilkPerformer V
SilkPerformer V
| USABILITY | A |
|---|---|
| CAPABILITY | B |
| PERFORMANCE | A |
| INTEROPERABILITY | C |
| MANAGEABILITY | B |
Segues SilkPerformer upgrade is a good choice for developers who need to stress test applications before deploying them to the Web. New tools ease simulation setup, and extended tests can be run with a changing number of simulated users.
SHORT-TERM BUSINESS IMPACT // SilkPerformer V should speed up regression tests, thereby reducing the time between Web application conception and actual use. Staff will likely spend far less time testing and more time fine-tuning.
LONG-TERM BUSINESS IMPACT // Because SilkPerformer requires very little maintenance while providing useful guidance to developers, it should contribute to reducing the costs associated with Web application development.
PROS // Pinpoints most application errors with screen shots; easy to set up tests.
CONS // Not as many predefined performance modules as competitors.
Segue Software Inc., Lexington, Mass.; (781) 402-1000; www.segue.com