By Peter Coffee  |  Posted 2006-10-16 Print this article Print

When eWEEK Labs looked at Parasofts Jtest 8 Java test tool, we were reminded of the travails of the Apollo 13 mission. We didnt consider Jtest 8 to be a desperately jury-rigged approach to programming problems; rather, we found it to be a valuable leap beyond past tools for Java-code-error diagnosis and correction, but one that brings with it some challenges.

The product, released in September, comes at what some may consider a moon-mission price: $50,000 for a Server Edition license that includes five developer seats and $3,495 for a single-user version that lacks some of Version 8s best new features. The product is offered on Microsofts Windows, Linux and Sun Microsystems Solaris platforms. eWeek Labs tested the single-user version on Windows and did a walk-through of the Server Editions high-end features using a Parasoft installation of the Windows server product.

At a time when a growing slate of first-rate development tools are offered at zero cost, were sure its harder than ever to get a four- or five-figure authorization for what some managers may still consider the luxury of test automation. We can only say that the cost of Jtest 8, measured in equivalent developer hours, seems likely to represent good value in most enterprise development shops.

Wed add that in the current environment of growing scrutiny of business process governance, theres an additional return to be realized from investment in any aspect of process formalization and documentation. If the chief technology officer cant find any budget for code-test tools, perhaps the chief compliance officer is the next potential source of funds.

Our last full review of Jtest was of Version 5 in October 2003, and we continue to like what impressed us then: The Jtest 8 environment makes it easy to specify and execute tests, and the product offers what are usually relevant suggestions for what it terms Quick Fix corrections.

We regret to note, though, the persistence of a problem that our review of Version 5 described in some detail: Undoing a Quick Fix edit reversed the change that had been made to our code, but the corresponding error alert was not restored unless we reran the code tests. We continue to hope that Parasoft will address this obvious and annoying flaw.

Beyond A Problem

When a programmer first suspects a serious bug, its common for those nearby to hear some muttered variation on an Apollo 13-inspired cliché: "Houston, we have a problem." Software-testing tools have typically contented themselves with sending a similar message to the developer: "Theres something wrong with the code." Were not minimizing the difficulty of crafting a tool that does this well, but its clearly not enough.

All too often, the defect being reported by a software-testing tool is likely to be a mere symptom—just as James Lovells radio transmission, in full, was, "Houston, weve had a problem: Weve had a Main B bus undervolt." Of course, the root of the problem was not the loss of electrical power but rather the serious explosion that only incidentally caused major damage to the electrical system.

Developers, like astronauts, want to get quickly to the point of knowing whats really wrong—with rapid further progress toward getting it fixed. Jtest 8 makes significant progress on both fronts. It offers more comprehensive tools for following a fault across the boundaries of a multimodule application, and it integrates useful aids to the process of code review and repair.

Leading the list of Jtest 8s diagnostic improvements is Bug Detective, which uses an innovative approach that we can best describe as combining elements of conventional static code analysis with execution simulation.

Bug Detective addresses the long-standing coding problem thats been colorfully described as "arson investigation." Conventional debugging, its often been said, is entirely too much like rooting through ashes in search of whats left of the match that started the fire. Jtest 8s Bug Detective instead anticipates problems by using customizable rules to identify what Parasoft terms "suspicious points" in the code. These might be, for example, points at which Java code might conceivably throw an exception due to the presence of a null pointer.

Bug Detective then traces back through the code to determine execution paths that might lead to that point and establishes whether its possible for (in this case) a null value to arrive there.

We liked Bug Detective because we observed it effectively solving two problems with other testing methods. First, it found problems that involved multiple classes without laborious construction of complex test cases. Second, it avoided false positive diagnoses by reporting only faults for which it could actually establish a triggering scenario. Potential buyers should note, though, that Bug Detective is a high-end feature reserved for the Server Edition of Jtest 8.

Reviewing the Situation

Also of note in Jtest 8 is the Code Review module, which extends the domain of software-testing tools into territory formerly considered the turf of source code control. Installation of the Code Review capability deploys a scanner utility on a single Windows or Linux machine that has access to a source code repository for a CVS, StarTeam, ClearCase or Subversion configuration management system.

A simple text file of property names and values then defines Code Review relationships, such as which e-mail address should receive notice from the scanner of code changes made by any given developer. Configuration and scheduling of scanner operations lack the polished convenience of Jtest 8s Eclipse-based integrated environment. Administrators must rely on the crude, if familiar, procedures of editing a text file and setting up a generic job scheduler. The resulting task assignments and workflow guidance, though, are smoothly integrated into Jtests graphical environment.

Further strengthening the governance proposition of Jtest 8 is its new facility for test case parameterization, which we found somewhat cumbersome to use, but still a valuable tool.

Technology Editor Peter Coffee can be reached at peter_coffee@ziffdavis.com.

Next page: Evaluation Shortlist: Related Products.

Peter Coffee is Director of Platform Research at salesforce.com, where he serves as a liaison with the developer community to define the opportunity and clarify developersÔÇÖ technical requirements on the companyÔÇÖs evolving Apex Platform. Peter previously spent 18 years with eWEEK (formerly PC Week), the national news magazine of enterprise technology practice, where he reviewed software development tools and methods and wrote regular columns on emerging technologies and professional community issues.Before he began writing full-time in 1989, Peter spent eleven years in technical and management positions at Exxon and The Aerospace Corporation, including management of the latter companyÔÇÖs first desktop computing planning team and applied research in applications of artificial intelligence techniques. He holds an engineering degree from MIT and an MBA from Pepperdine University, he has held teaching appointments in computer science, business analytics and information systems management at Pepperdine, UCLA, and Chapman College.

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel