Office Suites: How We Tested

eWEEK Labs' office productivity suite eValuation gauges usability, training costs and capability.

The intent of eWEEK Labs office productivity suite eValuation was to determine what types of training issues and costs organizations will likely face as their users make the transition from older versions of Microsoft Corp.s Office to Office 2003 or to an open-source alternative (namely, OpenOffice.orgs 1.1.1).


/zimages/3/28571.gifClick here to read the results of the eVal.
/zimages/3/28571.gifClick here to learn why we think open-source office suites are a better fit in small shops.

In building the eVal testbed at Corporate Partner site FN Manufacturing Inc., eWEEK Labs provided three groups—comprising users with varying degrees of Microsoft Office expertise—with an IBM ThinkPad T41 loaded with 1.1.1 or Office 2003. All the users who participated in the two-day test are using Microsoft Office 97 or Office 2000 in their daily work routines.

Although we were mainly interested in the usability and training issues associated with introducing a new office suite to the user population, we also wanted to test the viability of an open-source office suite on enterprise desktops.

Every office productivity suite is different in its own right, but the basic applications—and how people use them—are similar. At the most basic level, IT departments must provide users with word processing, spreadsheet and presentation capabilities. In addition, when migrating to any office suite—whether from an older version or to a completely new platform—macros must be transferable, and compatibility among applications must be ensured.

Office suite eVal: The Players
Ed Benincasa, vice president of MIS, FN Manufacturing
Kevin Wilson, product line manager, desktop hardware, Duke Energy
Jeff Worboys, product line manager, desktop productivity applications, Duke Energy
Testers from FN Manufacturing and related companies Level of skill* testers (had never worked with or Office 2003)
Deborah Hordych Light user
Suzan Widener Average user
Melinda Vause Average user
Doug Shaffer Average user
Office 2003 testers (had never worked with or Office 2003)
Rod Trace Advanced user
Mary Hayden Average user
Rod Sewell Average user
Rick Miller Light user
Jason Hitchcox Average user
Joe Lohre Light user
David Shealy Average user
Office 2003 testers (had tested previously)
Tonya Kennedy Average user
Joan Curfman Advanced user
Kevin Patten Advanced user
Romuald Dufour Average user
Philippe Nemery Average user
Jean-Luc Dubois Average user
Tina Sanzone Average user
*As determined by FN Manufacturing IT staff prior to test methodology development

During the eVal, which took place in late March at FN Manufacturings Columbia, S.C., headquarters, two groups of users tested Office 2003. One group had participated in a test of conducted last year at FN Manufacturing. (eWEEK Labs did not participate in that test.) The other group had used neither (any version) nor Office 2003.

A third test group evaluated Members of this group had no experience with any version of or with Office 2003.

The eVal was administered by eWEEK Labs analysts. The event was judged by Ed Benincasa, FN Manufacturings vice president of MIS and an eWEEK Corporate Partner; Benincasas IT managers; Kevin Wilson, another eWEEK Corporate Partner and product line manager of desktop hardware at Duke Energy Corp., in Charlotte, N.C.; and Jeff Worboys, product line manager of desktop productivity applications at Duke.

eWEEK Labs created a test script designed to evaluate each office suites ability to handle users daily tasks, including the creation of a macro in the word processing application, the addition of graphics to a presentation and the use of formulas within a spreadsheet program.

We provided each user with Microsoft Office-formatted sample documents. We also asked users to test how their own Microsoft Office-formatted documents rendered when launched using the test applications.

Each group had about 2 hours to work with the office suite being tested. Users were asked to report any training, compatibility or rendering problems they encountered.

After the three groups finished testing, eWEEK Labs met with all the testers to get feedback and discuss issues users ran into regarding training and compatibility.

Benincasa said he and his IT managers will use the results of the eVal to determine how FN Manufacturing will proceed with its office suite migration.

Senior Writer Anne Chen is at

/zimages/3/28571.gifCheck out eWEEK.coms Enterprise Applications Center at for the latest news, reviews, analysis and opinion about productivity and business solutions.
Be sure to add our enterprise applications news feed to your RSS newsreader or My Yahoo page: