Office Suites: How We Tested

By Anne Chen  |  Posted 2004-04-26 Print this article Print

eWEEK Labs' office productivity suite eValuation gauges usability, training costs and capability.

The intent of eWEEK Labs office productivity suite eValuation was to determine what types of training issues and costs organizations will likely face as their users make the transition from older versions of Microsoft Corp.s Office to Office 2003 or to an open-source alternative (namely, OpenOffice.orgs 1.1.1).

Click here to read the results of the eVal.
Click here to learn why we think open-source office suites are a better fit in small shops.
In building the eVal testbed at Corporate Partner site FN Manufacturing Inc., eWEEK Labs provided three groups—comprising users with varying degrees of Microsoft Office expertise—with an IBM ThinkPad T41 loaded with 1.1.1 or Office 2003. All the users who participated in the two-day test are using Microsoft Office 97 or Office 2000 in their daily work routines.

Although we were mainly interested in the usability and training issues associated with introducing a new office suite to the user population, we also wanted to test the viability of an open-source office suite on enterprise desktops.

Every office productivity suite is different in its own right, but the basic applications—and how people use them—are similar. At the most basic level, IT departments must provide users with word processing, spreadsheet and presentation capabilities. In addition, when migrating to any office suite—whether from an older version or to a completely new platform—macros must be transferable, and compatibility among applications must be ensured.

Office suite eVal: The Players
Ed Benincasa, vice president of MIS, FN Manufacturing
Kevin Wilson, product line manager, desktop hardware, Duke Energy
Jeff Worboys, product line manager, desktop productivity applications, Duke Energy
Testers from FN Manufacturing and related companies
Level of skill* testers (had never worked with or Office 2003)
Deborah Hordych
Light user
Suzan Widener
Average user
Melinda Vause
Average user
Doug Shaffer
Average user
Office 2003 testers (had never worked with or Office 2003)
Rod Trace
Advanced user
Mary Hayden
Average user
Rod Sewell
Average user
Rick Miller
Light user
Jason Hitchcox
Average user
Joe Lohre
Light user
David Shealy
Average user
Office 2003 testers (had tested previously)
Tonya Kennedy
Average user
Joan Curfman
Advanced user
Kevin Patten
Advanced user
Romuald Dufour
Average user
Philippe Nemery
Average user
Jean-Luc Dubois
Average user
Tina Sanzone
Average user
*As determined by FN Manufacturing IT staff prior to test methodology development
During the eVal, which took place in late March at FN Manufacturings Columbia, S.C., headquarters, two groups of users tested Office 2003. One group had participated in a test of conducted last year at FN Manufacturing. (eWEEK Labs did not participate in that test.) The other group had used neither (any version) nor Office 2003.

A third test group evaluated Members of this group had no experience with any version of or with Office 2003.

The eVal was administered by eWEEK Labs analysts. The event was judged by Ed Benincasa, FN Manufacturings vice president of MIS and an eWEEK Corporate Partner; Benincasas IT managers; Kevin Wilson, another eWEEK Corporate Partner and product line manager of desktop hardware at Duke Energy Corp., in Charlotte, N.C.; and Jeff Worboys, product line manager of desktop productivity applications at Duke.

eWEEK Labs created a test script designed to evaluate each office suites ability to handle users daily tasks, including the creation of a macro in the word processing application, the addition of graphics to a presentation and the use of formulas within a spreadsheet program.

We provided each user with Microsoft Office-formatted sample documents. We also asked users to test how their own Microsoft Office-formatted documents rendered when launched using the test applications.

Each group had about 2 hours to work with the office suite being tested. Users were asked to report any training, compatibility or rendering problems they encountered.

After the three groups finished testing, eWEEK Labs met with all the testers to get feedback and discuss issues users ran into regarding training and compatibility.

Benincasa said he and his IT managers will use the results of the eVal to determine how FN Manufacturing will proceed with its office suite migration.

Senior Writer Anne Chen is at

Check out eWEEK.coms Enterprise Applications Center at for the latest news, reviews, analysis and opinion about productivity and business solutions.
Be sure to add our enterprise applications news feed to your RSS newsreader or My Yahoo page:  
As a senior writer for eWEEK Labs, Anne writes articles pertaining to IT professionals and the best practices for technology implementation. Anne covers the deployment issues and the business drivers related to technologies including databases, wireless, security and network operating systems. Anne joined eWeek in 1999 as a writer for eWeek's eBiz Strategies section before moving over to Labs in 2001. Prior to eWeek, she covered business and technology at the San Jose Mercury News and at the Contra Costa Times.

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel