Is An Unbiased Study Possible?

 
 
By Jim Rapoza  |  Posted 2002-12-09 Email Print this article Print
 
 
 
 
 
 
 

Most people, including much of the press, don't look at the details.

A recent study found that I am the best-looking man in the Western Hemisphere. Sure, I commissioned the study, and the only participants were my wife, mom and two dogs, but all most people will notice is the headline.

Of course, this is a joke, not unlike the recent study commissioned by Microsoft that tried to show that owning Windows 2000 is cheaper than owning Linux (see related story). Sure, there are cost benefits to running Windows-based servers, and Im willing to bet that in a legitimate study, they could do well in many areas versus Linux.

But this study is full of questionable assumptions. For example, its based on a five-year technical lifetime, rather than the more common three years—spreading upfront costs over a longer period.

Unfortunately, most people (including much of the press) dont look at such details, meaning there is little downside in commissioning studies that are weighted ahead of time to provide the results the sponsor wants.

This is true not only in tech studies but also in most studies we are inundated with every day.

While these kinds of studies are valuable to companies, they can cause a lot of harm, both in peoples expectations and in the publics confidence in science and research.

One of my dreams has been to start an unbiased and independent center that would serve as a resource for people and journalists who want to find the truth behind studies. This center could analyze technology studies, government reports and medical findings and help point out discrepancies and outright mistakes. Hopefully, this would lead to the end of junk science studies.

In fact, 100 percent of people who sit at my desk think this is a good idea.

What can be done to stop biased studies? Let me know at jim_rapoza@ziffdavis.com.

 
 
 
 
Jim Rapoza, Chief Technology Analyst, eWEEK.For nearly fifteen years, Jim Rapoza has evaluated products and technologies in almost every technology category for eWEEK. Mr Rapoza's current technology focus is on all categories of emerging information technology though he continues to focus on core technology areas that include: content management systems, portal applications, Web publishing tools and security. Mr. Rapoza has coordinated several evaluations at enterprise organizations, including USA Today and The Prudential, to measure the capability of products and services under real-world conditions and against real-world criteria. Jim Rapoza's award-winning weekly column, Tech Directions, delves into all areas of technologies and the challenges of managing and deploying technology today.
 
 
 
 
 
 
 

Submit a Comment

Loading Comments...
 
Manage your Newsletters: Login   Register My Newsletters























 
 
 
 
 
 
 
 
 
 
 
Rocket Fuel