A recent study found that I am the best-looking man in the Western Hemisphere. Sure, I commissioned the study, and the only participants were my wife, mom and two dogs, but all most people will notice is the headline.
Of course, this is a joke, not unlike the recent study commissioned by Microsoft that tried to show that owning Windows 2000 is cheaper than owning Linux (see related story). Sure, there are cost benefits to running Windows-based servers, and Im willing to bet that in a legitimate study, they could do well in many areas versus Linux.
But this study is full of questionable assumptions. For example, its based on a five-year technical lifetime, rather than the more common three years—spreading upfront costs over a longer period.
Unfortunately, most people (including much of the press) dont look at such details, meaning there is little downside in commissioning studies that are weighted ahead of time to provide the results the sponsor wants.
This is true not only in tech studies but also in most studies we are inundated with every day.
While these kinds of studies are valuable to companies, they can cause a lot of harm, both in peoples expectations and in the publics confidence in science and research.
One of my dreams has been to start an unbiased and independent center that would serve as a resource for people and journalists who want to find the truth behind studies. This center could analyze technology studies, government reports and medical findings and help point out discrepancies and outright mistakes. Hopefully, this would lead to the end of junk science studies.
In fact, 100 percent of people who sit at my desk think this is a good idea.
What can be done to stop biased studies? Let me know at [email protected]