Can We Trust the Analysts?

Opinion: How should readers interpret the information analysts provide in industry articles? Here's some advice straight from the source.

One of the most important decisions each of us makes—and we do this hundreds, maybe thousands of times each day—is who or what to trust.

Who or what can I trust? Thats the question I ask myself when someone presents me with a research study or when I seek the opinion of an industry analyst. Its the same question you should be asking about the information you consume.

Is the information were presented "honest" or is it tainted by a particular vendors interest? More bluntly: Are we being lied to so people can take our money?

In the grand scheme of things, probably. But I dont think the computer industry is worse than any other, and it may be a good deal better. For example, can the analysts we so often see quoted be trusted? And should we ever trust the results of a study that a vendor paid to have done?

The answer to both these questions is "Yes." But its a qualified yes, and Id like to explain both the value and limitations of using analysts and their reports as sources. If you are a regular consumer of the computer trades this should be important information. Much of what you read either contains quotes from an industry analyst or in some ways reflects their thinking.

The good news: The established players are pretty trustworthy. The companies—and news organizations—whose names you recognize have been around a long time and have earned the industrys trust. They work hard to maintain it. That doesnt mean they are always right, but it does mean they arent corrupt.

Seeking to understand this issue, I did what I often do: I called a few analyst friends, people who have earned my respect over a period of years, and asked them how my readers could be better consumers of the information analysts generate.

Heres some of what I heard:

Dont expect a crystal ball—Analysts are professional watchers of a particular technology or market segment. Ideally, they know their areas better than anyone else, including executives of the companies they cover. Many sign nondisclosure agreements, which sometimes allow them to "predict" trends with 100 percent accuracy. But dont confuse that with omniscience. Also, remember the tendency for analysts to fall into the trap of seeing their world as the center of the universe, and making predictions accordingly.

If you see an analyst endorsing something, watch out—Reputable analysts will rarely recommend one product or vendor over another without solid research to back it up. The big-name analysts work for companies that get money from all the major vendors, and there is no value to them in angering all but one of their clients.

Just because a vendor pays doesnt mean the vendor calls the shots—One of my friends is working on a survey that is being sponsored by 10 vendors. One vendor asked if it was possible to drop a particular question if the vendor didnt like the results. My friend offered to let the vendor have its money back and not be included.

But thats at a major firm. My analyst friends stopped short of saying small firms cant be trusted, but do be wary. The research business is highly competitive and vendors are not beyond taking advantage of a startup or a company that is in trouble as the research business again consolidates. There is also a tendency of some small shops to be more cheerleader than steely-eyed research company. Especially when the vendor pays the bills.

Next Page: Evangelism, perspective, lies and ideals.