Close
  • Latest News
  • Artificial Intelligence
  • Video
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
Read Down
Sign in
Close
Welcome!Log into your account
Forgot your password?
Read Down
Password recovery
Recover your password
Close
Search
Logo
Subscribe
Logo
  • Latest News
  • Artificial Intelligence
  • Video
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
More
    Subscribe
    Home IT Management
    • IT Management

    Can We Trust the Analysts?

    Written by

    David Coursey
    Published April 7, 2005
    Share
    Facebook
    Twitter
    Linkedin

      eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

      One of the most important decisions each of us makes—and we do this hundreds, maybe thousands of times each day—is who or what to trust.

      Who or what can I trust? Thats the question I ask myself when someone presents me with a research study or when I seek the opinion of an industry analyst. Its the same question you should be asking about the information you consume.

      Is the information were presented “honest” or is it tainted by a particular vendors interest? More bluntly: Are we being lied to so people can take our money?

      In the grand scheme of things, probably. But I dont think the computer industry is worse than any other, and it may be a good deal better. For example, can the analysts we so often see quoted be trusted? And should we ever trust the results of a study that a vendor paid to have done?

      The answer to both these questions is “Yes.” But its a qualified yes, and Id like to explain both the value and limitations of using analysts and their reports as sources. If you are a regular consumer of the computer trades this should be important information. Much of what you read either contains quotes from an industry analyst or in some ways reflects their thinking.

      The good news: The established players are pretty trustworthy. The companies—and news organizations—whose names you recognize have been around a long time and have earned the industrys trust. They work hard to maintain it. That doesnt mean they are always right, but it does mean they arent corrupt.

      Seeking to understand this issue, I did what I often do: I called a few analyst friends, people who have earned my respect over a period of years, and asked them how my readers could be better consumers of the information analysts generate.

      Heres some of what I heard:

      Dont expect a crystal ball—Analysts are professional watchers of a particular technology or market segment. Ideally, they know their areas better than anyone else, including executives of the companies they cover. Many sign nondisclosure agreements, which sometimes allow them to “predict” trends with 100 percent accuracy. But dont confuse that with omniscience. Also, remember the tendency for analysts to fall into the trap of seeing their world as the center of the universe, and making predictions accordingly.

      If you see an analyst endorsing something, watch out—Reputable analysts will rarely recommend one product or vendor over another without solid research to back it up. The big-name analysts work for companies that get money from all the major vendors, and there is no value to them in angering all but one of their clients.

      Just because a vendor pays doesnt mean the vendor calls the shots—One of my friends is working on a survey that is being sponsored by 10 vendors. One vendor asked if it was possible to drop a particular question if the vendor didnt like the results. My friend offered to let the vendor have its money back and not be included.

      But thats at a major firm. My analyst friends stopped short of saying small firms cant be trusted, but do be wary. The research business is highly competitive and vendors are not beyond taking advantage of a startup or a company that is in trouble as the research business again consolidates. There is also a tendency of some small shops to be more cheerleader than steely-eyed research company. Especially when the vendor pays the bills.

      Next Page: Evangelism, perspective, lies and ideals.

      Evangelism, Perspective, Lies and


      Ideals”>

      This should not, however, be confused with good old-fashioned evangelism, in which people promote a cause because they deeply believe in it, have a financial or professional stake in it, or both. These are the people who have, to use a common if unfortunate phrase, “drunk the Kool-aid,” and thus become suspect.

      It is to be expected, for example, that people who have tied their wagon to the Linux star will be fervent supporters and even propagandists. If you agree with their viewpoints, these people can be excellent sources of information. But be careful of accepting their global view of how things fit together, as their perspective can be both skewed and narrow. In short: There are lots of people who need to get out more often and see how the world really works.

      Read the fine print—If you see survey results, try to understand who was questioned and what they were asked before accepting the conclusions offered. Polling can be skewed by a number of factors, resulting in some of the wildly different results we see. All polling is fraught with risk and few of us are really expert enough to separate good polls from bad ones. Often, the polling organizations dont seem to know.

      Likewise, if you are looking at product test results or performance claims, look at how the numbers were generated. The major publications and organizations that routinely do testing take pride in their methodologies. Some of their test results end up in product advertisements.

      What about vendor-sponsored testing, as in Microsoft comparing its products to Linux in its advertisements? Assuming the information is true, would anyone believe it if Microsoft did the testing in-house? And how is a testing lab supposed to fund its work if vendors dont pay? This is especially true of enterprise products, where testing can be complex and expensive.

      /zimages/3/28571.gifClick here to read about a recent study commissioned by Microsoft which concluded that Windows 2003 is more reliable than Red Hat Enterprise Linux AS 3.0.

      My own belief is that vendors publicize the most favorable numbers they can find short of making them up. And may disregard less favorable results along with way. Reading the fine print may show telltale signs of testing being, well, “rigged” isnt quite the word, but presented so as to produce results more favorable to the vendor.

      I filter these ads this way: They arent lies, but may compare a companys best case to a competitors worst case.

      In an ideal world, all research would be paid for by its ultimate consumer. This would align the researchers interests with those of the customer. Likewise, the ideal news organization would receive its funding only from readers, much in the way Consumer Reports eschews advertising.

      The problem with this is twofold: First, readers and customers dont like to pay for as much information as theyd happily consume. And, secondly, vendors need to get their messages to potential customers, and advertising is a legitimate way to accomplish this.

      I think probably 90 percent of the information I read from mainstream sources is, if not totally reliable, at least not intentionally slanted. The further the source is from mainstream, the more skeptical I become.

      And thats pretty much the conclusion of my analyst friends, who all said their own companies could be trusted but to treat everyone else with at least a wee bit of suspicion—and sometimes a whole lot more.

      /zimages/3/28571.gifCheck out eWEEK.coms for the latest news, reviews and analysis on IT management from CIO Insight.

      David Coursey
      David Coursey
      One of technology's most recognized bylines, David Coursey is Special Correspondent for eWeek.com, where he writes a daily Blog (blog.ziffdavis.com/coursey) and twice-weekly column. He is also Editor/Publisher of the Technology Insights newsletter and President of DCC, Inc., a professional services and consulting firm.Former Executive Editor of ZDNet AnchorDesk, Coursey has also been Executive Producer of a number of industry conferences, including DEMO, Showcase, and Digital Living Room. Coursey's columns have been quoted by both Bill Gates and Steve Jobs and he has appeared on ABC News Nightline, CNN, CBS News, and other broadcasts as an expert on computing and the Internet. He has also written for InfoWorld, USA Today, PC World, Computerworld, and a number of other publications. His Web site is www.coursey.com.

      Get the Free Newsletter!

      Subscribe to Daily Tech Insider for top news, trends & analysis

      Get the Free Newsletter!

      Subscribe to Daily Tech Insider for top news, trends & analysis

      MOST POPULAR ARTICLES

      Artificial Intelligence

      9 Best AI 3D Generators You Need...

      Sam Rinko - June 25, 2024 0
      AI 3D Generators are powerful tools for many different industries. Discover the best AI 3D Generators, and learn which is best for your specific use case.
      Read more
      Cloud

      RingCentral Expands Its Collaboration Platform

      Zeus Kerravala - November 22, 2023 0
      RingCentral adds AI-enabled contact center and hybrid event products to its suite of collaboration services.
      Read more
      Artificial Intelligence

      8 Best AI Data Analytics Software &...

      Aminu Abdullahi - January 18, 2024 0
      Learn the top AI data analytics software to use. Compare AI data analytics solutions & features to make the best choice for your business.
      Read more
      Latest News

      Zeus Kerravala on Networking: Multicloud, 5G, and...

      James Maguire - December 16, 2022 0
      I spoke with Zeus Kerravala, industry analyst at ZK Research, about the rapid changes in enterprise networking, as tech advances and digital transformation prompt...
      Read more
      Video

      Datadog President Amit Agarwal on Trends in...

      James Maguire - November 11, 2022 0
      I spoke with Amit Agarwal, President of Datadog, about infrastructure observability, from current trends to key challenges to the future of this rapidly growing...
      Read more
      Logo

      eWeek has the latest technology news and analysis, buying guides, and product reviews for IT professionals and technology buyers. The site’s focus is on innovative solutions and covering in-depth technical content. eWeek stays on the cutting edge of technology news and IT trends through interviews and expert analysis. Gain insight from top innovators and thought leaders in the fields of IT, business, enterprise software, startups, and more.

      Facebook
      Linkedin
      RSS
      Twitter
      Youtube

      Advertisers

      Advertise with TechnologyAdvice on eWeek and our other IT-focused platforms.

      Advertise with Us

      Menu

      • About eWeek
      • Subscribe to our Newsletter
      • Latest News

      Our Brands

      • Privacy Policy
      • Terms
      • About
      • Contact
      • Advertise
      • Sitemap
      • California – Do Not Sell My Information

      Property of TechnologyAdvice.
      © 2024 TechnologyAdvice. All Rights Reserved

      Advertiser Disclosure: Some of the products that appear on this site are from companies from which TechnologyAdvice receives compensation. This compensation may impact how and where products appear on this site including, for example, the order in which they appear. TechnologyAdvice does not include all companies or all types of products available in the marketplace.