Close
  • Latest News
  • Artificial Intelligence
  • Video
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
Read Down
Sign in
Close
Welcome!Log into your account
Forgot your password?
Read Down
Password recovery
Recover your password
Close
Search
Logo
Logo
  • Latest News
  • Artificial Intelligence
  • Video
  • Big Data and Analytics
  • Cloud
  • Networking
  • Cybersecurity
  • Applications
  • IT Management
  • Storage
  • Sponsored
  • Mobile
  • Small Business
  • Development
  • Database
  • Servers
  • Android
  • Apple
  • Innovation
  • Blogs
  • PC Hardware
  • Reviews
  • Search Engines
  • Virtualization
More
    Home IT Management
    • IT Management

    The Quality of Our Ignorance Is Rising

    Written by

    Peter Coffee
    Published April 14, 2003
    Share
    Facebook
    Twitter
    Linkedin

      eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

      In the past week, various people have been telling me about their initiatives in both enterprise data mining and ultrawideband (UWB) wireless networks. Besides growing momentum, these IT domains share a common demand for a statistical approach to system design: data mining with its emphasis on inference and significance, UWB and other shared-spectrum techniques with their sensitivity to worst-case behavior of random events.

      Statistics are themselves perhaps the worst-case example of a business or technical discipline thats both too rarely used and too often misused. As we build systems that are larger, that work in less controlled environments, and whose operation involves larger numbers of independent decision makers, its vital for IT professionals to get smarter about our use of statistical methods–and about the inferences we make from the numbers we collect.

      Telecommunication carriers have developed a huge body of statistical expertise on systems like the wired telephone network, although this may prove misleading if we try to apply that experience to the new world of always-on unwired access. The transactions of Web services will be, for the most part, less sensitive to split-second timing than voice communications (which have to satisfy our wetwares idea of what sounds natural). On the other hand, our models of when people want to connect, and for how long, have already failed to keep up with even the cumbersome mode of dial-up Internet access. Well have a lot to learn about the new “normal” behavior when people can access the network at will–or when devices and software agents can access each other directly at any time.

      Our need for broader statistical expertise is compounded by the tilt toward business intelligence and decision support–rather than mere bookkeeping–as the mission of new enterprise IT projects. Just as we would not willingly build a system that tried to do accounting with 8-bit integers, we owe it to business unit managers to warn them when theyre asking for a system that will mislead them with impressive but ill-founded analyses.

      Page Two

      Its a dangerously mixed blessing, moreover, that statistical tools are constantly getting easier to use. When anyone can find the best straight-line fit to a collection of data points, its inevitable that more relationships will be “found” that actually have no statistical significance–and even more whose statistical significance says nothing about a cause-and-effect connection.

      The other major problem in statistics is everyday misuse of the expression, “the law of averages.” Im constantly surprised to hear people use this phrase to suggest that because things have gone one way for longer than we would expect, theyre soon bound to swing in the other direction to even things out. This is bad enough when people think that a coin that comes up “heads” four times in a row is somehow more likely to come up tails the next time; its worse when they dont even consider the possibility that this coin might have heads on both sides, and that the information theyve gathered so far about its behavior should be used to adjust their model of the process.

      Ive previously suggested that biology, rather than physics or mathematics, may be the pure science that has the most to offer to our thinking about our future IT systems. If theres one thing that biology researchers can do, its define a “null hypothesis”–a statement that what they hope to find is not, in fact, the case–and then set themselves to the task of proving that null hypothesis wrong. Id rather see a software company, for example, prove that a system is not insecure, rather than demand that skeptics show them where its vulnerabilities lie.

      Getting smart about risks, uncertainties, bursts of demand and burdens of inference from data is a professional challenge that deserves our determined response.

      Tell me how you measure what you dont know.

      Peter Coffee
      Peter Coffee
      Peter Coffee is Director of Platform Research at salesforce.com, where he serves as a liaison with the developer community to define the opportunity and clarify developers' technical requirements on the company's evolving Apex Platform. Peter previously spent 18 years with eWEEK (formerly PC Week), the national news magazine of enterprise technology practice, where he reviewed software development tools and methods and wrote regular columns on emerging technologies and professional community issues.Before he began writing full-time in 1989, Peter spent eleven years in technical and management positions at Exxon and The Aerospace Corporation, including management of the latter company's first desktop computing planning team and applied research in applications of artificial intelligence techniques. He holds an engineering degree from MIT and an MBA from Pepperdine University, he has held teaching appointments in computer science, business analytics and information systems management at Pepperdine, UCLA, and Chapman College.

      Get the Free Newsletter!

      Subscribe to Daily Tech Insider for top news, trends & analysis

      Get the Free Newsletter!

      Subscribe to Daily Tech Insider for top news, trends & analysis

      MOST POPULAR ARTICLES

      Artificial Intelligence

      9 Best AI 3D Generators You Need...

      Sam Rinko - June 25, 2024 0
      AI 3D Generators are powerful tools for many different industries. Discover the best AI 3D Generators, and learn which is best for your specific use case.
      Read more
      Cloud

      RingCentral Expands Its Collaboration Platform

      Zeus Kerravala - November 22, 2023 0
      RingCentral adds AI-enabled contact center and hybrid event products to its suite of collaboration services.
      Read more
      Artificial Intelligence

      8 Best AI Data Analytics Software &...

      Aminu Abdullahi - January 18, 2024 0
      Learn the top AI data analytics software to use. Compare AI data analytics solutions & features to make the best choice for your business.
      Read more
      Latest News

      Zeus Kerravala on Networking: Multicloud, 5G, and...

      James Maguire - December 16, 2022 0
      I spoke with Zeus Kerravala, industry analyst at ZK Research, about the rapid changes in enterprise networking, as tech advances and digital transformation prompt...
      Read more
      Video

      Datadog President Amit Agarwal on Trends in...

      James Maguire - November 11, 2022 0
      I spoke with Amit Agarwal, President of Datadog, about infrastructure observability, from current trends to key challenges to the future of this rapidly growing...
      Read more
      Logo

      eWeek has the latest technology news and analysis, buying guides, and product reviews for IT professionals and technology buyers. The site’s focus is on innovative solutions and covering in-depth technical content. eWeek stays on the cutting edge of technology news and IT trends through interviews and expert analysis. Gain insight from top innovators and thought leaders in the fields of IT, business, enterprise software, startups, and more.

      Facebook
      Linkedin
      RSS
      Twitter
      Youtube

      Advertisers

      Advertise with TechnologyAdvice on eWeek and our other IT-focused platforms.

      Advertise with Us

      Menu

      • About eWeek
      • Subscribe to our Newsletter
      • Latest News

      Our Brands

      • Privacy Policy
      • Terms
      • About
      • Contact
      • Advertise
      • Sitemap
      • California – Do Not Sell My Information

      Property of TechnologyAdvice.
      © 2024 TechnologyAdvice. All Rights Reserved

      Advertiser Disclosure: Some of the products that appear on this site are from companies from which TechnologyAdvice receives compensation. This compensation may impact how and where products appear on this site including, for example, the order in which they appear. TechnologyAdvice does not include all companies or all types of products available in the marketplace.

      ×