Failure in technology projects is as sure as the setting of the sun. Trying to estimate what it costs businesses on a global scale, however, is a bit more of a guess game than an exact science.
A recent white paper on global tech failure and its monetary costs has been getting a fair amount of attention. Roger Sessions, a CTO, has concocted a paper entitled "The IT Complexity Crisis: Danger and Opportunity" (PDF) where he estimates that global IT failure is costing $6.2 trillion annually, or $500 billion a month.
Say what? I applaud the effort, but cringe at the extreme claims being made.
I'm not a numbers cruncher, but these estimates felt suspiciously exaggerated. Sure, white papers can lend themselves towards exaggeration. Most of us can read right through all the marketing mumbo jumbo. But when you start throwing numbers around with such large assumptions, the swiss-cheese detectives are bound to get on the case.
I called up an expert I trust, someone who has been at the forefront of IT projects and failure for a very long time-- Bruce Webster. He is an expert witness who is called in to testify on major technology lawsuits, as well as someone who worked in IT for 30 years. He's also an excellent IT writer, and someone--full disclosure-- I hired to write freelance articles for Baselinemag.com in the past. He knows this stuff.
After speaking with Webster on Monday on the phone, he wrote his own extensive blog post dissecting the Sessions paper. Please read it.
On Sessions estimates, Webster wrote to me:
"The short answer is that his numbers fall apart pretty quickly. He misreads the very report (US Budget FY 2009 Analytical Perspective) that he's citing, and it goes downhill from there, particularly when he tries to extrapolate his (misread) figures from the US government to all other organizations. He also seems totally unaware of what percentage of any organization's IT budget is spent on maintenance rather than new (and risky) IT projects, and his indirect-to-direct costs ratio is pulled completely out of his hat (sorry, the IRS is not a credible example of anything related to IT)."
So, what if anything is there to learn from all of this? I put a few more questions to Webster on IT failure, on the numbers, and on making some sense of it all.
What do you think of trying to peg a monthly and annual cost number on IT failure?
"I think it's a good thing for an organization to track, but again there are raging debates as to ROI for IT dating back 20 years to Paul Strassmann ("The Business Value of Computers"), not to mention Nicholas Carr ("IT Doesn't Matter")."
What does pegging figures to failure mean for an organization?
"Hopefully, it will provide a corrective measure to IT management and will argue for intervention earlier on trouble projects. On the other hand, there is a need to allow a degree of risk (including the potential for failure) in new IT projects."
What should an IT manager glean from these estimates?
"These [Sessions] estimates are, in my opinion, overblown and unsupportable."