IT projects have historically been known to have high failure rates. According to a recent report, 31 percent of all IT projects were complete failures in 1995. And only an alarming 16 percent of all IT projects were successes. By 2006, these figures changed to more favorable totals. For example, 35 percent of all IT projects were successful and only 19 percent of all projects were complete failures. Another 46 percent were considered to be “challenged,” as these projects did create a useful product but did not meet all of the project success criteria (such as scope, budget, schedule and quality).
Why the increase in IT project success rates? As a whole, project management did become much more sophisticated between 1995 and 2006. For instance, project management tools became much easier to use. In addition, the Internet made it easier to communicate successfully and much more rapidly than in the early 1990s. Users were also becoming more sophisticated when it came to describing what they, in fact, wanted in a project.
However, despite this increase, the majority of implemented projects still failed or did not become complete successes. This type of situation resulted in organizations losing a great deal of money. In 2001, a study by the Department of Commerce’s National Institute of Standards and Technology (NIST) stated that software “bugs” alone cost United States companies $60 billion dollars.
Why projects fail
Some of the causes of project failure include budgeting too little time or too little money, inadequate planning, constantly-changing goals, lack of software knowledge, insufficient knowledge of advanced technology, and inadequate communications. Further, in these types of situations, the management is usually the last to know about any problems.
Often the only difference between a project’s complete failure and its success is in spotting some specific warning signs. These signs allow us to notice the problem upfront and to recognize the real, measurable risk information we can get from historical data and from the time perspective. There are four important warning signs that allow us to mitigate the failure risks.
The first warning sign is employees working overtime. If a project is running smoothly, there usually is little to no need for overtime. The second warning sign is people being pulled off the project to work on another project. As time spent on other projects is cumulative, this factor could indicate a potential problem. The third warning sign is milestones are not being met. If official milestones are not met, this situation indicates that a project may be in trouble. And lastly, the fourth warning sign is project scope changes. While this factor may not necessarily be “bad,” it is a sign that the project may be in trouble.
Besides these four warning signs, there are also some more intangible signs that a project may be in trouble. These signs include a general lack of interest in the project, poor communication among workers, a fear of talking about project problems, and a generalized lack of project velocity. That said, there is a way to measure a project’s success while it is in progress by measuring certain metrics within a particular project. Measuring metrics is even more important today as an organization’s infrastructure and business applications continue to evolve or increase or both. Further, in today’s economy, it is more vital than ever to minimize costs and maximize project quality.
What Exactly Are Metrics?
What exactly are metrics?
Metrics measure processes, activities, resources, processes and deliverables within a quality control plan. More specifically, a metric is a unit that is used to collect data in order to report on the state of a particular IT service.
This data must be presented in a meaningful way so as to help project managers make the proper decisions or to take corrective steps on projects or both.
In order to measure these metrics effectively, indicators and measurable periods are utilized. Indicators are used to describe metric data that can provide information about a particular project. Conversely, it is important to define the right measurable periods that can cover possible gaps in the control of the measuring indicators-and also allow us to control the situation upfront if a failure occurs within the measurable intervals.
Examples of indicators used on a project include actual versus planned task completions, actual versus planned staffing, the number of trouble reports written and resolved over time, and the number of requirements changed over time.
Indicators are used in conjunction with one another to provide a more complete picture of a project or organizational behavior. For example, a progress indicator is related to requirements and size indicators. All three indicators should be used and interpreted together.
Types of Metrics
Types of metrics
There are three basic types of metrics: quantitative data (which collect the “raw” project data of the meaningful metrics), qualitative indicator (which provides the derived and calculated dependencies and correlations) and prognosed trends (which allows us to build and see the required extrapolations that permit us to make the correct decisions).
Moreover, metrics can be derived from observable information or “raw” data. For instance, the amount of source lines of code (SLOC) in a particular program or the number of documentation pages, staff hours, tests or requirements are examples of raw data. Conversely, an example of a “derived” metric can include the number of SLOC produced per day per worker, the defects per one thousand SLOC or a cost performance index. Additionally, often there are also six criteria that are used to illustrate the definition of metrics. These items are as follows:
1. Time: How the project is going against schedule?
2. Cost: How the project is going against budget?
3. Resources: How much time is being spent on this project?
4. Scope: Is the project’s scope kept in line with expectations?
5. Quality: Are quality problems being reviewed and fixed?
6. Actions: Are there any outstanding action items on the project?
Metrics can explain what exactly is going to be measured, how the item will be assessed, and the units of measurement that will be used (such as percentages, number, ratio or index).
Some individuals also believe that a metric should include how that particular metric should be used and by which person in the organization, as this act encourages better decision making and better end products.
Of course, metrics must have another figure with which to compare figures; this number can be an industry benchmark, regulatory guidelines or other similar figure. It is important to remember, though, that sometimes metrics are only applicable to a particular company and thus, there could be no industry benchmarks altogether in this situation.
Reasons to Use Metrics
Reasons to use metrics
While metrics can take time to create and to report on, there are numerous reasons why an organization should take advantage of them. One of the most well-known reasons why we measure project metrics is to determine the current state of the “health” of a particular project.
Metrics can also give important objective information about certain details of a project, and along similar lines, the metrics can be a source of imperative information for data control.
Specifically, metrics can also help with risk mitigation, crisis resolution, project control, team performance management, client expectations management, and adjustment of prognoses and development plans. Metrics can also help with understanding customer satisfaction, team satisfaction measurement, resource management efficiency, identifying and understanding systems development life cycle (SDLC) bottlenecks, and managing software quality.
Additionally, IT metrics can control and measure IT processes such as production operations, software development, security and application maintenance. Some of the other specific management areas that can benefit from metrics include gaining the ability to quantify staff activities and outsourcing. Specific IT operational areas that can benefit from metrics include IT security, server management, application development processes and data center operations.
Why Metrics Are Useful
Why metrics are useful
Metrics are extremely useful, as these measurements can be used in every stage of the management process. During the planning phase of a project, metrics can be used to plan the allocation of resources, budgets, schedules and cost estimations. Further, metrics can be used to discover information about the status and quality of projects. They can also identify specific areas that require improvement. However, in order to be useful, metrics must have the following three characteristics:
1. Functional: The metrics must measure exactly what numbers they are supposed to measure
2. Cost-effective: The metrics must deliver more value than the cost of measuring and analyzing the metric information
3. Consistent: The metrics must be reliable throughout various company operations
Of course, the metrics that your organization chooses to use would depend on the particular project that you are working on and what exactly you are trying to do. Also, before you choose the project metrics, you must also know your project objectives.
After you are certain of your objective, you will know what particular metrics you need to use. After all, the metrics that are required to measure the progress of a project are different than the metrics that are required to solve a crisis. At this time, it is also important to figure out exactly what question you need answered.
Above all though, your organization should not choose a particular metric based on the fact that a metric is used often. Why? One size does not fit all, and the fact that a particular metric is utilized does not mean that that metric will be a helpful measure for your project. Thus, by asking the proper questions at this stage, you will choose the metrics that will help you find the answers that you seek. After this step, you must be able to collect, measure and interpret this data in an effective manner.
Overall, if project metrics are developed and implemented in the proper manner, the use of project metrics can only help an organization’s bottom line, both now and in years to come.
Serhiy Kharytonov is the Executive VP of Consulting Services at SoftServe. Serhiy has been with SoftServe since 2000, previously in the positions of VP of Technology and VP of R&D Services. Serhiy holds a Master’s degree in Computer Science from the National Polytechnic University, Lviv, Ukraine, and is currently finishing his PhD. He can be reached at skhar@softserveinc.com.
