In our last column, we discussed the fact that most Web sites have various stakeholders, each of whom will want to measure different things to determine if the site is successful. We described a project, HRNet, which is an intranet site providing information on human resources at a large electronics corporation.
We were given the task of defining metrics to help management determine if the site was worth the expense of maintenance. We collected different measurement data of interest to the business stakeholders, the IT group, the employee group and the HR group.
One of the biggest problems with a measurement project of this nature is that it can generate a huge amount of data. It can take a considerable amount of time to go through all the minute details and synthesize exactly what we need to make our conclusions. This is particularly true of the hit rates and other site statistics.
To deal with the data overload, we find its best to develop a set of focus points, interesting data elements or correlations that are particularly compelling. After the data collection phase in the HRNet project, for example, we identified a small number of specific measures that could be defined from the actual data or from comparisons of the various data elements we had collected.
For example, one focus point related the employee retention rates over a two-year period with monthly hit rate averages on HRNet.
Once we had 10 to 12 such focus points, we met again with the primary stakeholders. As a group, we picked five of the focus points that we felt would make the strongest case for the success or the failure of the Web site. Of course, at this point in the process, its unknown what the data will show. But the five focus points could make the case that the site is succeeding or failing.
The whole point is to avoid the need to analyze all the data that has been collected. If the decision on what measurement data is most important is made by the group that defined the original set of metrics, the time-consuming task of looking at everything can be avoided.