How to Measure Findability in Enterprise Search Solutions

 
 
By Sid Probstein  |  Posted 2008-12-09 Email Print this article Print
 
 
 
 
 
 
 

Enterprises need to measure the findability of information in their internal search solutions. When it's easier for employees to find needed information on the corporate intranet, increased employee productivity results. Knowledge Center contributor Sid Probstein explains how to measure findability, and offers five steps to building a ROI case for your company's internal search solution.

The results of a recent survey on the findability of information within the enterprise are not encouraging. Roughly half of the responding knowledge workers stated that finding important information was difficult and time-consuming, and that the internal search capabilities provided by their company were "worse" to "much worse" than the equivalent functionality offered to consumers.

Neither of these findings is truly surprising. Corporate Internet sites tend to be directly involved in important and easily measured activities: selling products to new customers (generating revenues) or servicing existing customers (reducing costs). Consequently, Internet search is usually well-funded, staffed and tends to be more successful. Internal search, in contrast, is concerned with productivity-a fuzzier concept that is much harder to measure. Significant investment (and thus success) is therefore harder to achieve.

What is surprising is that roughly half of the survey respondents stated that their company had no formal goal for internal findability. In my view, this is a direct cause for the overall poor results. Companies that don't measure search won't be able to invest appropriately, let alone tune and improve a complicated system with which they likely don't have deep internal experience.

If your organization doesn't measure search, or more precisely, findability, you should start right away. Here are five steps to building a return on investment case for your company's internal search or information access solution. Keep in mind that no massive investment is required upfront. In fact, one person working a few hours a week can make a difference.

Step #1: Talk to your system administration team

For starters, find out if your organization is saving query logs-hopefully they are. If not, this is the first challenge to overcome. Talk to your system administration team and see if they can help. You don't need to save all logs for all time; just try to get your hands on a day or two of data. That's quite enough to get started.

Step #2: Identify and run a handful of queries

Assuming you have some data to look at, identify a handful (less than 50) of interesting queries. Ideally, you want them to fall into a few different categories: one-word, two-word, multiple words, questions, and a few different business domains.

Run the queries yourself and see what comes up. Look at the first few results and score them on a simple scale (such as 1=incoherent, 5=perfect). If you rate a result poorly, spend a few minutes trying to find out what the better answer might be. Then see if you can infer what's wrong. 

For example, does the document that answers your question (but doesn't appear in the results list) contain the terms you put in your query? If it does, you have a relevancy problem; if not, you have some sort of linguistic problem.



 
 
 
 
Sid Probstein is currently Chief Technology Officer at Attivio, responsible for technology strategy and innovation. Sid brings to Attivio more than 15 years experience leading successful engineering organizations and building complex, high-performance systems. Previously, he was CTO at GCi, where he headed up development of the company's next-generation commerce platform. He also served as vice president of technology at Fast Search & Transfer, where he developed next-generation search, text mining and multimedia capabilities. Sid also served as vice president of engineering at Northern Light Technology, where he produced the very first enterprise version of the award-winning search engine, director of software engineering at Freemark Communications, where he helped implement the first "free" e-mail service, and principal architect/system manager at John Hancock Financial Services, where the integrated sales illustration and client management system he designed was featured as a Microsoft Solution-in-Action case study. He can be reached at sid@attivio.com.
 
 
 
 
 
 
 

Submit a Comment

Loading Comments...
 
Manage your Newsletters: Login   Register My Newsletters























 
 
 
 
 
 
 
 
 
 
 
Thanks for your registration, follow us on our social networks to keep up-to-date
Rocket Fuel