Determine and Query Typical Internal Questions

By Sid Probstein  |  Posted 2008-12-09 Print this article Print

Step #3: Determine and query typical internal questions

Write down some typical internal questions such as "What is the company holiday schedule?" or "Which business unit is responsible for product X?" Then see if you can identify a document that best answers these questions. It might help to pick a domain you are knowledgeable about-at least initially.

Finally, see if you can find the document by querying. It may take you several tries. Keep track of how many times you have to revise the query to get the document, and then score your results similar to the scale mentioned earlier (1=impossible to find, 5=easy to find).

Step #4: Compile and analyze the results

Now compile the results and take a look. Are there any trends in the ratings? The odds are that you will observe one of the following:

1. The results are just incoherent.

This typically indicates that relevancy needs fine-tuning or is otherwise not configured correctly. For example, you may see articles that have nothing to do with your query terms, or you might see new articles but not relevant ones. Or, you may see relevant ones but not the latest information.

2. There are too many results or one source of data dominates the results.

Both of these indicate that the search solution needs to expose facets or "dimensions" that users can use to slice into the result set. It may also need to add entity or concept extraction capabilities.

3. There is misspelling or non-recognition of company terminology, jargon and acronyms

This issue indicates that query and/or content processing-especially linguistic processing such as tokenization and spelling-are not configured correctly, or that some work on acronym and synonym handling is required.

One of the most likely outcomes-regardless of the overall health of your internal search solution-will be that there is simply no appropriate content to find for many queries. One-third of the survey respondents noted this, claiming that less than half of the information they needed is searchable. Most organizations limit internal search to text files such as office documents, spreadsheets, PDFs, brochures and, of course, Web pages (both Internet and intranet).

Unfortunately, this ignores three of the most important corporate silos: e-mail and/or messaging, custom or departmental applications (such as databases), and complex enterprise applications built on top of databases-such as business intelligence, enterprise resource planning (ERP) and customer relationship management (CRM) systems.

Not surprisingly, these are the most challenging silos to work with, let alone link and correlate with the other fuzzier, unstructured data. Legacy enterprise search engines may simply not be up to it. One low-cost, quick and easy fix is to federate user queries against these sources and present the results side-by-side. Even if this doesn't represent a perfect solution, it will at least show users that improvement is possible.

Sid Probstein is currently Chief Technology Officer at Attivio, responsible for technology strategy and innovation. Sid brings to Attivio more than 15 years experience leading successful engineering organizations and building complex, high-performance systems. Previously, he was CTO at GCi, where he headed up development of the company's next-generation commerce platform. He also served as vice president of technology at Fast Search & Transfer, where he developed next-generation search, text mining and multimedia capabilities. Sid also served as vice president of engineering at Northern Light Technology, where he produced the very first enterprise version of the award-winning search engine, director of software engineering at Freemark Communications, where he helped implement the first "free" e-mail service, and principal architect/system manager at John Hancock Financial Services, where the integrated sales illustration and client management system he designed was featured as a Microsoft Solution-in-Action case study. He can be reached at

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel