Determine and Query Typical Internal Questions
Step #3: Determine and query typical internal questions Write down some typical internal questions such as "What is the company holiday schedule?" or "Which business unit is responsible for product X?" Then see if you can identify a document that best answers these questions. It might help to pick a domain you are knowledgeable about-at least initially.Step #4: Compile and analyze the results Now compile the results and take a look. Are there any trends in the ratings? The odds are that you will observe one of the following: 1. The results are just incoherent. This typically indicates that relevancy needs fine-tuning or is otherwise not configured correctly. For example, you may see articles that have nothing to do with your query terms, or you might see new articles but not relevant ones. Or, you may see relevant ones but not the latest information. 2. There are too many results or one source of data dominates the results. Both of these indicate that the search solution needs to expose facets or "dimensions" that users can use to slice into the result set. It may also need to add entity or concept extraction capabilities. 3. There is misspelling or non-recognition of company terminology, jargon and acronyms This issue indicates that query and/or content processing-especially linguistic processing such as tokenization and spelling-are not configured correctly, or that some work on acronym and synonym handling is required. One of the most likely outcomes-regardless of the overall health of your internal search solution-will be that there is simply no appropriate content to find for many queries. One-third of the survey respondents noted this, claiming that less than half of the information they needed is searchable. Most organizations limit internal search to text files such as office documents, spreadsheets, PDFs, brochures and, of course, Web pages (both Internet and intranet). Unfortunately, this ignores three of the most important corporate silos: e-mail and/or messaging, custom or departmental applications (such as databases), and complex enterprise applications built on top of databases-such as business intelligence, enterprise resource planning (ERP) and customer relationship management (CRM) systems. Not surprisingly, these are the most challenging silos to work with, let alone link and correlate with the other fuzzier, unstructured data. Legacy enterprise search engines may simply not be up to it. One low-cost, quick and easy fix is to federate user queries against these sources and present the results side-by-side. Even if this doesn't represent a perfect solution, it will at least show users that improvement is possible.
Finally, see if you can find the document by querying. It may take you several tries. Keep track of how many times you have to revise the query to get the document, and then score your results similar to the scale mentioned earlier (1=impossible to find, 5=easy to find).