Concept Search: A Better Way?

By Larry Seltzer  |  Posted 2008-10-21 Print this article Print


From my conversation with Magistrate Judge John M. Facciola of the United States District Court, D.C. Circuit, and from a lot of the lawyers I've talked to about this, I get the sense that they think that the technology is available, or will soon be, to improve these document searches, and they will expect companies to use it. The big item on that agenda is "concept search."

The notion of concept search is that you don't need to hit the keywords exactly in order to get a hit in the search. This should cut down both on false positives and false negatives. How do they do it? Recommind says you take an initial set of search results and pick out a few especially representative documents. The software analyzes these documents to see how the terms are used in them. It then feeds this analysis back into the document database and finds documents that relate to the concepts behind the use of those words and categorizes them by topic. At least, that's the claim. I've never seen it in action. There is something to theory, and in my conversation with Judge Facciola, who is influential in this field, he was enthusiastic about the concept of concept search.

On the legal hold end, Recommind has a new offering called Insite Legal Hold. Working with the same search software, it allows you to take search results and first examine them manually for false positives. Then it preserves the documents securely. It can also automatically preserve new documents that meet the search criteria.

Why is an electronic repository better than a CD in a filing cabinet in the lawyer's office? First, of course lawyers are all trustworthy, but the digital repository probably includes some way to demonstrate that the documents have not been tampered with. Perhaps it all still comes down to trusting the admin or corporate counsel anyway, but there's still some value in it. Also, the ability to preserve new documents as they are created sounds interesting, although it needs to be constructed carefully so that there is the opportunity for inside counsel to review before the document is locked away.

This new legal document technology is cool and promising, but it's probably wise to use it as a supplement to older, more established methods.

In a few years you can probably expect such applications to grow pretty rapidly as court cases establish the "need" for them. With their assistance, even companies as technologically behind the curve as Google can meet their legal obligations.

Security Center Editor Larry Seltzer has worked in and written about the computer industry since 1983.

For insights on security coverage around the Web, take a look at Security Center Editor Larry Seltzer's blog Cheap Hack.

Larry Seltzer has been writing software for and English about computers ever since—,much to his own amazement—,he graduated from the University of Pennsylvania in 1983.

He was one of the authors of NPL and NPL-R, fourth-generation languages for microcomputers by the now-defunct DeskTop Software Corporation. (Larry is sad to find absolutely no hits on any of these +products on Google.) His work at Desktop Software included programming the UCSD p-System, a virtual machine-based operating system with portable binaries that pre-dated Java by more than 10 years.

For several years, he wrote corporate software for Mathematica Policy Research (they're still in business!) and Chase Econometrics (not so lucky) before being forcibly thrown into the consulting market. He bummed around the Philadelphia consulting and contract-programming scenes for a year or two before taking a job at NSTL (National Software Testing Labs) developing product tests and managing contract testing for the computer industry, governments and publication.

In 1991 Larry moved to Massachusetts to become Technical Director of PC Week Labs (now eWeek Labs). He moved within Ziff Davis to New York in 1994 to run testing at Windows Sources. In 1995, he became Technical Director for Internet product testing at PC Magazine and stayed there till 1998.

Since then, he has been writing for numerous other publications, including Fortune Small Business, Windows 2000 Magazine (now Windows and .NET Magazine), ZDNet and Sam Whitmore's Media Survey.

Submit a Comment

Loading Comments...
Manage your Newsletters: Login   Register My Newsletters

Rocket Fuel