Really Bayesian

If people are going to underestimate the importance of new data, it's crucial to give them tools that help them use Bayes' insights.

The cover of the Jan. 15 issue of the prestigious science journal Nature is striking. Viewed from above, a tennis player swings her racket toward a contour map of concentric ovals, representing her estimate of the location of the ball, as it streaks toward her. The caption archly inquires, "Anyone for bayesian integration?"

If I were trolling for a high-IQ dinner partner, Id definitely try reading that magazine—with the cover plainly visible—at the courtside cafe of the local tennis club. Im concerned, though, that the adjective "Bayesian" has been getting an awful lot of sloppy play these days in connection with e-mail filtering. Like "artificial intelligence" and "expert system" before it, I fear that a useful term of art is in danger of being muddled by hype to the point that its meaning—and its value to critical decision-support applications, as well as to mundane junk-mail cleanup—are lost.

First, the word is properly rendered with a capital "B" out of respect for the work of 18th-century mathematician the Rev. Thomas Bayes. I suppose its a backhanded compliment to the gentleman that his name has become a part of the language: Bayes even has a fan club, the (International Society for Bayesian Analysis), celebrating its 12th birthday this year.

And Bayesian analysis yields more than 50,000 hits on Google, so Bayes is probably not spinning in his grave at any lack of attention to his name. Hed take exception, though, Im sure, to its vague application, as if it were merely a synonym for "statistical" or "probability-based." The essence of Bayesian analysis, hed be certain to say to anyone who would listen, is forming an updated estimate based on a combination of prior belief and objective observation, instead of starting from scratch without regard for prior experience. Mathematically, Bayes theorem gives us an objective way of taking what we expect before we get a new piece of information and changing that expectation based on what weve just learned.

Lets translate this to specifics of product evaluation. If a company wants to call an e-mail filter Bayesian, it should be able to answer two questions. First, how good are the initial assessments of the chance that something is unwanted e-mail, before obtaining feedback from the user? Id want to know how a particular filtering technology expresses those likelihoods, based on what criteria, and how it updates those base-line estimates as mass e-mailers adopt new techniques.

Second, Id want to know how well the tool incorporates an individual users feedback. How much of a nuisance is it for the user to provide that input, and how well does the filtering tool use it? Which is not the same, I hasten to point out, as asking how faithfully the tool does what I tell it to do because people themselves arent as consistently analytical—that is, as Bayesian—as they might like to think.