In the first broadcast of the much-hyped Jeopardy! battle of man vs. machine, Brad Rutter tied with Watson for first with Ken Jennings not so far behind.
While it initially looked like a rout, the humans clawed
their way back during the first round of Jeopardy! and forced IBM's
supercomputer to tie for the lead.
The first segment of the long-anticipated two-game Jeorpardy
tournament between former Jeopardy! champions, Ken Jennings and Brad Rutter, and
Watson aired on Feb. 14. Watson and Rutter both had $5,000 on the board and
Jennings trailed behind with $2,000.
"I had a good feeling at the end of the first show," said IBM's
David Ferrucci, the lead researcher and principal investigator on the Watson
project in a post-game analysis posted on the company's A
Smarter Planet blog. "I thought: Everybody will realize the computer is
competitive," he said.
The first day's final scores revealed only part of the
story. Rutter made the first selection, and beat Watson to the buzzer to answer
the question. After that, Watson dominated the round up to the commercial
break, buzzing in with 11 correct answers out of 15 questions, including the
Daily Double (it wagered $1,000 for Literary Character APB). The second half of
the round started with $5,200 for Watson, $1,000 for Rutter, and $200 for
Jennings on the scoreboard, but Jennings and Rutter beat Watson to the buzzer
several times during the course of the round. Most of the wrong answers during
this round were also Watson's.
Viewers who expected the computer to get every question
right were treated to several of Watson's wrong answers in the second half. Unlike
a human player, Watson can't adjust its answers to what other players say and
answers whatever it picked as its top answer during initial processing. After Jennings
incorrectly answered "20s" was the decade in which Oreo cookies were
introduced, Watson answered with "1920s."
"Watson is very bright, very fast, but he has some
weird little moments," Trebek said.
Another Watson mistake illustrated the challenges of natural
language processing. The category was Olympic Oddities, and the answer was a
gymnast with an unusual physical feature.
Watson answered "leg," but was ruled incorrect, because the proper
answer (edited: no one got right) was that the gymnast's leg was missing.
In this case, Watson very likely didn't understand what an "oddity"
is, according to Ferrucci's post-game analysis on Smarter Planet.
"The computer wouldn't know that a missing leg is odder than
anything else," said Ferrucci.
Watson could come to understand what an odditiy is over
time, "by reading more material and playing more games," according to A Smarter
Planet.
Jeopardy requires an enormously broad domain of knowledge
with "infinite ways" to present the question, Ferrucci has said in the past.
Even if Watson knew a topic well, it still needed to understand what the clue
was asking for, he said. The categories of the first round were: Literary
Characters APB, Beatles People, Olympic Oddities, Name the Decade, Final
Frontiers, and Alternate Meanings. The literary characters category was heavy
on puns and wordplay, and while Watson was supremely comfortable in the Beatles
category, it got every single question in the decades category wrong.
Watson has an "Achilles heel," because its betting is
restricted to its confidence level, according to "Final Jeopardy," a book
about Watson by technology journalist Stephen Baker. Despite previous
reports, Watson actually lost the practice
match in January to Jennings in Final Jeopardy, because Jennings had bet
aggressively on the Daily Doubles and Final Jeopardy, Baker said in an excerpt to final chapter of
his book. The remainder of the chapter will be posted after the final game,
and the hardcover of the book will be available on Thursday.
Email
Print
