This website is no longer actively maintained
Some material and features may be unavailable

The Daily Need

Fear not, humans: Watson the new Jeopardy champion won’t take over the world — yet

Ken Jennings, left, Brad Rutter and a computer named Watson compete on the game show "Jeopardy!" Photo: AP/Jeopardy Productions, Inc., Carol Kaelson

After “Jeopardy” began taping its three man-vs.-machine matches last month, pitting the IBM artificial intelligence software Watson against two of the game show’s most celebrated champions, host Alex Trebek confessed some concern about the contest to author Stephen Baker.

“Is this going to be fair?” he and the producers of “Jeopardy” asked, repeatedly. It wasn’t just a matter of ensuring an honest fight. The show’s producers wanted the Watson challenge to be compelling television. And if the machine — powered by a cluster of 90 servers with nearly 3,000 processing cores and 16 terabytes of data storage — made mincemeat of its human opponents, the show would be a pretty dull affair.

The IBM team, led by principal investigator David Ferrucci, reassured the show’s producers that the terms of the contest would, if anything, favor the humans.

“What we’re doing is we’re building a machine, and the machine has all kinds of weaknesses. It doesn’t understand language very well. And it doesn’t know anything,” Baker said, recalling the IBM team’s argument. “So we’re putting an ignorant machine that has a language handicap up, and you’re saying it’s not fair because it happens to be fast on the buzzer?”

Now, of course, observers are crying foul all over again, after Watson dispatched his human competitors with apparent ease. In Tuesday’s match, for example, Watson answered all but two questions correctly. The machine made a rather glaring flub by offering “Toronto” as the answer to a Final Jeopardy question whose category was “U.S. Cities,” but otherwise seemed unbeatable.

Watson cruised to victory in the final installment Wednesday, prompting Ken Jennings to write for his Final Jeopardy answer, “I, for one, welcome our new computer overlords.”

It’s easy to watch Watson steamroll his sentient opponents and fear for the future of humanity. But amid all the hand-wringing, Baker noted, nobody seems to remember that there was a point during Watson’s four-year evolution when it took the machine more than two hours just to answer a question. Then, Baker added, “Nobody was talking about how fast Watson was on the buzzer.”

Baker, the former senior technology writer for BusinessWeek, interviewed the team behind the Watson program for his behind-the-scenes account, “Final Jeopardy: Man vs. Machine and the Quest to Know Everything.” As Baker notes, Watson is neither the product of computing “magic” nor he is anything close to a human brain. IBM’s researchers have focused only on one specific yet fundamental aspect of artificial intelligence: the ability to process natural language.

“It’s only relevant in the area of artificial intelligence that has to do with answering natural-language questions,” Baker said. “It doesn’t have anything to do with consciousness or vision or voice recognition or all these other areas of artificial intelligence.”

So Watson isn’t HAL 9000, the sentient and ultimately devious computer from “2001: A Space Odyssey” (a popular comparison in news reports this week). Nor is Watson just an advanced version of Google. The genius of Watson’s programming is not its ability to scour a vast database of information instantaneously, which search engines can already do, but its talent for parsing and understanding irony, wordplay and other qualities of human language.

“In that little corner of artificial intelligence,” Baker said, referring to natural language processing, “it’s very significant. This machine is by no means limited to Jeopardy. It can use its ability to understand English and go through big piles of documents and do analytics on those and try to come up with answers, and it could do it in many different fields.”

Perhaps even more significant than the answers Watson spits out, or how fast it buzzes in, is the degree of confidence the machine assigns to its results.

If you watched “Jeopardy” this week, for example, you may have noticed a series of bars and percentages at the bottom of the screen every time Watson answered a question. These figures describe the amount of confidence Watson has in each of its potential answers. Based on that confidence, Watson decides whether or not it’s smart to buzz in (as Ferrucci noted on the IBM blog Wednesday, Watson was not very confident in his “Toronto” guess and probably would not have attempted to answer had it not been Final Jeopardy).

That shows, as Baker explained, that Watson is “coming to an understanding of what it’s supposed to look for.”

Noting that Watson is “learning” and “understanding,” Baker said, one can only wonder, “Where will it be if it has 100 times as much computing power, and the algorithms are 100 times more intelligent?”

The answer, though, shouldn’t be cause for concern. Humans still control the terms of what machines like Watson do. Were we to tweak the rules of Jeopardy to emphasize humor over logic, for example, Watson might not dominate the way it did this week. We still determine the rules.

Even if supercomputers gain the ability to produce accurate, instantaneous answers to complex, natural-language questions, how we use that computing power is up to us.

“We’re not in any danger of losing our thought leadership in the world,” Baker said. “Each of us, in our own domain, are going to have to deal with machines like this and figure out how to use them, how to combine our smarts with their power to give ourselves an edge.”

  • thumb
    A talk on the dark side
    Three American astronomers specializing in dark energy were awarded the Nobel Prize in physics this week. Win Rosenfeld talks to Brian Greene about why dark energy has been a top priority for the astrophysics community in recent years.
  • thumb
    A hot rock and a gas blob
    It was a big week for cosmological discoveries, with an Earth-like planet and a star-making machine keeping astronomers riveted.
  • thumb
    The revolution will be … faxed?
    Google CEO Eric Schmidt extols the untapped potential of the fax machine.


  • Rsalvatore

    Bring it on!
    Watson is the master of Jeopardy.

    He beat the two all-time best contestants from over 30 years of humans, right?

    Computers have exceeded humans in
    1. numerical analysis
    2. chess
    3. and now Jeopardy.

    The humans continue to write articles about how humans are superior, but their domain of expertise is shrinking.

    Watson’s 2nd cousin

  • PM

    What most people don’t realize is that Watson was fed the information electronically. So by the time Alex finished reading the question Watson had already recieved it seconds ago and already had the answer. So it wasn’t a fair fight. In order for it to be fair, Watson needs to be able to read or understand speech. If the players were able to get the questions seconds in advance they could probably search the internet and come up with the correct answers too!

  • Orean Keels Jr.

    Watson had no internet access. It has never been connected to the internet.

  • David F., N.A.

    With all the commercials, which led up to the competition, and 99 percent of the introduction, we were led to believe that Watson and its competitors would be on a level playing field. And then, out of the [big] blue, Trebek spent a whole whopping 8 seconds to tell us that Watson would be fed the clues through text files. This was when my son and I immediately turned to each other and, at the exact same time, said “that’s cheating.” What PM posted is right; most people don’t know the significance of this. IBM needs to invest in a couple microphones, and maybe try stealing a voice recognition guru, or two, from Microsoft or Google.

    WATSON: I’ll take “Geraldo Rivera Moments” for two thousand, Alex.
    ALEX: This game show was paid a pretty penny to jeopardize its reputation by airing three 30-minute IBM infomercial.
    [middle lights light]
    ALEX: Watson?
    WATSON: What is “The Price is Right?”
    ALEX: We’ll accept that.

  • Luzmac

    Very entertaining. Worth watching

  • Crhilton

    No. He got a text message when the question appears in text before them.

    Yea, he can read quicker. Welcome to an advantage of being a machine.

  • Frank Fazekas

    I am not sure I agree with PM and David F that the way Watson received the clues gave it an unfair advantage. PM says that, “by the time Alex finished reading the question Watson had already recieved it seconds ago and already had the answer.” I watched the games on TV and was able to read the clues and come up with answers before Trebek had finished reading the clues also.

    However, this is not to say that Watson didn’t have unfair advantages. First of all, I believe Watson had a clear advantage in being able to “ring in” faster then either Rutter or Jennings. Another clear advantage was that Watson had no fear of failure. Watson also was not distracted by the audience, the lighting in the studio, the cameras, and most importantly, Alex Trebek’s tired old schtick and his pompous attitude. In other words, no emotion. Jennings and Rutter answered a vast majority of the questions correctly _when they were able to ring in before Watson_. But they are human, and no matter how hard you try, humans cannot block out emotion.

    While I am not surprised that Watson won by a huge margin, I was pleased with the performance of Ken Jennings and Brad Rutter, especially their improvement as the game went on. They got better and perhaps if they continued playing, we might have seen either contestant beat Watson sooner rather then later.

  • Weathervane

    I think one glaring weakness showed up in Watson’s inability to learn from the mistakes of prior answers to the same question. In at least one instance, I noticed a contestant incorrectly responding and then Watson ringing in second and giving the same response. A natural limitation given how Watson receives input, but still that could have been remedied by having someone provide a real-time transcript.

    I was unimpressed by Watson’s ability to retrieve answers to questions of pure fact (something like “This famous conflict involved the Hatfields and McCorys). That is exactly where I would expect it to excel. A more impressive feat would have been to correctly answer questions involving word play or questions where words had multiple meanings depending on context or how they were read (as in “read” being pronounced as “reed” or “red”).

    All in all, though, it was an interesting infomercial and got IBM some time in the news and entertainment cycle. Given IBM’s push to off-shore its operations and research, I wonder if it’ll be one of the company’s last shining moments of creativity in this country’s borders.

  • Liz

    I think the text file and speed is what gave Watson the advantage. I wonder how the outcome would have changed had the two human contestants not had to wait for Alex to finish reading the clue as usual – if they could read ahead and buzz in as soon as they knew the answer. This is how I beat my husband all the time – I read faster than he does.

  • Kirsten Zielinski

    i have one thing to say…. Colossus: The Forbin Project…

  • amber

    What I found most interesting about the contest were Watson’s second choice responses. His overall degree of confidence usually resulted in the correct response, but if you looked at his next two choices, they could be all over the map—often way off the map actually. I would be interested in a ‘silent test’ between Watson and the other two players where they each had a fixed amount of time after they “received” the question (orally, by reading, or by text) to provide their top 3 answers. Something tells me that the humans would provide better answers as their second choices. I know there are a lot of flaws in this scenario, I’m just saying that I found Watson’s other two choices much more interesting than how often he got the right answer.

  • Vincent Scarafino

    HAL gets a bad rap. He wasn’t devious, it was a programming error. In his instructions for the trip to Jupiter, there were certain instances where he was told to lie. HAL didn’t know how. He was never programmed to enable that. The error was interestingly called a Mobius loop. Cool, but not very informative.

  • Brucker

    Yes, this was the interesting part for me as well. While Watson did very well over all, when it was wrong, it was phenomenally wrong, which makes one wonder what sort of processes were going on. A lot of the time the secondary answers (and occasionally even the first answer) indicated that Watson failed to understand the category in particular.

  • Harold

    Except that the human contestants can read the clue with their eyes. Sure they don’t read quite as fast as Watson would receive the text, but they still don’t have to play entirely based on hearing the question.

  • Lyle Walford

    I agree that the rule requiring Alex to read the entire question before the buzzer could be pushed gave Watson the advantage. I would enjoy a rematch in which the contestants could press the buzzer instinctively or intuitively.