After “Jeopardy” began taping its three man-vs.-machine matches last month, pitting the IBM artificial intelligence software Watson against two of the game show’s most celebrated champions, host Alex Trebek confessed some concern about the contest to author Stephen Baker.
“Is this going to be fair?” he and the producers of “Jeopardy” asked, repeatedly. It wasn’t just a matter of ensuring an honest fight. The show’s producers wanted the Watson challenge to be compelling television. And if the machine — powered by a cluster of 90 servers with nearly 3,000 processing cores and 16 terabytes of data storage — made mincemeat of its human opponents, the show would be a pretty dull affair.
The IBM team, led by principal investigator David Ferrucci, reassured the show’s producers that the terms of the contest would, if anything, favor the humans.
“What we’re doing is we’re building a machine, and the machine has all kinds of weaknesses. It doesn’t understand language very well. And it doesn’t know anything,” Baker said, recalling the IBM team’s argument. “So we’re putting an ignorant machine that has a language handicap up, and you’re saying it’s not fair because it happens to be fast on the buzzer?”
Now, of course, observers are crying foul all over again, after Watson dispatched his human competitors with apparent ease. In Tuesday’s match, for example, Watson answered all but two questions correctly. The machine made a rather glaring flub by offering “Toronto” as the answer to a Final Jeopardy question whose category was “U.S. Cities,” but otherwise seemed unbeatable.
Watson cruised to victory in the final installment Wednesday, prompting Ken Jennings to write for his Final Jeopardy answer, “I, for one, welcome our new computer overlords.”
It’s easy to watch Watson steamroll his sentient opponents and fear for the future of humanity. But amid all the hand-wringing, Baker noted, nobody seems to remember that there was a point during Watson’s four-year evolution when it took the machine more than two hours just to answer a question. Then, Baker added, “Nobody was talking about how fast Watson was on the buzzer.”
Baker, the former senior technology writer for BusinessWeek, interviewed the team behind the Watson program for his behind-the-scenes account, “Final Jeopardy: Man vs. Machine and the Quest to Know Everything.” As Baker notes, Watson is neither the product of computing “magic” nor he is anything close to a human brain. IBM’s researchers have focused only on one specific yet fundamental aspect of artificial intelligence: the ability to process natural language.
“It’s only relevant in the area of artificial intelligence that has to do with answering natural-language questions,” Baker said. “It doesn’t have anything to do with consciousness or vision or voice recognition or all these other areas of artificial intelligence.”
So Watson isn’t HAL 9000, the sentient and ultimately devious computer from “2001: A Space Odyssey” (a popular comparison in news reports this week). Nor is Watson just an advanced version of Google. The genius of Watson’s programming is not its ability to scour a vast database of information instantaneously, which search engines can already do, but its talent for parsing and understanding irony, wordplay and other qualities of human language.
“In that little corner of artificial intelligence,” Baker said, referring to natural language processing, “it’s very significant. This machine is by no means limited to Jeopardy. It can use its ability to understand English and go through big piles of documents and do analytics on those and try to come up with answers, and it could do it in many different fields.”
Perhaps even more significant than the answers Watson spits out, or how fast it buzzes in, is the degree of confidence the machine assigns to its results.
If you watched “Jeopardy” this week, for example, you may have noticed a series of bars and percentages at the bottom of the screen every time Watson answered a question. These figures describe the amount of confidence Watson has in each of its potential answers. Based on that confidence, Watson decides whether or not it’s smart to buzz in (as Ferrucci noted on the IBM blog Wednesday, Watson was not very confident in his “Toronto” guess and probably would not have attempted to answer had it not been Final Jeopardy).
That shows, as Baker explained, that Watson is “coming to an understanding of what it’s supposed to look for.”
Noting that Watson is “learning” and “understanding,” Baker said, one can only wonder, “Where will it be if it has 100 times as much computing power, and the algorithms are 100 times more intelligent?”
The answer, though, shouldn’t be cause for concern. Humans still control the terms of what machines like Watson do. Were we to tweak the rules of Jeopardy to emphasize humor over logic, for example, Watson might not dominate the way it did this week. We still determine the rules.
Even if supercomputers gain the ability to produce accurate, instantaneous answers to complex, natural-language questions, how we use that computing power is up to us.
“We’re not in any danger of losing our thought leadership in the world,” Baker said. “Each of us, in our own domain, are going to have to deal with machines like this and figure out how to use them, how to combine our smarts with their power to give ourselves an edge.”