Stephen Baker

The Numerati
Home - Viewing one post

How Watson thinks

January 30, 2011Travel

If you have ever tried to converse in a foreign language, you can understand an important aspect of Watson's thinking process. Human language is inherently foreign to the Jeopardy computer. With each sentence it's making the same kind of calculations we do when listening to a waiter recite the specials in French. Did he really say "intestines?" ...I could swear he said something about "snakes"...

Each time you hear a sentence in a foreign language (or even listen to a foreigner speaking your language with a strong accent) you calculate your confidence in your comprehension. If someone pronounces a simple sentence with a clear subject and object, and a familiar verb in the present tense, such confidence score will soar--especially if it fits with what you're expecting to hear. "It's a good hotel.... Barcelona is the best soccer team... You hear those sentences, give them high scores, and (if you're like me), you occasionally slip into brief euphoria: I understand this language!

More common though, are sentences that sound to foreign ears like this: "The people (unknown verb) to vote yesterday, but the police (verb) things and (missing words here) on the head."

At that point, your confidence in the sentence is low. You have to figure out what to do with less than 100% confidence that you know what's being said. This is Watson's life (if it had one). It is always calculating its level of comprehension, both of the Jeopardy clue and the documents its reading. Then it uses them to carry out extremely complex analysis involving probability. In many cases, it boils down to risk management calculations.

Like a foreigner, Watson operates in varying degrees of doubt. Yet based on its uncertain read of clues and evidence, it has to come up with a response and a betting strategy within three seconds.  Because it never knows anything the way we do, it has to allow for all sorts of contingencies. (There's a chance, though small, that waiter was talking about snakes.) It makes no safe assumptions. It ratchets down its confidence level, but never definitively rules anything out. While we often allow ourselves to think in black and white, Watson sticks to gray.

This is important in Jeopardy, because the game's language is full of puns and misdirection. Once on Jeopardy, there was a category "Country Clubs." A traditional information-retrieving computer would simply direct its hunt to a limited domain of golf and tennis spots, including the Merion Cricket Club,  Torrey Pines, Pebble Beach, and a few thousand others. That literal-minded strategy spells doom in Jeopardy. Here's a clue from the category:

"A French riot policeman may wield this, simply the French word for stick."

"Country club," it turns out, refers to the tools people batter each other with in various countries. In Artificial Intelligence, a computer that limits itself to one set of answers is "brittle." That's one thing Watson cannot be.

The flip side of brittleness is open-mindedness. That includes a readiness to consider answers that any reasonable person can see are foolish, or contradictory. Since Watson isn't a reasonable person, it can often look dumb. That's part of the fun. But it's the price the Jeopardy team pays for placing the computer in game built upon language that is endlessly, maddenly and, at times, hilariously complex.


©2021 Stephen Baker Media, All rights reserved.     Site by Infinet Design

Kirkus Reviews -

LibraryJournal - Library Journal

Booklist Reviews - David Pitt

Locus - Paul di Filippo

read more reviews

Prequel to The Boost: Dark Site
- December 3, 2014

The Boost: an excerpt
- April 15, 2014

My horrible Superbowl weekend, in perspective
- February 3, 2014

My coming novel: Boosting human cognition
- May 30, 2013

Why Nate Silver is never wrong
- November 8, 2012

The psychology behind bankers' hatred for Obama
- September 10, 2012

"Corporations are People": an op-ed
- August 16, 2011

Wall Street Journal excerpt: Final Jeopardy
- February 4, 2011

Why IBM's Watson is Smarter than Google
- January 9, 2011

Rethinking books
- October 3, 2010

The coming privacy boom
- August 17, 2010

The appeal of virtual
- May 18, 2010