Tech: Big Blue's Chinese Room

Tomorrow, you'll get the rare chance to see a philosophical thought experiment come to life as IBM's Jeopardy!-playing computer faces off against two illustrious former champions on national television:



The thought experiment in question is John Searle's Chinese room. Here's the original version:

Suppose that I’m locked in a room and given a large batch of Chinese writing. Suppose furthermore (as is indeed the case) that I know no Chinese, either written or spoken . . . Chinese writing is just so many meaningless squiggles. Now suppose further that after this first batch of Chinese writing I am given a second batch of Chinese script together with a set of rules for correlating the second batch with the first batch. The rules are in English, and I understand these rules as well as any other native speaker of English. They enable me to correlate one set of formal symbols with another set of formal symbols, and all that "formal" means here is that I can identify the symbols entirely by their shapes. Now suppose also that I am given a third batch of Chinese symbols together with some instructions, again in English, that enable me to correlate elements of this third batch with the first two batches, and these rules instruct me how to give back certain Chinese symbols with certain sorts of shapes in response to certain sorts of shapes given me in the third batch. . .

Suppose also that after a while I get so good at following the instructions for manipulating the Chinese symbols and the programmers get so good at writing the programs that from the external point of view—that is, from the point of view of somebody outside the room in which I am locked—my answers to the questions are absolutely indistinguishable from those of native Chinese speakers. Nobody just looking at my answers can tell that I don’t speak a word of Chinese. . .

. . . it seems to me quite obvious in the example that I do not understand a word of the Chinese stories. I have inputs and outputs that are indistinguishable from those of the native Chinese speaker, and I can have any formal program you like, but I still understand nothing. For the same reasons, Schank’s computer understands nothing of any stories, whether in Chinese, English, or whatever, since in the Chinese case the computer is me, and in cases where the computer is not me, the computer has nothing more than I have in the case where I understand nothing.

IBM's Watson operates in much the same way as Searle's Chinese room. IBM's programmers have fed it enormous amounts of data, including enyclopedias, novels, plays, the Bible, and, of course, thousands of past Jeopardy! questions and answers (there's nothing unfair or foreign about this - human Jeopardy! players do the same kind of research, after all).

After receiving the Jeopardy! clue, Watson runs a large number of independent search algorithms in parallel on its databases, and then picks whichever answer is reached by the majority of the algorithms. It then takes this result and runs it back through its databases (including the past Jeopardy! questions) to see whether the answer conforms to past correct answers.

As you can tell, the way Watson "thinks" is completely unlike how a human being thinks. When Ken Jennings gets a Jeopardy! clue about a Gilbert and Sullivan opera, he doesn't have to eliminate "What's Eating Gilbert Grape" or New York Times Co. v. Sullivan in order to get to the correct answer.

In essence, Watson is a Chinese room - it can divine the grammar behind the question only because it has a huge set of complex rules telling it the grammar to follow. For Watson, there's no difference between a straight Jeopardy! clue and one laden with corny humor - they're treated the same and answered in exactly the same fashion.

All of this does nothing to lessen the fine achievement of IBM's designers; Watson is doing something that was once considered impossible. The applications are tantalizing - if a personal computer of tomorrow could "understand" human speech and commands the same way Watson does, it would be revolutionary (imagine a doctor being able to search for contraindications simply by feeding a patient's chart to a computer and asking it questions). Still, given that a 36 year-old computer scientist can match a computer with 16 terabytes of memory and ten refrigerators' worth of processing cores and networking hardware, I think we're a long way from SkyNet taking over the world: