Jeopardy! computer makes basic Canadian geography blunder

Computers make mistakes too. After all, they're only not-quite-human.

IBM's mega-machine Watson has so far been routing its human competitors in a three-day man-versus-machine Jeopardy! showdown. Last night, Watson racked up $35,734 on day two of the quiz show, leaving Brad Rutter in second with $10,400 and Ken Jennings in the rear with $4,800.

But the supercomputer's easy lead throughout the game made its mistake all the more puzzling in the Final Jeopardy! round.

Under the category of "U.S. Cities," the competitors were asked to name a city with an airport named after a Second World War hero and one after a Second World War battle. While Watson's human contestants correctly responded with Chicago, the clearly befuddled computer answered: "What is Toronto?????"

Jeez, you might think. Any fool knows Toronto isn't a U.S. city. That's, like, so elementary, dear Watson.

On the IBM-sponsored blog Building a Smarter Planet, however, IBM's David Ferrucci explains Watson's mistake was actually not so foolish at all.

In preparation for the match, Watson had been trained to recognize that the category labels of the game only loosely hint at the answer. Had the clue mentioned "what U.S. city" in the question, Watson would have given U.S. cities more consideration, Mr. Ferrucci said. Further confusing the computer, there are cities named Toronto in the U.S., while our Toronto Blue Jays play in the American League.

What's promising, Mr. Ferrucci said, is the fact Watson was aware it wasn't confident with its answer, and had bet intelligently by risking only $947 to keep its wide-margin lead.

"That's smart," he said. "You're in the middle of the contest. Hold onto your money. Why take a risk?"

Building a Smarter Planet also suggests Watson possesses the intelligence to understand subtleties and not make simplistic assumptions.

"Think about how Watson could be used in medicine, as a diagnostic aid," it says, explaining it could alert doctors to less obvious clues about a patient's symptoms.

Meanwhile, the human-machine Jeopardy battle continues tonight.

Who are you betting on?