In all fairness the Toronto answer isn't as ridiculous as it seems at first blush. If you confuse Lester B. Pearson with Alastair Pearson (something quite possible by NLP when you have to match "Pearson international airport" then you at least match you end up with a British World War II hero. Not being a U.S. city would involve either underfitting or ignoring category information, and trying to stretch buttonville or billy bishop into a WWII battle takes work, but it seems like precisely the kind of error that would be caused by actual NLP rather than optimizing page rank variants.
I also think IBM has the right model by creating Watson as a service on 30% revenue share -> Let external developers find all the various products and business models. They'll be better at finding and optimizing new products than a services companies would be and some of them will be able to go after much smaller products. You end up with more diversification and less of IBM's capital at risk for 30% of what's likely to be a much larger pie than IBM can generate on their own.
It would be very interesting to find out how Watson actually interpreted both the category and the "answer".
I don't know the actual amounts of time contestants are given, but on TV the final category is disclosed many minutes ahead of the "answer". Then a contestant has perhaps 30 seconds to formulate a "question". An eternity for a massively parallel computer like Watson. Quite different from the rest of the show, which relies on lightning reflexes when played by "champions".
Viewed from 30,000 feet (quite appropriate for an airport question, eh) I think that Watson didn't understand the category. Watson did not know what the words "US cities" meant. Those aren't words that would normally have any sort of double meaning, so if Watson understood the category, why would it have "ignored" category information? That doesn't make sense, especially for final Jeopardy.
As Dr. Venkman might say: "good guess, but wrong!" In Watson's defense, it did not have very high confidence in its answer.
We'll probably never know the real story. That's not the kind of information that IBM would want to disclose, mainly because it would probably make Watson look bad.
I also think IBM has the right model by creating Watson as a service on 30% revenue share -> Let external developers find all the various products and business models. They'll be better at finding and optimizing new products than a services companies would be and some of them will be able to go after much smaller products. You end up with more diversification and less of IBM's capital at risk for 30% of what's likely to be a much larger pie than IBM can generate on their own.