While I haven't done a deep dive on Watson's utility, here's an example of a scenario where Watson was successful in matching humans, but the implementation was challenging, customer interest was low, and the IBM Watson cut didn't get past a pilot phase.[0] This came after spending $50-100 million.

To your question, the hospital wasn't "done" with teh effort and is still shopping around for IBM Watson competitors as a substitute to go forward with.

[0] https://www.wsj.com/articles/hospital-stumbles-in-bid-to-tea...

Non-paywalled version: https://ih.advfn.com/p.php?pid=nmona&article=74041396

Yeah seems pretty damning, at least for that effort given how much money was spent. Perhaps not damning Watson's capacity, but without a good integration strategy you can have the best AI on the planet and it'll be a race car without any gas.

From the article: "He also added IBM "wasn't responsible" for integrating Watson with MD Anderson's new electronic medical records."

Also: "The way medical information is stored and labeled can differ widely, even between departments at the same institution, he said. For instance, "there's no standard way to record a heart rate, a blood-glucose value or temperature measured at the bedside," he said. If the way data is stored or labeled changes, often the artificial-intelligence software must be retrained, he said. "

All valid complaints, but clearly IBM failed to properly consider the implementation challenges. That's never a good thing from a business perspective.

Except Watson, Siri, Alexa, etc aren't real AI. They are very specific applications for a specific task. The idea that they are AI has been great for marketing and creating another "Strong AI is around the corner bubble."

But look at how slowly they have evolved. Siri is barely better than she was 5 years ago. I haven't seen Watson really do much beyond the same language processing and information lookup stuff it did on Jeopardy years ago. The other assistants are somewhat evolved versions of Siri. But they all require special coding to teach them any new task. None of them is really anything like true AI. They are AI only in the sense that they are programs which appear to exhibit intelligent behavior in a very narrow and specific role, like an AI opponent in a video game. But I strongly believe we are not appreciably closer to true AI than we were 10 years ago. We are just in another bubble like the 80s because the people with money don't understand the massive gap between Siri and Hal.

> Except Watson, Siri, Alexa, etc aren't real AI.

From the Wikipedia article on Artificial Intelligence: "In computer science, the field of AI research defines itself as the study of "intelligent agents": any device that perceives its environment and takes actions that maximize its chance of success at some goal."

A bimetallic mechanical thermostat perceives its environment (the temperature) and takes actions (turning the heat on and off) to achieve a goal (maintaining a target temperature)

Is a thermostat an AI?

Yes. It's just a very boring one.

Something something...AI...once it's understood...

You think AI is something like "true AI", I think a toaster that exhibits a basic decision tree is AI, which of us is right? There's just no way to know, we've come to an impasse :)

Yes. But very limited. AI is a meaningless term. Soft AI is often just search and applied statistics, applied signal processing (or statistics applied to signal processing), or a rediscovery of inefficient methods for mimicking control theory. Hard AI is a dream that some people have with no clear way to get there.

We should stop using the term since every decade AI applies to a different set of techniques and tools, and the old ones become standard algorithms and data science.

I guess you're one of the people to whom there is no "real" AI below general purpos, human-level intelligence. To me, the "simple" language processing and information lookup that Watson does is most definitely AI. Jeopardy is not an easy game and beating the best humans at it was an amazing break-through.

> Jeopardy is not an easy game and beating the best humans at it was an amazing break-through.

The Jeopardy game was marketing. The Kasparov game was marketing. Both were very impressive, but they were marketing.

I think it is telling that whenever anyone mentions IBM and AI near each other in speech that mention of a public AI project (see:marketing demo) is usually just a moment away.

It would be much more impressive to me if someone once said "Oh Company X saves Y amount using Watson" or "Watson tech enables X" which are examples that are easy to come up with when speaking of IBM's competitors.

>Both were very impressive, but they were marketing.

Determining product-market-fit, and go-to-market strategy are also 'marketing.' Most of the time (and, definitely in this case), when people use the term 'marketing' on HN, it is better replaced by promotional/advertising which are much more specific/ meaningful.

It beat them because most of the questions were only related to factual knowledge. It utterly failed on the wordplay questions.

What is Toronto?

> I guess you're one of the people to whom there is no "real" AI below general purpos, human-level intelligence.

I don't think we need human level intelligence to have a more real intelligence. After all there is not just one tier of human intelligence, even among "normal" human brains. A toddler isn't very intelligent compared to an adult.

For me the difference is between:

A: A piece of software designed to fulfill a narrow task which appears to be intelligent to those who don't know how it works.

B: A piece of software which can actually understand and reason in some capacity about things more generally and especially does not need explicit programming to add new features.

As per my previous comment, I think Siri, Watson, Alexa, etc. are applications which appear intelligent as opposed to actual AI because they do not really understand anything. They appear to understand things, but it's a trick. They understand the command "Give me directions to Starbucks." but only in the same sense that bash understands "egrep -r 'PizzaController' ./bloated-app/classes". bash only knows those are arguments that get passed into the egrep program. The current assistants are just audio command lines.

This is a long way of saying; yes I guess you could say that I think only "general AI" is real AI. That's because anything short of that, at least that we've seen so far, is just really applications that appear intelligent but aren't in any real sense intelligent. Importantly, like I said before, I don't think they've moved us forward toward general AI. Look how slowly they have advanced and the techniques used to create them, other than the voice processing and voice responding, are not techniques that will lead to general AI, because all the features are explicitly individually coded. Which is something you absolutely cannot do for general AI.

EDIT:

I realize it sounds like I'm just shitting all over Alexa, Siri, etc. but I don't mean to. They are useful pieces of software, but I think they are over hyped by calling them AI and especially how the media and certain people who should know better take them and say "strong AI is just around the corner!"

The whole "AI is just around the corner!" thing has happened in the past (the last major investment boom being in the 80s), and I see no reason to be any more optimistic about this one. Our computers are more powerful and our datasets are bigger, but I don't see any techniques that are revolutionary, and the datasets and computer power are still a joke compared to a human brain.

To me there is no difference between appearing intelligent and being intelligent. That's what the Turing test is all about.

Yeah it is what the Turing Test is all about and it's why I (like a lot of people) have always believed the Turing Test is insufficient as a test for AI.

If you don't believe the Turing test is sufficient, how you find out whether anybody (except yourself I guess) is intelligent or merely pretending?

I don't know about "just in another bubble" but I agree with the general sense of your comment.

The voice recognition of digital assistants has gotten decent (for appropriate speakers and languages). But we don't seem to be remotely close to, for example, a digital assistant that can compete with the most marginally competent human admin--even for digital-native tasks.

Watson does do marketing well. There may be some domain-specific tasks, e.g. in medicine, that it turns out to be good for. But, again, at IBM's scale the business opportunity has to be huge to move the needle.

Doesn't have to be true AI to be worth investing in.

At SXSW one year IBM had Watson come up with a bunch of new recipes to try. I thought they all tasted awful. Some things are just not suited to the current AI tools.

Edit: typo

I'd say they have the leading household name and a good amount of marketing behind it, but I don't see any of their offerings as superior to Google's, and AWS is catching up quickly.

Perhaps not superior, but they're there and have the capacity to compete, which is more than can be said for basically everyone else. If voice and natural language processing truly are the future then I wouldn't think IBM would be a bad bet. But then again I don't own a massive chunk of IBM and am reliant on public info. So perhaps they're less competitive than it appears. I suppose they have roughly the same problem Apple has with a general lack of information (or willingness to use said information) about their customers.

"Watson seems to be one of the leading contenders there" ... Could you elaborate? I'm interested to know why. The only group of people I know who are interested in Watson are government folks, everyone else seems to be into AWS of Google Cloud products.

Watson is effectively just a brand for machine learning consulting.

I played around with Watson's visual recognition api and thought it was decent. Care to elaborate?

A "decent" visual recognition API at this point isn't very good or useful. The other comment in this thread is correct - if you want to make a truly useful service out of Watson, they do it as a consultation and do it essentially from scratch.