Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

IBM Watson Overpromised and Underdelivered on AI Health Care

spectrum.ieee.org

Post reply on HN
Print view View original

91–100 of 127 posts

Re: IBM Watson Overpromised and Underdelivered on AI Health Care

#91

Post by nickpsecurity »

seibelj wrote: post #43
taylodl wrote: post #14

I figure we’re going to be reading articles similar to this about blockchain in a couple of years.

The fact Bitcoin is worth $90 billion and has survived a decade of endless criticism, means blockchain is successful beyond anyone's wildest dreams a decade ago. I would say Bitcoin and Ethereum have been way more successful than Watson has.

The Federal Reserve is worth maybe every dollar on the planet. It's survived almost a hundred years. It bankrolled Bitcoin with it and most cryptocurrencies depending on dollars for investments, paying employees, etc. From centralized banking to sustaining decentralized platforms, the Federal Reserve is successful beyond anyone's wildest dreams in the 1910's. Except maybe the people that created it.

See why your argument doesn't prove Bitcoin is a good thing in the long term and/or if you're a believer in its superiority to centralized banking?

Re: IBM Watson Overpromised and Underdelivered on AI Health Care

#92

Post by craftinator »

return0 wrote: post #47
bitcharmer wrote: post #2

Don't want to sound too mean but this is not the first time IBM overpromised and underdelivered. Anyone surprised by this?

Yep they are e.g. overhyping their neurmorphic platform too I hope that's not the case for their quantum computer

But can their quantum computer factor the number "15"?

Re: IBM Watson Overpromised and Underdelivered on AI Health Care

#93

Post by AlotOfReading »

djakjxnanjak wrote: post #85
ethbro wrote: post #78

Earlier quoted context omitted.

I call this the Netflix ML effect. If your answer is mission critical, probabilistic ML isn't advanced, explainable, or reliable enough for your problem. If your answer is of the great-to-be-right, meh-to-be-wrong sort (e.g. ranking movie recommendations), then you can and should go nuts with ML. And if someone really wants to do an ML project on the former, do everything you can to transform it into the latter.

If you can tune the specifics-sensitivity curve, you should be able to handle both cases. You have to be willing to refuse to provide an answer when you have low confidence.

The existence of adversarial attacks with high confidence on virtually all production ML systems should indicate that confidence numbers are not enough to rely on.

Re: IBM Watson Overpromised and Underdelivered on AI Health Care

#94

Post by tosser1234 »

telchar wrote: post #32
tosser1234 wrote: post #23

I worked on this project and there were a lot of issues. Two of the biggest were: * Whatever the quality of the technology (which I personally never saw as that compelling) was wrapped up in terribly written research code, making it practically impossible to setup and use. * The Jeopardy demo was made possible by the existence of a marked-up source of general knowledge (Wikipedia), a ready-made bank of questions and…

That's not really true. The UMLS has a large graph of marked up medical domain knowledge that can be used. It's not as specific as one might want for developing an AI autodoc, but it's quite a bit better than what is available in most fields. It's actually quite similar to what one can derive from Wikipedia.

Having that taxonomy is definitely a great starting point for the field, but just to build a model to extract patient data from the text of a medical record you'd need to:

* Procure patient medical files. This rabbit hole includes things like HIPPA compliance and sufficiently anonymizing the files. Also, hospitals consider 3 years of information on 100 patients to be a lot of data.

* Have trained medical professionals annotate the files. Good luck convincing doctors to do this. Also good luck in determining what to do if the handful of doctors you're granted a few hours with a week can't agree on annotations.

* Actually produce a model that is reasonably good at pulling information out of patient charts,

Even after all the above, you still only have a solution that is giving you structured data, which is something that other EMRs can provide w/o the problems of accuracy & completeness of NLP process. There is still the process of diagnosis and treatment and also the fact that this is all a loop ('active learning').

Re: IBM Watson Overpromised and Underdelivered on AI Health Care

#96

Post by seibelj »

nickpsecurity wrote: post #91
seibelj wrote: post #43

Earlier quoted context omitted.

The fact Bitcoin is worth $90 billion and has survived a decade of endless criticism, means blockchain is successful beyond anyone's wildest dreams a decade ago. I would say Bitcoin and Ethereum have been way more successful than Watson has.

The Federal Reserve is worth maybe every dollar on the planet. It's survived almost a hundred years. It bankrolled Bitcoin with it and most cryptocurrencies depending on dollars for investments, paying employees, etc. From centralized banking to sustaining decentralized platforms, the Federal Reserve is successful beyond anyone's wildest dreams in the 1910's. Except maybe the people that created it. See why your argu…

I made no comment on Bitcoin or other cryptocurrencies replacing the Federal Reserve. I am arguing that Bitcoin and other cryptocurrencies are, 1) Not worthless, 2) Not a scam, 3) Have interesting technical properties that are worthy of study and improvement, 4) Have real-world utility, and 5) Should not be immediately dismissed by HN groupthink.

Re: IBM Watson Overpromised and Underdelivered on AI Health Care

#97

Post by tosser1234 »

nickpsecurity wrote: post #90
telchar wrote: post #32

Earlier quoted context omitted.

That's not really true. The UMLS has a large graph of marked up medical domain knowledge that can be used. It's not as specific as one might want for developing an AI autodoc, but it's quite a bit better than what is available in most fields. It's actually quite similar to what one can derive from Wikipedia.

Quora has a nice set of answers about why machine learning isn't used so much in medical: https://www.quora.com/Why-isn%E2%80%99t-machine-learning-mor... Skip to Jae Won Joe's answer to see a case study with a patient showing the interactive process. Then, especially look at ending questions about missing or deceptive data. Seems like their good diagnoses comes from a combination of domain data and expertly reading p…

This is a great run-down of the realizations that a lot of the engineering team came to and what I was trying to touch on in my reply above. There is just so much information & context within a doctor's head and so much that is going on during an examination that looks simple to our eyes. Even in the idealized case of having accurate, structured patient data and focusing only on a simple disease, it's a challenge to diagnose and prescribe a correct treatment. Even a system that can accept the text of a patient's medical record and successfully pull out relevant details would be challenge.

Re: IBM Watson Overpromised and Underdelivered on AI Health Care

#98

Post by thatoneuser »

rocgf wrote: post #58

At one point, there was an API available via the IBM Cloud that you could use in order to ask Watson questions. I've rarely been that underwhelmed. After using that, I realized that Watson was mostly a gimmick.

What were the limits?

Re: IBM Watson Overpromised and Underdelivered on AI Health Care

#99

Post by Randypea »

1e-9 wrote: post #81
Randypea wrote: post #46

I was recently diagnosed with neuroendocrine cancer, which my PCP had been misdiagnosed for 10 years as IBS. This is more the norm than the exception for people with this type of cancer, Steve Jobs included. It's a perfect example of where AI can likely diagnosis what my PCP could not. AI tools need to find cancer problems to solve which are more suited to their capabilities. e.g. does anybody know of a company worki…

Unfortunately, there are still many cancers detected far too late for effective treatment. It sounds like you were indeed fortunate to have a less aggressive form. AI for cancer screening generally falls under the category of "Computer-Aided Detection" or CAD. The commercial and academic CAD efforts tend to be organized by the primary anatomical site of cancer and the detection method (e.g. X-Ray, CT-Scan, PET, ultra…

My cancer was finally found in my intestine. This is another opportunity. My primary oncologist and local surgeon were telling me the primary tumor, which has metastasized to a very large liver tumor, could not be found. I did my own research and found they had ordered the wrong type of imaging scan. Only after I pushed to have the correct scan (Gallium 68 PET/CT scan) was the primary tumor found. This was a "lack of information" for my local oncologist. Computer-aided diagnosis would have helped him. An additional new symptom (flushing) appeared and my PCP recognized a specific cheap blood test was needed that led to the cancer being found. I am happy to contribute to an imaging study. But I want to work on cancer screening. What kind of automation/screening would be needed to prevent 10 years of misdiagnosis by my PCP? ... not only for this type of cancer but for all of the top 15-20 types of cancer. People are not being screened. How can we make screening affordable? And how can we raise awareness of possible misdiagnosis and or affordable screening?

Re: IBM Watson Overpromised and Underdelivered on AI Health Care

#100

Post by sk5t »

scottlegrand2 wrote: post #80
scrooched_moose wrote: post #54

Earlier quoted context omitted.

I've always had a 20% threshold If anyone is promising a 20% or greater improvement over the current leaders, I immediately file it away as a scam until I see extraordinary proof. Other than some very immature industries, even 20% leaps don't happen without earth shattering once-in-a-generation breakthroughs. Some call me a pessimist, and occasionally I'm unfairly dismissive of new technologies. But for every one I g…

CUDA delivered a 10x to 100x performance boost on legacy code for real. The hardware had been capable of doing this for at least a generations previously but it lacked a few operations to make it accessible and efficient. What I think made people miss it is no one wanted to refactor their code to run efficiently on GPUs. It wasn't that hard to do so in a lot of cases but it was work. And all this set the stage for de…

> CUDA delivered a 10x to 100x performance boost on legacy code

CUDA offered a way to access a limited set of operations on special-purpose hardware with a rewrite of critical sections. A pretty far cry from 'boosting legacy code.'

Post reply on HN