Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine learning is booming in medicine, but also facing a credibility crisis

statnews.com

Post reply on HN
Print view View original

51–60 of 100 posts

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#51

Post by gjulianm »

klodolph wrote: post #49
gjulianm wrote: post #45

Earlier quoted context omitted.

We were talking not about about clear signs but about small signs that could be artifacts or actual issues. In that case you are going to have to do some complimentary or repeated tests to verify, so the ML output saying something is an artifact will have no effect on those procedures unless it's 100% accurate. But in general, these are matters of life and death, literally. You're still going to have someone looking…

I still don’t understand why decisions about life or death have to involve 100% certainty or 100% correctness. I would think that it’s the opposite, that when the stakes are high, you want anything that improves your odds. If your life were on the line, you’d accept treatments or diagnoses with much less than 100% certainty.

See my comment below for the same discussion: https://news.ycombinator.com/item?id=27433368

My point isn't that you need 100% accuracy. It's that a diagnosis is a process, not a single test. If your model applies to a single step of the process, and if it doesn't remove enough uncertainty about that step, you're still going to continue the diagnostic process and the model is not going to change anything really.

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#52

Post by exporectomy »

gjulianm wrote: post #48
exporectomy wrote: post #25

Earlier quoted context omitted.

If we can have cheap scanners or other instruments to go with the AI, I think it could easily appear as a screening tool that less qualified medical professionals (midwives, nurses, GPs) or even the patients themselves can use. Those people already do various rough screening tests and refer the patient up the chain of expertise according to the outcome.

> If we can have cheap scanners or other instruments to go with the AI That's a big if already. Not to mention that a lot of times you don't want to give unnecessary radiation to patients, so they wouldn't be interested in making, say, X-rays more accesible. > screening tool that less qualified medical professionals (midwives, nurses, GPs) or even the patients themselves can use But how beneficial would that be? When…

I was thinking more like ultrasound. If some crude scanner was cheap enough for every GP's office to have, and an AI could decipher the noisy signal which is perhaps too messy for a human to read, it could be used routinely like blood pressure measurement and stethoscopes are.

If it's a regular screening test rather than a response to the patient's complaint, then false negatives wouldn't be as much of a problem because at least it's better than nothing.

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#53

Post by gjulianm »

exporectomy wrote: post #52
gjulianm wrote: post #48

Earlier quoted context omitted.

> If we can have cheap scanners or other instruments to go with the AI That's a big if already. Not to mention that a lot of times you don't want to give unnecessary radiation to patients, so they wouldn't be interested in making, say, X-rays more accesible. > screening tool that less qualified medical professionals (midwives, nurses, GPs) or even the patients themselves can use But how beneficial would that be? When…

I was thinking more like ultrasound. If some crude scanner was cheap enough for every GP's office to have, and an AI could decipher the noisy signal which is perhaps too messy for a human to read, it could be used routinely like blood pressure measurement and stethoscopes are. If it's a regular screening test rather than a response to the patient's complaint, then false negatives wouldn't be as much of a problem beca…

Ultrasound scanners are fairly cheap. But operating them is not easy, you still need training to use them and know what you're looking at.

> it's better than nothing.

Not necessarily, that's the issue with screening asymptomatic people. You have to balance the consequences and rates of false positives with the benefits of true positives. If ultrasound screening mostly catches indolent diseases, or those where catching them so early doesn't affect outcomes too much; and you have a lot of people going through unnecessary/risky tests, you might end up doing more damage with those screenings. It's not so easy.

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#54

Post by klodolph »

gjulianm wrote: post #51
klodolph wrote: post #49

Earlier quoted context omitted.

I still don’t understand why decisions about life or death have to involve 100% certainty or 100% correctness. I would think that it’s the opposite, that when the stakes are high, you want anything that improves your odds. If your life were on the line, you’d accept treatments or diagnoses with much less than 100% certainty.

See my comment below for the same discussion: https://news.ycombinator.com/item?id=27433368 My point isn't that you need 100% accuracy. It's that a diagnosis is a process, not a single test. If your model applies to a single step of the process, and if it doesn't remove enough uncertainty about that step, you're still going to continue the diagnostic process and the model is not going to change anything really.

You said ML will have “no effect on those procedures unless it's 100% accurate.” I disagree with that, and think it’s a bit absurd—the 100% figure is not possible anyways, so you might as well stick some other impossible predicate in there and the statement is equally insightful.

E.g., “it will have no effect on outcomes until pigs fly.”

There will always be uncertainty, and uncertainty isn’t the only relevant parameter.

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#55

Post by pc86 »

klodolph wrote: post #16
gjulianm wrote: post #14

Earlier quoted context omitted.

The issue with that is that misclassifying something in an image can have serious consequences. So unless ML is 100% accurate, you'll always want an alternative way to check if it's really an artifact or something serious, so, why bother making an accurate model when you'll still double check the results?

I don't follow. Why would anything need to be 100% accurate? Humans aren't 100% accurate, and there's no reason ML would need to be 100% accurate either.

The is a gigantic difference between a doctor looking at something in a scan and deciding it's an image artifact when it's not, subsequently missing a diagnosis, and an ML algorithm looking at that same image, deciding something is an artifact and removing it from the image. At that point you're not looking at an actual scan, you're looking at something different.

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#56

Post by hdhjebebeb »

exporectomy wrote: post #52
gjulianm wrote: post #48

Earlier quoted context omitted.

> If we can have cheap scanners or other instruments to go with the AI That's a big if already. Not to mention that a lot of times you don't want to give unnecessary radiation to patients, so they wouldn't be interested in making, say, X-rays more accesible. > screening tool that less qualified medical professionals (midwives, nurses, GPs) or even the patients themselves can use But how beneficial would that be? When…

I was thinking more like ultrasound. If some crude scanner was cheap enough for every GP's office to have, and an AI could decipher the noisy signal which is perhaps too messy for a human to read, it could be used routinely like blood pressure measurement and stethoscopes are. If it's a regular screening test rather than a response to the patient's complaint, then false negatives wouldn't be as much of a problem beca…

I had an abdominal ultrasound recently - the actual process of placing the probe and annotating the images seems like a pretty specific technical skill. I don't think any old person (or even a doctor) off the street can just rub the wand on you and find your kidneys for instance.

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#57

Post by delfinom »

gjulianm wrote: post #8

Some radiologists think that AI will be really good for filtering out normal images, so that they only have to review anomalies. But I don't think that a model that detects diseases will be too successful, even if they manage to make it really work. For one, it's actually difficult to interpret and find signs in radiologic images. Obvious signs are obvious, but there are others that could be image artifacts, or just…

>Some radiologists think that AI will be really good for filtering out normal images, so that they only have to review anomalies. But I don't think that a model that detects diseases will be too successful, even if they manage to make it really work.

Radiologists say this until they get malpractice lawsuits. Seriously, I once injured my wrist and the first radiologist said it was just a sprain, but no surprise the hospital was using a outsourced...overseas provider for that quick analysis. The hospital had a inhouse radiologist review it as a course of business later on in a few days and the hospital panicked with phone calls to get me back in because there were numerous wrist bone fractures and they took the first step towards liability issues

Could an AI do it? Sure. But you sure as hell need a second opinion

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#58

Post by delfinom »

debarshri wrote: post #2

This is true for every other domain. When every company calls itself a AI company these days. Therefore value of AI/ML and credibility gets diluted. It becomes very difficult to classify real orgs using hardcode stats from fake ones. When I hear "we are solving x with AI" or "AI driven", it gives me jitters.

Most of the "AI" I see is just linear regressions run on meager datasets which lets them stick on that AI label but could have been written by a person with a little effort.

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#59

Post by delfinom »

eduramiba wrote: post #21
bayesian_horse wrote: post #13

Earlier quoted context omitted.

Medical ML is not about making decisions but about informing decisions. It is a tool for the doctor, not a replacement.

I think AI should always be a support tool, but people are always talking about replacing doctors... And a high volume if diverse quality data is fundamental: https://en.capillary.io/posts/ai-medical-diagnosis-guide/

>I think AI should always be a support tool, but people are always talking about replacing doctors...

Because Americans are trying to solve their healthcare crisis by attacking the doctors and literally not the entire rest of the industry which controls the spiraling healthcare costs. The end of result if we manage to eliminate doctors is still spiraling out of control healthcare.

Re: Machine learning is booming in medicine, but also facing a credibility crisis

#60

Post by lordgrenville »

qayxc wrote: post #46
f6v wrote: post #5

Earlier quoted context omitted.

The elephant in the room is that AI does not exist, even in "hardcore" orgs. A transformer model is not AI.

While I share the sentiment that the term "AI" is applied way too broadly these days, there is a point to be made that it's somewhat difficult to draw a clear line anyway. Back in the early 20th century, chess was considered to be something only an intelligent agent can do. A few decades later programs became competent enough that it took a grand master to beat them. Today, it's hopeless to try and beat a chess progr…

"As soon as it works, no one calls it AI anymore" - John McCarthy
Post reply on HN