Live data from Hacker News

IBM's Watson recommended 'unsafe and incorrect' cancer treatments

gizmodo.com

51–60 of 68 posts

Re: IBM's Watson recommended 'unsafe and incorrect' cancer treatments

#51
post #45

Earlier quoted context omitted.

Having systems like autopilot in planes can make pilots worse at flying because they don't spend as much time practicing at the controls and monitoring what's happening. Then when something goes wrong, they risk not paying close enough attention to catch it, or not being practiced well enough to correct it. If you get complacent and just assume the computer knows what it's doing (because it usually does) this can and…

That's one of the implicit reasons why US based airlines in the past used to recruit retired pilots from U.S Air Force. In case instruments fail or contradictory signals are present, pilots with physical skills (kinesthesia, etc) can steer away from dangers. Two incidents demonstrate that: 1. https://en.wikipedia.org/wiki/Asiana_Airlines_Flight_214 2. https://en.wikipedia.org/wiki/Air_France_Flight_447 Pilots on both…

Aren't both of those just examples of pilot error, regardless of autopilot?

In the case of the Asiana flight, the captain chose a visual approach when he could have let the autopilot land it. And from there it's simple pilot error.

In the case of the air France flight, one of the pilots chose to ignore a stall warning and pull up instead of pushing down to gain speed. Sadly, in the case of no pilot input the plane actually would have ended up avoiding the stall on its own.

I of course agree that military pilots with real experience will likely perform better than civilian pilots, I just doubt autopilot had any part to play in either of those accidents, like the comment you replied to implies.

Re: IBM's Watson recommended 'unsafe and incorrect' cancer treatments

#52
post #45

Earlier quoted context omitted.

Having systems like autopilot in planes can make pilots worse at flying because they don't spend as much time practicing at the controls and monitoring what's happening. Then when something goes wrong, they risk not paying close enough attention to catch it, or not being practiced well enough to correct it. If you get complacent and just assume the computer knows what it's doing (because it usually does) this can and…

That's one of the implicit reasons why US based airlines in the past used to recruit retired pilots from U.S Air Force. In case instruments fail or contradictory signals are present, pilots with physical skills (kinesthesia, etc) can steer away from dangers. Two incidents demonstrate that: 1. https://en.wikipedia.org/wiki/Asiana_Airlines_Flight_214 2. https://en.wikipedia.org/wiki/Air_France_Flight_447 Pilots on both…

There's also been a lot of discussion and research about how to best interact with autopilot, and when the best times to turn it off and operate the airplane manually are.

There is a talk[1] about this that is really popular among pilots, and I honestly think it seems increasingly relevant to the rest of the world as more and more things become computer automated in some fashion.

[1] - https://www.youtube.com/watch?v=pN41LvuSz10

Re: IBM's Watson recommended 'unsafe and incorrect' cancer treatments

#54

That's okay, though, right? We're not talking about fully automating diagnosis and treatment. We're talking about using a tool to help doctors diagnose and treat more effectively. i.e. it's more like predictive text input than anything. If it makes it faster for you to diagnose, makes harder to diagnose stuff easier, and recommends the right treatment most of the time then it saves the doctor some time and energy. Th…

Having systems like autopilot in planes can make pilots worse at flying because they don't spend as much time practicing at the controls and monitoring what's happening. Then when something goes wrong, they risk not paying close enough attention to catch it, or not being practiced well enough to correct it. If you get complacent and just assume the computer knows what it's doing (because it usually does) this can and…

That's the difference between Oracles and Agents.

Oracle = make suggestion

Agent = Act on your behalf.

Waze might be starting off as an Oracle. but once you tie it into self driving cars, it'll become an agent acting on your behalf.

At a certain tipping point, creates really interesting questions. For example, if you re-direct 100% of traffic to less congested route, you end up creating traffic jams, so then you have to decide who goes on which route... maybe people who pay more get faster routing, or people deemed more important or busier by the algorithm...

Re: IBM's Watson recommended 'unsafe and incorrect' cancer treatments

#55

That's okay, though, right? We're not talking about fully automating diagnosis and treatment. We're talking about using a tool to help doctors diagnose and treat more effectively. i.e. it's more like predictive text input than anything. If it makes it faster for you to diagnose, makes harder to diagnose stuff easier, and recommends the right treatment most of the time then it saves the doctor some time and energy. Th…

The other question is how does it compare to human doctors, they also sometimes recommend unsafe or wrong treatment.

One way I tend to view ML is that, when its wrong, its catastrophically wrong. Because it doesn't actually understand anything its doing, and simply looking at a probability model and picking based off it, you end up with the issue that a few carefully selected pixel changes in a picture of a cat takes the model from a cat to an ostrich.

The model does not see a significant difference between the cat/ostrich, or cancer/cold, whereas we do; this implies that, when the model is wrong, it is likely to not just provide an incorrect treatment, but a catastrophically incorrect treatment.

Where the human sees a cat-like creature, and if not guess a cat, then something similar to a cat (4 legged, furry, etc), the ML model is willing to jump anywhere, in the worst case.

So its not just rate of misdiagnosis, but by how much as well.

Re: IBM's Watson recommended 'unsafe and incorrect' cancer treatments

#58

remember there was an article about how Watson was recommending treatments that doctors were never considering and that they were more effective. i guess no, they were not even relevant, which is why doctors were not recommending them

Both that story and this one can be true.

It can recommend unconsidered, more effective treatments just as readily as it can recommend unconsidered, dangerous ones — particularly if the system was as extensively trained using MSK's physicians' preferred treatments, instead of (or even additionally to) actual clinical data as The Fine Article suggests.

Re: IBM's Watson recommended 'unsafe and incorrect' cancer treatments

#59
post #26

Earlier quoted context omitted.

We're talking about using a tool to help doctors diagnose and treat more effectively. A very expensive tool when healthcare systems are strapped for cash. Super unethical of IBM to try and extract profits here.

>A very expensive tool when healthcare systems are strapped for cash. Super unethical of IBM to try and extract profits here. Do you realize they would have never developed Watson if there weren't profits involved?

I have no objection to them selling a working technology but they knew Watson was pure snake oil and did it anyway. Same as Theranos.

Re: IBM's Watson recommended 'unsafe and incorrect' cancer treatments

#60
post #59

Earlier quoted context omitted.

>A very expensive tool when healthcare systems are strapped for cash. Super unethical of IBM to try and extract profits here. Do you realize they would have never developed Watson if there weren't profits involved?

I have no objection to them selling a working technology but they knew Watson was pure snake oil and did it anyway. Same as Theranos.

If people require 100% accuracy out of the gate from any software then we might as well give up now. It's not as if doctors get it right all the time, if you know anyone with medical issues they've likely heard different opinions from different doctors themselves. Anything more complicated then a broken bone seems to get mixed responses about what should be done, or even what the cause of the complication is in the first place.
Post reply on HN