IBM's Watson recommended 'unsafe and incorrect' cancer treatments • Hacker News Board

Re: IBM's Watson recommended 'unsafe and incorrect' cancer treatments

#31

Post by wlesieutre » Thu, Jul 26, 2018, 4:57 PM UTC

scarejunba wrote: post #12

That's okay, though, right? We're not talking about fully automating diagnosis and treatment. We're talking about using a tool to help doctors diagnose and treat more effectively. i.e. it's more like predictive text input than anything. If it makes it faster for you to diagnose, makes harder to diagnose stuff easier, and recommends the right treatment most of the time then it saves the doctor some time and energy. Th…

Having systems like autopilot in planes can make pilots worse at flying because they don't spend as much time practicing at the controls and monitoring what's happening. Then when something goes wrong, they risk not paying close enough attention to catch it, or not being practiced well enough to correct it.

If you get complacent and just assume the computer knows what it's doing (because it usually does) this can and will end very badly.

The person whose Tesla drove into a concrete barrier at 65 mph earlier this year was perfectly capable of driving a car, but they mistakenly believed that the computer had it under control.

99% Invisible has a pair of podcast episodes on the subject (2015):

https://99percentinvisible.org/episode/children-of-the-magen...

https://99percentinvisible.org/episode/johnnycab-automation-...

Re: IBM's Watson recommended 'unsafe and incorrect' cancer treatments

#32

Post by deviationblue » Thu, Jul 26, 2018, 4:58 PM UTC

malchow wrote: post #22

deviationblue wrote: post #4

We're putting a lot of faith in tools that can best be described as immature. I don't think it's out of the question to get A.I. (not speaking of Watson, which is A.I. adjacent) to the point where it can perform perfectly at human intelligence tasks. The point is to have these systems perform better than their human counterparts (at least, I think that should be aim for something like Watson for Health). The people w…

"We're putting a lot of faith in..." OK. Back in the real world, successful AI companies build useful software products that inform human decision making but do not directly, unilaterally, unintermediatedly result in real-world kinetic action.

I suggest holding on to the knee jerk reaction till you read the whole post..? I never said these tools will never be worthy of trust and performance at the levels we attribute to humans, but at their current point, there are some tasks that they're not good at because either the tool is limited by design or hardware, or we just haven't found the right solution yet.

Re: IBM's Watson recommended 'unsafe and incorrect' cancer treatments

#33

Post by throwaway5752 » Thu, Jul 26, 2018, 5:02 PM UTC

HlessClaudesman wrote: post #28

jobigoud wrote: post #18

Earlier quoted context omitted.

The other question is how does it compare to human doctors, they also sometimes recommend unsafe or wrong treatment.

The some other other questions: how much does it cost, compared to training a human? Does it actually save any time / resources if it's outputs must always be scrutinized?

That may be a question someone outside of an FDA approval process might asked. But if you have been, you understand the importance of "safety and effectiveness", and you will note there is nothing there about "cost effective" or "marketable".

Re: IBM's Watson recommended 'unsafe and incorrect' cancer treatments

#34

Post by deviationblue » Thu, Jul 26, 2018, 5:04 PM UTC

binarymax wrote: post #13

deviationblue wrote: post #4

We're putting a lot of faith in tools that can best be described as immature. I don't think it's out of the question to get A.I. (not speaking of Watson, which is A.I. adjacent) to the point where it can perform perfectly at human intelligence tasks. The point is to have these systems perform better than their human counterparts (at least, I think that should be aim for something like Watson for Health). The people w…

Totally agree. Applying existing ML/"AI" to treating cancer is extremely misguided, and IMO dangerous and unethical. ML performs very well for specific well defined tasks that have an obvious outcome and are highly narrow in scope. Cancer is a disease that we can't even treat ourselves in many cases. It requires a great deal of creativity and critical thinking to reach solutions on a case by case basis. Who is so arr…

It's not dangerous and unethical if humans are there to pass final judgment on the answers. I was saying we can get to point where we might not even need that. This technology is in development, it has a lot of potential it can, will possibly, reach. Stories like these are good for caution, but doesn't mean we shouldn't use these tool there.

Re: IBM's Watson recommended 'unsafe and incorrect' cancer treatments

#35

Post by ajmurmann » Thu, Jul 26, 2018, 5:05 PM UTC

scarejunba wrote: post #12

That's okay, though, right? We're not talking about fully automating diagnosis and treatment. We're talking about using a tool to help doctors diagnose and treat more effectively. i.e. it's more like predictive text input than anything. If it makes it faster for you to diagnose, makes harder to diagnose stuff easier, and recommends the right treatment most of the time then it saves the doctor some time and energy. Th…

To me the key is how you use the system. I like the idea of using this not to help but second guess the doctor. The doctor has to make their own diagnosis. If Watson differs a arbiter has to look at it. This obviously optimizes for quality and not cost.

Re: IBM's Watson recommended 'unsafe and incorrect' cancer treatments

#36

Post by xyhopguy » Thu, Jul 26, 2018, 5:08 PM UTC

I work in this space. The breakthrough will most likely come from computational biologists, not an external ML group. Liquid biopsy is close and that's the domain that reeeallly needs ML/AI in additional to strong mechanistic models.

Re: IBM's Watson recommended 'unsafe and incorrect' cancer treatments

#37

Post by ballenf » Thu, Jul 26, 2018, 5:17 PM UTC

wlesieutre wrote: post #31

scarejunba wrote: post #12

That's okay, though, right? We're not talking about fully automating diagnosis and treatment. We're talking about using a tool to help doctors diagnose and treat more effectively. i.e. it's more like predictive text input than anything. If it makes it faster for you to diagnose, makes harder to diagnose stuff easier, and recommends the right treatment most of the time then it saves the doctor some time and energy. Th…

Having systems like autopilot in planes can make pilots worse at flying because they don't spend as much time practicing at the controls and monitoring what's happening. Then when something goes wrong, they risk not paying close enough attention to catch it, or not being practiced well enough to correct it. If you get complacent and just assume the computer knows what it's doing (because it usually does) this can and…

Instead of autopilot, think of this as (or should be) more like fly-by-wire systems in every modern fighter jet. The computer reads the pilot's inputs and translates them to the most effective possible control adjustments, making many more changes than a pilot would be able to make even given the controls.

Or maybe the tool should be limited to retrospective evaluation of doctors' decisions. Basically an automated peer review.

Re: IBM's Watson recommended 'unsafe and incorrect' cancer treatments

#38

Post by FussyZeus » Thu, Jul 26, 2018, 5:24 PM UTC

ballenf wrote: post #37

wlesieutre wrote: post #31

Earlier quoted context omitted.

Having systems like autopilot in planes can make pilots worse at flying because they don't spend as much time practicing at the controls and monitoring what's happening. Then when something goes wrong, they risk not paying close enough attention to catch it, or not being practiced well enough to correct it. If you get complacent and just assume the computer knows what it's doing (because it usually does) this can and…

Instead of autopilot, think of this as (or should be) more like fly-by-wire systems in every modern fighter jet. The computer reads the pilot's inputs and translates them to the most effective possible control adjustments, making many more changes than a pilot would be able to make even given the controls. Or maybe the tool should be limited to retrospective evaluation of doctors' decisions. Basically an automated pe…

I would argue against this one, simply because of the Airbus incident from some time ago where to avoid pushing feedback into the controls, the plane was designed to average out the inputs from the two pilots. The older, more experienced pilot was attempting to maneuver the plane in such a way to correct their problem (I believe it was a stall) and the younger pilot was panicking and attempting the opposite maneuver, the plane averaged it to roughly no course adjustment, and they hit a mountainside.

I'm all for automating driving and other such tasks once the computers are ready, but until we know they're at least most likely ready to do it, I want the ability to turn it off and do it myself.

(And frankly, I want that ability anyway because I genuinely love driving and it makes me sad to think someday I won't be able to do it.)

Re: IBM's Watson recommended 'unsafe and incorrect' cancer treatments

#39

Post by wlesieutre » Thu, Jul 26, 2018, 5:24 PM UTC

ballenf wrote: post #37

wlesieutre wrote: post #31

Earlier quoted context omitted.

Having systems like autopilot in planes can make pilots worse at flying because they don't spend as much time practicing at the controls and monitoring what's happening. Then when something goes wrong, they risk not paying close enough attention to catch it, or not being practiced well enough to correct it. If you get complacent and just assume the computer knows what it's doing (because it usually does) this can and…

Instead of autopilot, think of this as (or should be) more like fly-by-wire systems in every modern fighter jet. The computer reads the pilot's inputs and translates them to the most effective possible control adjustments, making many more changes than a pilot would be able to make even given the controls. Or maybe the tool should be limited to retrospective evaluation of doctors' decisions. Basically an automated pe…

Even if you let the computer run the show wholesale it could work out net positive. Autopilot might cause problems, but it might also prevent twice as many accidents that would have been caused by pilot error.

We're still in pretty early days here, so I don't know if Watson's advice is at a point where that could be true. In the meantime, I do like the idea of having an unaided doctor and the computer program evaluate it independently, then afterward say "Ok, let's see what the computer thought" before coming to any final conclusions. Maybe it'd help avoid any dangerous recommendations. Or maybe people would say "Well the computer has a lot more data than me, it's probably right." I think that depends on whether the differences are errors similar to what a human would make, or if they're the "oh my god how did it even come up with this result" nonsensical error that are easily spotted by human review.

Re: IBM's Watson recommended 'unsafe and incorrect' cancer treatments

#40

Post by [deleted] » Thu, Jul 26, 2018, 5:25 PM UTC

lawrenceyan wrote: post #14

fatjokes wrote: post #8

Honest question: how much top-tier ML research comes out of IBM Research? I feel like they get what they paid for---and they pay shit from what I hear.

Should have partnered with Google if you actually wanted something that works. IBM had its heyday, but the brain drain of talent and knowledge has already happened and it likely will never recover.

[deleted]