Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Detects Heart Failure from One Heartbeat: Study

forbes.com

Post reply on HN
Print view View original

61–70 of 74 posts

Re: AI Detects Heart Failure from One Heartbeat: Study

#61

Post by carbocation »

brilee wrote: post #52
carbocation wrote: post #42

Earlier quoted context omitted.

The study looks at 33 patients total, and the cases and controls come from entirely different data sets, with data coming from different devices that recorded signal at different frequencies. There is nothing to see here.

Apologies - I didn't catch that the HF / healthy patients are from two different datasets. Agreed that this essentially invalidates the result. The missing experiment is to have a third dataset from yet another machine, with both positive/negative examples, and use it as the test dataset. Then transferability questions are at least somewhat addressed.

No apologies needed, we're all in agreement on the broad strokes! Your proposal is good: I would be quite surprised if it generalized, but that is definitely the way to find out.

Re: AI Detects Heart Failure from One Heartbeat: Study

#62

Post by carbocation »

3b18 wrote: post #50
Sanzig wrote: post #35

Earlier quoted context omitted.

Would you happen to have a source for that story? My workplace has really swallowed the AI Kool-Aid lately, so I would like to have some cautionary counterexamples to demonstrate potential pitfalls of the technology. It's got a lot of interesting applications for our field which I am excited about, but there seems to be a tendency among non-experts to consider it a magic bullet that can solve any sort of problem. In…

Check this link out http://132.206.230.229/e706/gaming.examples.in.AI.html There was a hn discussion about it a while back: https://news.ycombinator.com/item?id=18415031

What a great resource! The first example is perfect:

> Aircraft landing

> Evolved algorithm for landing aircraft exploited overflow errors in the physics simulator by creating large forces that were estimated to be zero, resulting in a perfect score

Re: AI Detects Heart Failure from One Heartbeat: Study

#63

Post by kbody »

Clickbait title aside. I find that ethical issues around AI raised by Musk etc. shouldn't be around AI taking over the planet, but rather have ethics around overfitted models or otherwise unrealistic models being pushed for PR or whatever and responsibly playing with people's health and hopes.

Re: AI Detects Heart Failure from One Heartbeat: Study

#64

Post by Cass »

As a doctor as opposed to an AI researcher, so many of the choices this study makes are baffling to me.

First of all, why just one heartbeat? You never capture just one heartbeat on an ECG anyway, and "Is the next heartbeat identical to the first one?" is such an important source of information, it seems completely irrational to exclude it. At least pick TWO heartbeats. If you're gonna pick one random heartbeat, how do you know you didn't pick an extra systole on accident? (Extra systoles look different, and often less healthy, than "normal" heart beats, as they originate from different regions of the heart.)

Secondly, why heart failure and not a heart attack? One definition of heart failure is "the heart is unable to pump sufficiently to maintain blood flow to meet the body's needs," which can be caused by all sorts of factors, many of them external to the actual function of the heart - do we even know for sure that there are ANY ECG changes definitely tied to heart failure? Why not instead try to detect heart attacks, which cause well-defined and well-researched known ECG changes?

(I realize AIs that claim to be able to detect heart attacks already exist. None of the ones I've personally worked with have ever been usable. The false positive rate is ridiculously high. I suppose maybe some research hospital somewhere has a working one?)

Re: AI Detects Heart Failure from One Heartbeat: Study

#65

Post by Cass »

Cass wrote: post #64

As a doctor as opposed to an AI researcher, so many of the choices this study makes are baffling to me. First of all, why just one heartbeat? You never capture just one heartbeat on an ECG anyway, and "Is the next heartbeat identical to the first one?" is such an important source of information, it seems completely irrational to exclude it. At least pick TWO heartbeats. If you're gonna pick one random heartbeat, how…

To add to this, looking at figure 4, why is their "average" heartbeat so messed up? That's not what a normal average heartbeat looks like. P is too flat, Q is too big, R is blunted, and there's an extra wave between S and T that's not supposed to be there at all. If their "healthy patient" ECGs were bad enough to produce this mess on average, it's no surprise their AI had no trouble telling the data sets apart.

(For comparison, the "CHF beat" looks a lot more like a healthy heartbeat.)

Re: AI Detects Heart Failure from One Heartbeat: Study

#66

Post by Camillo »

joker3 wrote: post #30
et2o wrote: post #5

I'm going to sound like a skeptical jerk here, but 490,000 heartbeats is how many patients? From what I recall these public ECG datasets are like 20 patients who underwent longitudinal ECGs. 500k heart beats is like 5 person-days of ECG recordings. Ninja Edit: N=~30 patients. For something like ECGs which are readily available, they really should have tried to get more patients. A single clinic anywhere does than 30…

A lot of machine learning people don't really understand study design or power or things like that. It's gotten a little better over the past decade or so, but this is an area where the field has a lot of room to improve.

"A lot" of ML people perhaps, but also the overwhelming majority of clinical scientists and the near totality of doctors.

Re: AI Detects Heart Failure from One Heartbeat: Study

#67

Post by braindeath »

Camillo wrote: post #66
joker3 wrote: post #30

Earlier quoted context omitted.

A lot of machine learning people don't really understand study design or power or things like that. It's gotten a little better over the past decade or so, but this is an area where the field has a lot of room to improve.

"A lot" of ML people perhaps, but also the overwhelming majority of clinical scientists and the near totality of doctors.

There are plenty of academic and academic center trained physicians that understand study design and are competent in research. They aren’t typically primary care/general practitioners so you just don’t encounter them as much. And yes they are the minority. But it’s not the totality.

Clinical research that isn’t making ridiculous claims tends to get much less press.

Furthermore, of all places to lap that crap up... this hacker news site is frankly one of the worst.

I mean look at this submission. Yes it’s true people are pillorying it here (including some doctors), but i don’t recall much interesting well designed medical research being discussed here (though arguably maybe not the place for it)

Re: AI Detects Heart Failure from One Heartbeat: Study

#68

Post by stevula »

3b18 wrote: post #50
Sanzig wrote: post #35

Earlier quoted context omitted.

Would you happen to have a source for that story? My workplace has really swallowed the AI Kool-Aid lately, so I would like to have some cautionary counterexamples to demonstrate potential pitfalls of the technology. It's got a lot of interesting applications for our field which I am excited about, but there seems to be a tendency among non-experts to consider it a magic bullet that can solve any sort of problem. In…

Check this link out http://132.206.230.229/e706/gaming.examples.in.AI.html There was a hn discussion about it a while back: https://news.ycombinator.com/item?id=18415031

Very amusing. Some of my favorites:

> In an artificial life simulation where survival required energy but giving birth had no energy cost, one species evolved a sedentary lifestyle that consisted mostly of mating in order to produce new children which could be eaten (or used as mates to produce more edible children).

> (Tetris) Agent pauses the game indefinitely to avoid losing

>Agent kills itself at the end of level 1 to avoid losing in level

>Robot hand pretending to grasp an object by moving between the camera and the object

Re: AI Detects Heart Failure from One Heartbeat: Study

#69

Post by jvm_ »

Sanzig wrote: post #35
corodra wrote: post #8

Wasn't a similar claim made about an AI detecting skin cancer from moles? Once the AI was deployed in the real world is failed miserably. I think it was a ton of false-positives because it was trained on images where cancerous moles all had images of rulers with them and the benign ones didn't have rulers in the image. So it just picked up on the ruler as a cancer indicator.

Would you happen to have a source for that story? My workplace has really swallowed the AI Kool-Aid lately, so I would like to have some cautionary counterexamples to demonstrate potential pitfalls of the technology. It's got a lot of interesting applications for our field which I am excited about, but there seems to be a tendency among non-experts to consider it a magic bullet that can solve any sort of problem. In…

Stolen from another HN comment about using AI on medical records.

"Finasteride is a compound that is used in two drugs. Proscar is used for prostate enlargement. It is old, out-of-patent, and has cheap generics. Propecia is used for hair loss. It is a newer, and (at the time) very expensive. The only difference is that Propecia is a lower-dose formulation.

What people did was to ask their doctors to perscribe generic Proscar, and then break the pills up to take for hair loss. Doctors would then justify the prescription by "diagnosing" enlarged prostate. This would enter the patient's health records. If you apply deep learning without being aware of this "trick", you would learn that a lot of young men have enlarged prostates, and that Proscar is an effective, well-tolerated treatment for it.

Health records are often political-economic documents rather than medical."

Post reply on HN