AI Detects Heart Failure from One Heartbeat: Study • Hacker News Board

Re: AI Detects Heart Failure from One Heartbeat: Study

#51

Post by [deleted] » Fri, Oct 18, 2019, 6:34 PM UTC

3b18 wrote: post #50

Sanzig wrote: post #35

Earlier quoted context omitted.

Would you happen to have a source for that story? My workplace has really swallowed the AI Kool-Aid lately, so I would like to have some cautionary counterexamples to demonstrate potential pitfalls of the technology. It's got a lot of interesting applications for our field which I am excited about, but there seems to be a tendency among non-experts to consider it a magic bullet that can solve any sort of problem. In…

Check this link out http://132.206.230.229/e706/gaming.examples.in.AI.html There was a hn discussion about it a while back: https://news.ycombinator.com/item?id=18415031

[deleted]

Re: AI Detects Heart Failure from One Heartbeat: Study

#52

Post by brilee » Fri, Oct 18, 2019, 6:43 PM UTC

carbocation wrote: post #42

brilee wrote: post #17

So first clarification is that heart failure != heart attack. Heart failure is a chronic condition where the heart is unable to pump hard enough to keep blood flowing through the body. Typically results in blood pooling in the leg, shortness of breath, etc. The study avoids the obvious pitfall, which is to put different slices of one patient's data into both training and test. The press also reports the training accu…

The study looks at 33 patients total, and the cases and controls come from entirely different data sets, with data coming from different devices that recorded signal at different frequencies. There is nothing to see here.

Apologies - I didn't catch that the HF / healthy patients are from two different datasets. Agreed that this essentially invalidates the result.

The missing experiment is to have a third dataset from yet another machine, with both positive/negative examples, and use it as the test dataset. Then transferability questions are at least somewhat addressed.

Re: AI Detects Heart Failure from One Heartbeat: Study

#53

Post by pkaye » Fri, Oct 18, 2019, 6:56 PM UTC

nradov wrote: post #28

Congestive heart failure can also be detected fairly reliably based on a sudden increase in weight. It causes fluid retention. There are several programs underway to give Internet connected scales to high risk patients and those report weight every day.

It could also be kidney failure. My weight ballooned by 30 lbs when I was progressing towards it. Even the doctor was saying the usual "you need to exercise and eat healthy" until he got my blood test results.

Re: AI Detects Heart Failure from One Heartbeat: Study

#54

Post by et2o » Fri, Oct 18, 2019, 7:34 PM UTC

paulcole wrote: post #21

et2o wrote: post #5

I'm going to sound like a skeptical jerk here, but 490,000 heartbeats is how many patients? From what I recall these public ECG datasets are like 20 patients who underwent longitudinal ECGs. 500k heart beats is like 5 person-days of ECG recordings. Ninja Edit: N=~30 patients. For something like ECGs which are readily available, they really should have tried to get more patients. A single clinic anywhere does than 30…

> I'm going to sound like a skeptical jerk here Well you're on the right site for it! At least you didn't claim that you could come up with something better by tinkering on a rainy Sunday afternoon.

Probably could though

Re: AI Detects Heart Failure from One Heartbeat: Study

#55

Post by gwern » Fri, Oct 18, 2019, 7:45 PM UTC

corodra wrote: post #8

Wasn't a similar claim made about an AI detecting skin cancer from moles? Once the AI was deployed in the real world is failed miserably. I think it was a ton of false-positives because it was trained on images where cancerous moles all had images of rulers with them and the benign ones didn't have rulers in the image. So it just picked up on the ruler as a cancer indicator.

> Once the AI was deployed in the real world is failed miserably.

They didn't show that.

Re: AI Detects Heart Failure from One Heartbeat: Study

#56

Post by VHRanger » Fri, Oct 18, 2019, 7:46 PM UTC

fencepost wrote: post #20

This is interesting, but more because it indicates that there's adequate data in a single heartbeat to do such diagnosis. In practical terms it's probably not nearly so relevant because it sounds like they were working with the raw data not tracing. By the time you have a patient hooked up to the proper equipment to do this diagnosis you're going to be getting adequate data anyway. The main impact might be that if th…

The study is not worth paying attention to.

The datasets for positive cases and negative cases come from different databases. n=30 patients, on top of it.

All this does is recognize the patient/ECG technician who recorded the data. It's basically certain it doesnt generalize

Re: AI Detects Heart Failure from One Heartbeat: Study

#57

Post by ryanmcbride » Fri, Oct 18, 2019, 8:28 PM UTC

Don't see any mention of how many false-positives they had in the article so... Yeah we'll see how effective this actually is.

Re: AI Detects Heart Failure from One Heartbeat: Study

#58

Post by kingkawn » Fri, Oct 18, 2019, 8:54 PM UTC

carbocation wrote: post #37

et2o wrote: post #5

I'm going to sound like a skeptical jerk here, but 490,000 heartbeats is how many patients? From what I recall these public ECG datasets are like 20 patients who underwent longitudinal ECGs. 500k heart beats is like 5 person-days of ECG recordings. Ninja Edit: N=~30 patients. For something like ECGs which are readily available, they really should have tried to get more patients. A single clinic anywhere does than 30…

Yes, Table 5 shows that N is 18 without CHF and 15 with CHF. These come from separate data sets that have EKG data sampled at different frequencies. Basically, they took 18 electrocardiographic tracings (sampled at 128 Hz) from participants without CHF, of whom 13 come from women. They compared them to 15 electrocardiographic tracings (sampled at 250 Hz) from participants with CHF, of whom 4 come from women. Hard to…

Statistically insignificant sample sizes showing desirable outcomes are the bread and butter of getting more funding in medical research

Re: AI Detects Heart Failure from One Heartbeat: Study

#59

Post by ryanschneider » Fri, Oct 18, 2019, 9:25 PM UTC

IMO, the important part is Section 3.3 of the [paper](https://www.sciencedirect.com/science/article/pii/S174680941...), particularly the image at [https://ars.els-cdn.com/content/image/1-s2.0-S17468094193017.... To my eye the difference in shape of the orange and green signals could also be found through more traditional signal processing/statistical means that machine learning.

In a past job I did a combination of manual and machine-learning-based analysis of cardiac signals. We didn't have ECG, but did have PPG (blood flow) and PCG (sound) signals, and a pretty large study group. I recall there being one study participant who's signals were very clearly indicative of heart failure, enough that we raised the issue with our medical advisor about whether the subject should be deanonymized and contacted. In the paper they state that "the CHF subjects used in this study suffer from severe CHF only"; my suspicion is that a simpler, "hand rolled" model based on the features of the ECG could compete very well with this CNN approach for finding the same level of pathology in the ECG signal, without the "black box" of a CNN casting doubt on the technique.

Re: AI Detects Heart Failure from One Heartbeat: Study

#60

Post by mlthoughts2018 » Fri, Oct 18, 2019, 9:36 PM UTC

joker3 wrote: post #30

et2o wrote: post #5

I'm going to sound like a skeptical jerk here, but 490,000 heartbeats is how many patients? From what I recall these public ECG datasets are like 20 patients who underwent longitudinal ECGs. 500k heart beats is like 5 person-days of ECG recordings. Ninja Edit: N=~30 patients. For something like ECGs which are readily available, they really should have tried to get more patients. A single clinic anywhere does than 30…

A lot of machine learning people don't really understand study design or power or things like that. It's gotten a little better over the past decade or so, but this is an area where the field has a lot of room to improve.

I disagree. Machine learning education almost always involves a lot of focus on design of experiments, causal inference, A/B testing and related topics.

I could agree with your claim if you meant bootcamp programs or data science sorts of coursework, but machine learning is generally grounded in both measure theoretic probability theory and a robust understanding of applied statistics before moving on. After that will be the basics of pattern classification, clustering, regression and dimensionality reduction. Last of all will be very domain-specific tools for NLP, computer vision, audio processing involving e.g. deep neural networks.