Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When they warn of rare disorders, these prenatal tests are usually wrong

nytimes.com

Post reply on HN
Print view View original

51–60 of 176 posts

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#51

Post by halpert »

adjkant wrote: post #48
halpert wrote: post #40

Earlier quoted context omitted.

It's true they are focusing on false positives, but the authors are using the ratio of false positives to true positives to paint a picture that the tests are inaccurate, when in reality the tests are accurate. What this article is looking at is called the "sensitivity" of a test: https://en.wikipedia.org/wiki/Sensitivity_and_specificity

While the author may not be well versed or focusing on the stats side, you're missing the human side here I think. > the tests are inaccurate, when in reality the tests are accurate If the test make someone consider terminating a pregnancy or even considering it, that's a lot of pain. So for that human, the test is failing its purpose potentially, depending on the value calculation of terminating a viable pregnancy v…

Or maybe you’re missing the human side of having a child born with a serious genetic defect?

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#52

Post by raymondh »

siganakis wrote: post #26

My wife and I went through this a couple of years ago, with a 10 week NIPT calling a rare trisomy (chr 9), which is always fatal within a few weeks of birth. It was absolute hell. The key problem here is the waiting and uncertainty. You have the NIPT at 10w, but you can’t have the amniocentesis until several weeks later. When that came back fine, there were questions about whether it was a “mosaic” meaning only a sma…

Thank you for sharing this.

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#53

Post by andreilys »

adjkant wrote: post #48
halpert wrote: post #40

Earlier quoted context omitted.

It's true they are focusing on false positives, but the authors are using the ratio of false positives to true positives to paint a picture that the tests are inaccurate, when in reality the tests are accurate. What this article is looking at is called the "sensitivity" of a test: https://en.wikipedia.org/wiki/Sensitivity_and_specificity

While the author may not be well versed or focusing on the stats side, you're missing the human side here I think. > the tests are inaccurate, when in reality the tests are accurate If the test make someone consider terminating a pregnancy or even considering it, that's a lot of pain. So for that human, the test is failing its purpose potentially, depending on the value calculation of terminating a viable pregnancy v…

Usefulness and helpfulness are far better metrics, and such a high false positive rate is clearly causing issues in respect to those

How exactly do you plan on codifying usefulness and helpfulness?

A high false positive rate is not necessarily a bad thing and may instead be the catalyst for additional tests to confirm the first one. The tests accuracy may actually be 100%, which is great because it avoids a child being born with a fatal genetic disease. Would you prefer a high false negative rate that misses these diseases instead?

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#54

Post by wizee »

halpert wrote: post #40
ellisv wrote: post #36

Earlier quoted context omitted.

I disagree. It is clear from the title, “When They Warn of Rare Disorders, These Prenatal Tests Are Usually Wrong”, and the lead that they’re focusing on false positives.

It's true they are focusing on false positives, but the authors are using the ratio of false positives to true positives to paint a picture that the tests are inaccurate, when in reality the tests are accurate. What this article is looking at is called the "sensitivity" of a test: https://en.wikipedia.org/wiki/Sensitivity_and_specificity

The issue is that the tests portray themselves as being accurate (in the sense of low false positive rates), and portray the result as “your baby has XYZ rare syndrome” instead of “your baby has a 15% change of having XYZ rare syndrome”. If the test providers stated the false positive rate for their results more clearly, parents would be in a better position to make informed decisions.

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#55

Post by tambeb »

A tweet about this very article caught my eye yesterday, and I'm glad HN's taken notice too.

https://twitter.com/JohnFPfaff/status/1477382805583716353?t=...

'For a disease w a 1-in-20,000 risk, a test w a false positive rate of 1% and a false negative rate of 0%—an insanely accurate test—would identify 1 correct case and 200 false positives every time. Or would be wrong 99.5% of the time.

This isn’t “bad tests.” This is… baserates.'

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#56

Post by andreilys »

isoprophlex wrote: post #41
halpert wrote: post #30

Earlier quoted context omitted.

Their graphics say the tests are “84% wrong.” Do you really feel that’s an accurate description? That doesn’t feel like an accurate description to me, and their usage of “wrong” in this context highlights that they don’t understand the distinction and importance of true positives, false positives, true negatives, and false negatives when measuring accuracy.

Going through something like this is very VERY stressful. When you get a negative you immediately forget about it. When you get a positive you die inside. Speaking from experience here. 84% wrong sounds, to me, as an accurate description. Experiencing this from the inside out, only the false/true positive ratio matters. (Given sufficiently low false negative rates, of course) 84% of people whose world is turned upsid…

You’re talking about precision (true positive / true positive + false negative) but that’s only one part of the story.

There is a real human cost to having a child born with a rare genetic disease (and I would argue is immensely more stressful). You can easily adjust the sensitivity to the test but at the cost of detecting actual true positive cases. The correct response to receiving a positive is to do another test to ensure it’s not a false positive.

To say 84% wrong is clickbait and used to elicit a legislative response (FDA regulation), which will help the reporters career.

The actual ratio to tell if something is “wrong” is accuracy (True positive + true negative) / (true positive + true negative + false positive + false negative)

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#57

Post by andreilys »

SpicyLemonZest wrote: post #33
halpert wrote: post #30

Earlier quoted context omitted.

Their graphics say the tests are “84% wrong.” Do you really feel that’s an accurate description? That doesn’t feel like an accurate description to me, and their usage of “wrong” in this context highlights that they don’t understand the distinction and importance of true positives, false positives, true negatives, and false negatives when measuring accuracy.

I really feel it's an accurate description. If you get a positive result on the test, there's a 16% chance your fetus has a 1p36 deletion and an 84% chance they don't.

Ignoring all the true and false negatives which themselves are markers of how accurate the test is.

16% precision is the correct statement, saying the test is wrong 84% of the time implies that those getting negative results might actually have positive results.

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#58

Post by creata »

gwern wrote: post #9

This article is a confused mess. It's something of a Gish gallop in conflating all the different issues they could come up with, while leaving out all the necessary vocabulary (C-f "Bayes" "posterior" "decision theory" [Phrase not found]) making it almost impossible to consider each issue in adequate detail. It mixes up poor communication (reporting false-positive/negative rates as if posterior probabilities, & exagg…

> implicit calls for the FDA to Do Something and ban the tests

Not that you're necessarily wrong, but how did you get that from the article? It didn't seem to me like they wanted a ban.

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#59

Post by divbzero »

Isn’t that often true with screens in general? The threshold often allows a good number of false positives in order to minimize false negatives. The goal is to know when to seek further diagnostics. Communicating that to patients can be a challenge but it doesn’t mean the screens were designed incorrectly.

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#60

Post by robbedpeter »

andreilys wrote: post #57
SpicyLemonZest wrote: post #33

Earlier quoted context omitted.

I really feel it's an accurate description. If you get a positive result on the test, there's a 16% chance your fetus has a 1p36 deletion and an 84% chance they don't.

Ignoring all the true and false negatives which themselves are markers of how accurate the test is. 16% precision is the correct statement, saying the test is wrong 84% of the time implies that those getting negative results might actually have positive results.

He framed his statement correctly, limiting his observation to the condition that the test returned a positive result. Saying that 84% of positive results are false is correct if only 16% are true. You'd need to know false negative rates and base occurrence rates (modified by whatever other factors are unique to your situation) to inform the nature of information you get by performing the test.
Post reply on HN