Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When they warn of rare disorders, these prenatal tests are usually wrong

nytimes.com

Post reply on HN
Print view View original

71–80 of 176 posts

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#71

Post by rflrob »

wizee wrote: post #54
halpert wrote: post #40

Earlier quoted context omitted.

It's true they are focusing on false positives, but the authors are using the ratio of false positives to true positives to paint a picture that the tests are inaccurate, when in reality the tests are accurate. What this article is looking at is called the "sensitivity" of a test: https://en.wikipedia.org/wiki/Sensitivity_and_specificity

The issue is that the tests portray themselves as being accurate (in the sense of low false positive rates), and portray the result as “your baby has XYZ rare syndrome” instead of “your baby has a 15% change of having XYZ rare syndrome”. If the test providers stated the false positive rate for their results more clearly, parents would be in a better position to make informed decisions.

The larger issue as I see it is that the medical system around these screenings are not well versed in the statistics and able to communicate that to patients. "Eight [patients] said they never received any information about the possibility of a false positive, and five recalled that their doctor treated the test results as definitive." It's hard to know what happened in the room when the doctor spoke with them or what was on those particular patients tests, and that's (one hopes) the worst medical news those people will receive for a long time so listening comprehension is understandably impaired, but there needs to someone available who can help them interpret, even days or weeks later, and these people were let down by the entire system, not just the test manufacturers.

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#72

Post by loeg »

mcguire wrote: post #67
halpert wrote: post #51

Earlier quoted context omitted.

Or maybe you’re missing the human side of having a child born with a serious genetic defect?

Is it better to terminate 85 pregnancies which do not have a serious defect in order to catch 15 which do? At what point is it not better to terminate 100% of pregnancies?

> Is it better to terminate 85 pregnancies which do not have a serious defect in order to catch 15 which do?

Yes, it’s absolutely better to do that. Of course, the actual ratio is much better than that because we do follow-up tests after the screen.

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#73

Post by paulryanrogers »

mcguire wrote: post #67
halpert wrote: post #51

Earlier quoted context omitted.

Or maybe you’re missing the human side of having a child born with a serious genetic defect?

Is it better to terminate 85 pregnancies which do not have a serious defect in order to catch 15 which do? At what point is it not better to terminate 100% of pregnancies?

> At what point is it not better to terminate 100% of pregnancies?

Everyone should decide for themselves. Having seen the long term consequences I would rather err on the side of caution, even if it were difficult to become pregnant.

Such diseases are often incurable and significantly degrade the quality of life of not only the person to be born but the whole immediate family. At least in the US the there isn't enough social safety net or support too offset the crushing costs.

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#74

Post by jrockway »

mcguire wrote: post #65
gwern wrote: post #9

This article is a confused mess. It's something of a Gish gallop in conflating all the different issues they could come up with, while leaving out all the necessary vocabulary (C-f "Bayes" "posterior" "decision theory" [Phrase not found]) making it almost impossible to consider each issue in adequate detail. It mixes up poor communication (reporting false-positive/negative rates as if posterior probabilities, & exagg…

So you are saying the testing companies in the article aren't fraudulently claiming much more effective tests than they are providing?

Specificity and sensitivity are two dimensions that you can measure tests in. You can claim your test is 99% accurate if you mean that "if the test says you don't have the disease, there is a 99% chance that you don't have the disease". That same test can still be 85% wrong if it says you DO have the disease, though.

I doubt that hyping one side of this equation is fraud. Pushing the error in this direction seems like a good idea, anyway. If you have some weird illness, and the test comes back as a false positive, at least you'll continue to explore that possibility for a while. If it comes back as a false negative, then you'll spend a ton of time exploring alternatives which will be true negatives. Probably infuriating.

https://en.wikipedia.org/wiki/Sensitivity_and_specificity

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#75

Post by halpert »

mnw21cam wrote: post #61
halpert wrote: post #40

Earlier quoted context omitted.

It's true they are focusing on false positives, but the authors are using the ratio of false positives to true positives to paint a picture that the tests are inaccurate, when in reality the tests are accurate. What this article is looking at is called the "sensitivity" of a test: https://en.wikipedia.org/wiki/Sensitivity_and_specificity

No, the article isn't talking about sensitivity. We don't actually know what the sensitivity is from the data the article gives us. We are told that lots of people were screened and a small number had a positive result, of which a proportion were actually positive. You can't calculate sensitivity from that because you don't know how many actually positive cases were missed. This article is talking about precision, wh…

Yes you are correct.

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#76

Post by nohuck13 »

hn_throwaway_99 wrote: post #16
gwern wrote: post #9

This article is a confused mess. It's something of a Gish gallop in conflating all the different issues they could come up with, while leaving out all the necessary vocabulary (C-f "Bayes" "posterior" "decision theory" [Phrase not found]) making it almost impossible to consider each issue in adequate detail. It mixes up poor communication (reporting false-positive/negative rates as if posterior probabilities, & exagg…

> implicit calls for the FDA to Do Something and ban the tests (not an iota of cost-benefit considered nor any self-reflection about whether we want the FDA involved in anything at all these days)... This is true in so many areas of journalism but lately seems especially egregious in the NYT. And I don't really blame them, as the incentives for any individual reporter are just too great - having the government make a…

"the incentives for any individual reporter are just too great - having the government make a major policy change based on your article is basically the brass ring for an investigative reporter"

Yep, this is the framing I came here looking for.

Investigative journalists live in the same asymmetrically-incentivized world as social science researchers. If the reporter had looked into the phenomenon and concluded "yeah, boring technical logic pretty much works as expected here" then there's no story.

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#77

Post by taeric »

This is an example of a problem that is so hard to explain. The vast majority of folks getting these tests will get a true negative. Such that for most people, this is not an issue. So I get that it takes effort to make people care.

That said, I do feel that pulling in abortions to the debate is specifically to trigger a set of readers. But to what aim? They have not established that the tests could be better. Just that when they say yes, they are still not perfect.

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#78

Post by tmnstr85 »

My 2nd daughter was flagged during our 20 week for something having to do with the way her skull was forming and they wanted to do a series of genetic test. They charged us through the wazoo and everything came back negative. She arrived 3.5 weeks early and contracted bacterial meningitis shortly after birth. We found her code blue in the crib. She ended up having a bilateral craniotomy to relieve the empyema that had formed. CP, CVI, global TBI - every day is hell on earth. This was 2019, so the nightmare of the last few years started early for our family. We've had a number of medical professionals drop hints at the fact there might be something wrong from a rare disorder perspective but we're in a league of our own and that is hindsight - the damage and trauma are non-stop. Anyone trying to shickle a few dollars from the medical system to provide "pre-natal diagnosis" without sound science - they can come burn in the same hell I live in every day.

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#79

Post by rflrob »

There's a lot of sibling comments going on about whether the value they're looking at is the right one. What the Times is showing as their headline number is Positive Predictive Value (True positive/(TP+FP)), which depends on the prevalence in the population. The "methods section" here is a little vague, but given the low prevalence I'm willing to accept on face value that it's basically accurate (i.e. that it's not assuming that the families getting these tests are not orders of magnitude more likely to be positive for these diseases). If the test result truly said one patient's 'daughter had a “greater than 99/100” probability of being born with Patau syndrome', then that's concerning, but given the fairly narrow quotes around the number, I'd suspect that what is actually on the test result is not inconsistent with the fairly low PPV on these screens.

Re: When they warn of rare disorders, these prenatal tests are usually wrong

#80

Post by [deleted] »

gwern wrote: post #9

This article is a confused mess. It's something of a Gish gallop in conflating all the different issues they could come up with, while leaving out all the necessary vocabulary (C-f "Bayes" "posterior" "decision theory" [Phrase not found]) making it almost impossible to consider each issue in adequate detail. It mixes up poor communication (reporting false-positive/negative rates as if posterior probabilities, & exagg…

[deleted]
Post reply on HN