Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Study: No evidence of efficacy of hydroxychloroquine in hospitalized patients

medrxiv.org

Post reply on HN
Print view View original

331–340 of 379 posts

Re: Study: No evidence of efficacy of hydroxychloroquine in hospitalized patients

#331

Post by tptacek »

kazinator wrote: post #328
tptacek wrote: post #101

Earlier quoted context omitted.

That's kind of a ghoulish sentiment to attribute to someone else.

I don't believe that the sentence pattern which I used attributes sentiments to others. "You want ..." functions as as rhetorical idiom. "You want X or else Y" roughly means "Unless X, Y, thus you will be objectively disadvantaged". The sentiment, if there is one, remains mine; on the other hand, this "ghoulish" is in fact an external interpretation of sentiment being ascribed to me.

I don't know you enough to call you "ghoulish" and wouldn't be inclined to, but the sentiment itself hopes that a treatment for an illness killing tens of thousands of people per week will fail in order to secure supplies for a chronic but nonfatal ailment. I feel like the absolute morality of that sentiment is something we can actually evaluate.

(It's all hypothetical, of course, because in fact HCQ does not appear to be effective for C19).

Re: Study: No evidence of efficacy of hydroxychloroquine in hospitalized patients

#332

Post by rodonn »

dwaltrip wrote: post #253
thedudeabides5 wrote: post #75

Earlier quoted context omitted.

This is a great comment thanks. Title of the paper could (with a tongue firmly in cheek) be called, "Weak evidence for effectiveness of hydroxychloroquine but our sample was likely too small for it to pass a T-test"

It is not evidence (weak or otherwise) for the effectiveness of hydroxychloroquine. Your version is misleading.

It is weak evidence for the effectiveness of hydroxychloroquine. e.g. if you ran this as a Bayesian analysis you would find that these results should shift your beliefs towards it being more likely that hydroxychloroquine reduces deaths + ICU visits.

However you are correct that the medical community has decided to be conservative to reduce the likelihood people make decisions based on weak evidence. That may be the correct decision from a overall social welfare perspective. In that framework all we can say is "this experiment did not generate enough evidence to reject the null hypothesis".

Re: Study: No evidence of efficacy of hydroxychloroquine in hospitalized patients

#333

Post by capnrefsmmat »

URSpider94 wrote: post #195
capnrefsmmat wrote: post #49

It's important to interpret these results carefully: "no evidence of efficacy" does not imply evidence of no efficacy, and in fact the uncertainty in this study is quite large. The sample size is small enough that it would be very hard for this study to detect the effect of hydroxychloroquine unless that effect is very, very large. From the abstract: > In the HCQ group, 2.8% of the patients died within 7 days vs 4.6%…

This is not the bar that's typically applied for drug trials. The goal of a drug trial is not to prove beyond a shadow of a doubt that it DOESN'T work. The goal is to prove to a high level of confidence that it DOES work. For example, it's highly unlikely (roughly less than 2.5%) that HCQ reduces the onset of ARDS by half. If it ultimately turns out that HCQ reduces the rate of death by 5%, we'd have to do a tremendo…

To be clear, this claim

> For example, it's highly unlikely (roughly less than 2.5%) that HCQ reduces the onset of ARDS by half.

is not one that can be justified by frequentist statistics such as those used in the article. We can't make probabilistic claims like that unless we go Bayesian.

Well-designed drug trials typically plan in advance to have the sample size necessary to detect an effect of clinical significance. If, say, a 20% reduction in ARDS would be valuable, you would work out what sample size you'd need to reliably detect such a reduction. In this case, we simply don't have the sample size to conclude much of anything, because the confidence intervals include what I (as a non-physician, admittedly) see as a pretty wide range of clinically meaningful effect sizes, and we can't distinguish between them.

Re: Study: No evidence of efficacy of hydroxychloroquine in hospitalized patients

#334

Post by downerending »

DoreenMichele wrote: post #323
downerending wrote: post #298

Earlier quoted context omitted.

Yeah, I think we're all pretty unnerved. And we're far from knowing just how deep this rabbit hole is going to be. (Or how shallow.) If you look at the charts of per-capita deaths in various countries, there's not much of a pattern, in terms of places we'd think of as having "good" government and/or public health versus "bad". Belgium, for example, is the worst right now (or among them), and many European countries a…

Sadly, people who get it right are frequently remembered as "lucky" after the fact or even sometimes get maligned in some manner. When I took archery in college, I was the only one in class who could consistently hit the bullseye. I also practiced up to two hours a day. A classmate who didn't practice, who had terrible form and was holding the bow all wrong and who couldn't manage to hit the target at all snidely inf…

Agree.

In general, I'm not particularly a fan of Trump, but I don't buy gibe that he's a lucky idiot. Nobody's luck is that good.

As to whether he's helped or hurt in this crisis, I hope someone evenhanded like Ken Burns will pick it apart later.

Re: Study: No evidence of efficacy of hydroxychloroquine in hospitalized patients

#335

Post by emn13 »

lonelappde wrote: post #74
blakesterz wrote: post #69

Earlier quoted context omitted.

Would it be the same thing to do 1,000 small studies like this one as doing one LARGE study with 1000x more people? Would the results be considered just as useful?

Similar. That's called a "meta analysis" and tends to happen after many independent uncoordinated small studies have published.

Also, it's extremely hard to account for all kinds of biases and systematic errors in a meta-analysis. I'd take most meta-analyses, even by renowned statiticians, very, very skeptically. Teasing out small effects from samples that need corrections for lots of confounding factors, and then trying do combine many such results is next to impossible.

If your effect is small enough that you need a meta-analysis in the first place, that's a warning sign.

Some background: https://journals.plos.org/plosone/article?id=10.1371/journal... (with perhaps more approachable explanation here https://www.talyarkoni.org/blog/2016/06/11/the-great-minds-j... )

Obviously it's sometimes possible to do this right, but the reproducibiility crisis didn't appear out of nowhere; it's best to be wary of overly clever statistics; hard to know what you don't know.

Re: Study: No evidence of efficacy of hydroxychloroquine in hospitalized patients

#336

Post by capnrefsmmat »

asdfasgasdgasdg wrote: post #102
capnrefsmmat wrote: post #49

It's important to interpret these results carefully: "no evidence of efficacy" does not imply evidence of no efficacy, and in fact the uncertainty in this study is quite large. The sample size is small enough that it would be very hard for this study to detect the effect of hydroxychloroquine unless that effect is very, very large. From the abstract: > In the HCQ group, 2.8% of the patients died within 7 days vs 4.6%…

> "no evidence of efficacy" does not imply evidence of no efficacy That's incorrect. Absence of evidence is evidence of absence. It's just not proof of absence. See https://www.lesswrong.com/posts/mnS2WYLCGJP2kQkRn/absence-of...

I do not think this argument is relevant to estimating effect sizes, rather than binary outcomes as the Less Wrong post does.

If the claim had been "no patients treated hydroxychloroquine recover", and the study found no patients that recovered, the argument would apply: there's no evidence that patients recover, which starts to suggest that indeed, patients do not recover.

But here we are interested in estimating the size of the effect, which can take any continuous value. The statistical result is that the relative risk of death is, with 95% confidence, somewhere between 0.13 and 2.89. You can't interpret that as evidence relative risk is exactly 1 (equal risk with or without the drug) -- if you could, you could also take it as evidence the relative risk is 1.2, or 2.7, or 0.75.

Re: Study: No evidence of efficacy of hydroxychloroquine in hospitalized patients

#337

Post by rodonn »

asdfasgasdgasdg wrote: post #102
capnrefsmmat wrote: post #49

It's important to interpret these results carefully: "no evidence of efficacy" does not imply evidence of no efficacy, and in fact the uncertainty in this study is quite large. The sample size is small enough that it would be very hard for this study to detect the effect of hydroxychloroquine unless that effect is very, very large. From the abstract: > In the HCQ group, 2.8% of the patients died within 7 days vs 4.6%…

> "no evidence of efficacy" does not imply evidence of no efficacy That's incorrect. Absence of evidence is evidence of absence. It's just not proof of absence. See https://www.lesswrong.com/posts/mnS2WYLCGJP2kQkRn/absence-of...

In this case it really is not evidence of an absence of an effect. In fact it should increase your belief that the treatment has an effect (e.g. if you did a Bayesian analysis), since the treated group did have lower death and ICU usage than the untreated. However this evidence does not rise to the 1% or 5% that we traditionally use, so it's fair to conclude that we don't know if the drug works or not.

This experiment is so small that it had virtually no chance of ever being able to detect an effect on the probability of death (what statisticians call the "statistical power" of the experiment is only 10% whereas a well designed study should aim for power above 80%). Even if the drug cured 100% of people it was given to, a study this small would not be able to statistically conclude whether the drug works. Since the baseline death rate is only 4%, it takes fairly large samples to be able to detect any change. A 200 person experiment has basically no hope of being able to detect a difference in the death endpoint (there is a bit more hope for the more common ICU endpoint).

Re: Study: No evidence of efficacy of hydroxychloroquine in hospitalized patients

#338

Post by kazinator »

tptacek wrote: post #331
kazinator wrote: post #328

Earlier quoted context omitted.

I don't believe that the sentence pattern which I used attributes sentiments to others. "You want ..." functions as as rhetorical idiom. "You want X or else Y" roughly means "Unless X, Y, thus you will be objectively disadvantaged". The sentiment, if there is one, remains mine; on the other hand, this "ghoulish" is in fact an external interpretation of sentiment being ascribed to me.

I don't know you enough to call you "ghoulish" and wouldn't be inclined to, but the sentiment itself hopes that a treatment for an illness killing tens of thousands of people per week will fail in order to secure supplies for a chronic but nonfatal ailment. I feel like the absolute morality of that sentiment is something we can actually evaluate. (It's all hypothetical, of course, because in fact HCQ does not appear…

Have you considered that I might be thinking of the problem that it will be hoarded up by some individuals who have neither the illness nor the ailment? Few who needs it for any reason will be able to get their hands on it. That's my thinking, more or less.

Re: Study: No evidence of efficacy of hydroxychloroquine in hospitalized patients

#339

Post by jeffdavis »

mcguire wrote: post #99
jeffdavis wrote: post #62

Given the study's small sample size, it seems the only way it would have produced a conclusive result is if the effect size was huge (i.e. miracle cure). The study found some evidence of efficacy, but it was not significant given the sample size. A better headline would say something like: "study finds that HCQ is not a miracle cure for already-hospitalized covid-19 patients".

How much are you willing to pay for "not a miracle cure" on the basis of "eh, it can't hurt"?

Your point doesn't really make sense, because the purpose of a study like this is not for making treatment decisions; it's for making research decisions.

I don't see anything in the study that discredits HCQ as a potential treatment, and therefore it makes sense to keep researching.

Re: Study: No evidence of efficacy of hydroxychloroquine in hospitalized patients

#340

Post by randcraw »

jeffdavis wrote: post #62

Given the study's small sample size, it seems the only way it would have produced a conclusive result is if the effect size was huge (i.e. miracle cure). The study found some evidence of efficacy, but it was not significant given the sample size. A better headline would say something like: "study finds that HCQ is not a miracle cure for already-hospitalized covid-19 patients".

No. This study found no evidence of effectiveness as a treatment for COVID-19 in patients who need it.

A sample size of 20 is sufficient to satisfy a .05 statistical significance threshold that hydrochloroquine had even the slightest positive effect. In this population, it did not.

All therapeutic drugs are developed using in vitro and pre-clinical (non human animal) treatment groups no bigger than 20. When a well controlled and randomized study comparng two groups that are each four times larger than 20, we have a VERY strong indication that this drug delivers not only too little effect to be a useful treatment for COVID-19, but it very likely delivered absolutely NO EFFECT AT ALL in hospitalized patients. Don't expect any reputable physician or scientist to dismiss this evidence.

This is very bad news for the prospect for treating patients who need it using this drug -- those people who are likeliest to die or endure lasting harm. The other 80% of positive patients don't need any therapy except time.

Post reply on HN