Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

nature.com

Post reply on HN
Print view View original

31–40 of 493 posts

Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

#31

Post by nradov »

Spellman wrote: post #10
nradov wrote: post #3

Yes the quality of scientific research in this area has been rather bad with rushes to publish and ineffective peer review. Drs. Paul Marik and Pierre Kory recently updated their meta analysis to remove the retracted Elgazzar paper, and still found a significant effect. I'm not endorsing their letter, just pointing out that it exists and isn't mentioned in this Nature article. https://journals.lww.com/americantherape…

Looking at the two papers (original, and updated) they only removed the Elgazzer paper but not the contested Shakhsi Niaee also mentioned in the Nature article. This is noteworthy because it's also the only other RCT to show strong effect on mortality (see Fig3 in the two papers). Original Marik, Kory paper Fig3 is here: https://pubmed.ncbi.nlm.nih.gov/34375047/#&gid=article-figur... Also, the Nature article is conce…

Sure that's a good point. But is mortality the appropriate clinical trial endpoint? Mortality rates are already low when following the NIH treatment guidelines, so it would take a large study population to discern any statistically significant effects.

https://www.covid19treatmentguidelines.nih.gov/

In addition to mortality it probably makes sense to look at other endpoints such as patient reported symptoms, RT PCR test cycle count, and time to hospital discharge. Those should allow us to tell if there is a real effect (or not) with smaller subject groups.

Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

#33

Post by droptablemain »

programmarchy wrote: post #15

How does the current field of studies compare to Remdesivir? Remdesivir is a repurposed antiviral (originally designed for Ebola) with fairly low efficacy, and very severe side effects, and it breezed through FDA emergency authorization. I don't understand why it seems that the bar is set so low for Remdesivir, but so high for Ivermectin.

I think you answered your own question -- Remdesivir is an antiviral and Ivermectin is an antiparasite.

Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

#35

Post by gambler »

nostoc wrote: post #16
gambler wrote: post #12

So, they demand to change an established scientific practice because they don't like the results it produced in a particular case. And then everyone wonders why people no longer "trust the science".

Is meta analysis of unpublished result really such an established scientific practice?

From the article you're commenting on:

"We recognize that this is a change to long-accepted practice and is substantially more rigorous than the standards that are typically currently applied"

Meta-analyses generally follow certain pre-established standards. That's the whole point of such studies.

It's one thing to claim that authors of a particular meta-study broke the standards. This is not being claimed here. It's an entirely different thing to claim that the standards themselves are broken and need to be changed generally. This is a huge claim that casts doubt on tons of prior research.

Maybe we do need more rigorous rules for meta-analysis. However, if that's the case, it should be true for all research of this type, not just papers examining "interventions for COVID-19" (as the article suggests).

Hence, my take on this. It's reasonable to question scientific standards if you are willing to demonstrate that they are really broken. (This was done with p-hacking, for example.) It's not reasonable to say that a specific subject in research should have different evidentiary standards because of controversy around one drug.

Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

#36

Post by shadowgovt »

rory wrote: post #25

Ivermectin probably doesn't do anything to fight Covid-19, but the American media has really shown their corruption in the way they've treated it. It's clearly a safe drug to take in human-designed doses, and it's cheap to produce. Laughing at people for poisoning themselves with "horse dewormer" instead of pointing out that they are turning to the vet store because their access to medicine has been marginalized is s…

How many users have been taking it in human-designed doses, given that it's not generally prescribed for COVID-19 (and therefore those who are taking it for that purpose are mostly self-medicating)?

Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

#37

Post by dekhn »

gorwell wrote: post #11
dekhn wrote: post #4

Earlier quoted context omitted.

that doesn't make scientific sense. Replicating the original experiment and doing a meta-analysis have two completely different goals. I would have argued that back to the professor (and been right).

How useful is meta analysis if half or more of the experiments don't replicate? As we increasingly discover with the replication crisis.

Medical studies can't be replicated in experiments. They are almost always one-shot. That's not how medical studies (that we do meta analysis) work. It's too expensive and too complicated.

Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

#38

Post by buescher »

The lesson of ivermectin: a lot of medical studies that go uncriticized because they're less controversial probably have problems similar to the problems of the retracted Elgazzar study and the other studies mentioned. Why else would these people think they would get away with it?

Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

#39

Post by greenonions »

motoboi wrote: post #32

Someone said it and I totally agree: Society is not ready to watch science in realtime.

I agree, though there are two obvious courses of action that result from this:

1. Prevent the viewing 2. Increase the readiness

Of which I think #2 is most preferable.

Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

#40

Post by simonh »

eigengrau5150 wrote: post #2

I've suspected that for years. I remember my undergrad biology prof coming down on students depending on meta-analysis and insisting they were no substitute for attempting to replicate the experiments themselves.

Ok you replicate the original study. Presumably you want to use this new result to increase your statistical power compared to the original result, combining the results of the two studies.... which... is.. er... meta analysis.

The whole point of meta analysis is that you have multiple studies of the same experiment.

Post reply on HN