Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

nature.com

Post reply on HN
Print view View original

11–20 of 493 posts

Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

#11

Post by gorwell »

dekhn wrote: post #4
eigengrau5150 wrote: post #2

I've suspected that for years. I remember my undergrad biology prof coming down on students depending on meta-analysis and insisting they were no substitute for attempting to replicate the experiments themselves.

that doesn't make scientific sense. Replicating the original experiment and doing a meta-analysis have two completely different goals. I would have argued that back to the professor (and been right).

How useful is meta analysis if half or more of the experiments don't replicate? As we increasingly discover with the replication crisis.

Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

#13

Post by dreyfan »

gonational wrote: post #9

Because there are so many studies of ivermectin and because it is so widely used, of course there are going to be bunk studies and scientifically invalid studies. Do you know how I know this? I know this because it was recently shown (within the past ten years) that something like 70% of scientific studies can't be reproduced (read: they're BS). So if were going to use scientific studies at all, to inform our decisio…

I think you’re late for your chiropractor appointment.

Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

#14

Post by marcodiego »

nradov wrote: post #3

Yes the quality of scientific research in this area has been rather bad with rushes to publish and ineffective peer review. Drs. Paul Marik and Pierre Kory recently updated their meta analysis to remove the retracted Elgazzar paper, and still found a significant effect. I'm not endorsing their letter, just pointing out that it exists and isn't mentioned in this Nature article. https://journals.lww.com/americantherape…

Can we infer that since "we need a real large scale controlled trial to settle the issue", the positive effect (if any) is necessarily small?

Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

#15

Post by programmarchy »

How does the current field of studies compare to Remdesivir? Remdesivir is a repurposed antiviral (originally designed for Ebola) with fairly low efficacy, and very severe side effects, and it breezed through FDA emergency authorization. I don't understand why it seems that the bar is set so low for Remdesivir, but so high for Ivermectin.

Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

#16

Post by nostoc »

gambler wrote: post #12

So, they demand to change an established scientific practice because they don't like the results it produced in a particular case. And then everyone wonders why people no longer "trust the science".

Is meta analysis of unpublished result really such an established scientific practice?

Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

#17

Post by ajross »

nradov wrote: post #3

Yes the quality of scientific research in this area has been rather bad with rushes to publish and ineffective peer review. Drs. Paul Marik and Pierre Kory recently updated their meta analysis to remove the retracted Elgazzar paper, and still found a significant effect. I'm not endorsing their letter, just pointing out that it exists and isn't mentioned in this Nature article. https://journals.lww.com/americantherape…

> Yes the quality of scientific research in this area has been rather bad with rushes to publish and ineffective peer review

Well, sure, but what else is new? The Ivermectin paper was surely bad, but it wasn't outrageously bad nor unexpectedly so. The mania that resulted isn't, at it's core, about bad science. This won't fix the problem.

Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

#19

Post by tptacek »

nostoc wrote: post #16
gambler wrote: post #12

So, they demand to change an established scientific practice because they don't like the results it produced in a particular case. And then everyone wonders why people no longer "trust the science".

Is meta analysis of unpublished result really such an established scientific practice?

I think you're dignifying this comment, which objects to the authors request that meta-studies not include fraudulent analyses in their inputs as an unwarranted "change to established practice".

Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable

#20

Post by marcodiego »

dekhn wrote: post #4
eigengrau5150 wrote: post #2

I've suspected that for years. I remember my undergrad biology prof coming down on students depending on meta-analysis and insisting they were no substitute for attempting to replicate the experiments themselves.

that doesn't make scientific sense. Replicating the original experiment and doing a meta-analysis have two completely different goals. I would have argued that back to the professor (and been right).

Hmmm... How about meta-analysis considering only results that were successfully replicated?

Why not consider an index that is higher with the number of successful replications and lower of number of replication tries?

Post reply on HN