Lesson of ivermectin: meta-analyses based on summary data alone are unreliable
1–10 of 493 posts
Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable
#2I've suspected that for years. I remember my undergrad biology prof coming down on students depending on meta-analysis and insisting they were no substitute for attempting to replicate the experiments themselves.
Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable
#3Yes the quality of scientific research in this area has been rather bad with rushes to publish and ineffective peer review. Drs. Paul Marik and Pierre Kory recently updated their meta analysis to remove the retracted Elgazzar paper, and still found a significant effect. I'm not endorsing their letter, just pointing out that it exists and isn't mentioned in this Nature article.
https://journals.lww.com/americantherapeutics/fulltext/2021/...
Ultimately we need a real large scale controlled trial to settle the issue so I'm looking forward to seeing results from NIH ACTIV-6.
https://www.nih.gov/research-training/medical-research-initi...
Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable
#4I've suspected that for years. I remember my undergrad biology prof coming down on students depending on meta-analysis and insisting they were no substitute for attempting to replicate the experiments themselves.
that doesn't make scientific sense. Replicating the original experiment and doing a meta-analysis have two completely different goals. I would have argued that back to the professor (and been right).
Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable
#5Mind that this is correspondence, not a peer reviewed paper.
For what is worth most meta-study will check if they reach the same results leaving-n out (typically one), but I agree that they could do much better accessing the underlying data itself.
Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable
#6Yes the quality of scientific research in this area has been rather bad with rushes to publish and ineffective peer review. Drs. Paul Marik and Pierre Kory recently updated their meta analysis to remove the retracted Elgazzar paper, and still found a significant effect. I'm not endorsing their letter, just pointing out that it exists and isn't mentioned in this Nature article. https://journals.lww.com/americantherape…
That meta-analysis still includes the Carvallo studies, which some people have doubts about: https://www.buzzfeednews.com/article/stephaniemlee/ivermecti...
Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable
#7Yes the quality of scientific research in this area has been rather bad with rushes to publish and ineffective peer review. Drs. Paul Marik and Pierre Kory recently updated their meta analysis to remove the retracted Elgazzar paper, and still found a significant effect. I'm not endorsing their letter, just pointing out that it exists and isn't mentioned in this Nature article. https://journals.lww.com/americantherape…
This article also says that another trial that were included in previous meta-analyses should probably also be excluded and once all that's it's unlikely the meta-analysis will show an effect:
As these two papers1,6 were the only studies included in that meta-analysis to demonstrate an independently significant reduction in mortality, the revision will probably show no mortality benefit for ivermectin.
Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable
#8Yes the quality of scientific research in this area has been rather bad with rushes to publish and ineffective peer review. Drs. Paul Marik and Pierre Kory recently updated their meta analysis to remove the retracted Elgazzar paper, and still found a significant effect. I'm not endorsing their letter, just pointing out that it exists and isn't mentioned in this Nature article. https://journals.lww.com/americantherape…
This letter opens with a defense of the ElGazzar paper, but the analyses published by these researchers strongly suggests that the paper is indefensible and likely a simple fraud. I'm skeptical.
Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable
#9Because there are so many studies of ivermectin and because it is so widely used, of course there are going to be bunk studies and scientifically invalid studies. Do you know how I know this? I know this because it was recently shown (within the past ten years) that something like 70% of scientific studies can't be reproduced (read: they're BS). So if were going to use scientific studies at all, to inform our decisions, as a society, then we have to accept that there's going to be flaws in some of them. And if we know that there are flaws in some of them, we can also conclude that a large quantity of corroborating studies is as important as the scale of each individual study. The same applies to meta-analyses. For example, it's going to be pretty difficult to publish over 100 studies[1] that all corroborate the almost-same set of basic facts, and then have all +100 of them be completely invalid. However, having over 100 different studies of a 50-year old medicine, from different scientists in different parts of the world based on different data sets over different spans of time and under different circumstances, and having the data corroborate the same key facts and numbers, I would trust this any day over a government funded study of a brand new medicine that is in a brand new category of medicines which has never been used in history.
There are literally hundreds of studies at this point proving that ivermectin is safe and effective for treating many things, including influenza and coronavirus, including COVID-19.
There are literally no studies proving that the Pfizer or Moderna gene therapies are safe and effective, or even that they are safe or effective, at both treating and preventing the spread of COVID-19. The latter of the two is the most important in the debate over whether people should be forced to take said medicines. And it is the inverse of that which has been proven through many studies now.
Re: Lesson of ivermectin: meta-analyses based on summary data alone are unreliable
#10Yes the quality of scientific research in this area has been rather bad with rushes to publish and ineffective peer review. Drs. Paul Marik and Pierre Kory recently updated their meta analysis to remove the retracted Elgazzar paper, and still found a significant effect. I'm not endorsing their letter, just pointing out that it exists and isn't mentioned in this Nature article. https://journals.lww.com/americantherape…
Looking at the two papers (original, and updated) they only removed the Elgazzer paper but not the contested Shakhsi Niaee also mentioned in the Nature article.
This is noteworthy because it's also the only other RCT to show strong effect on mortality (see Fig3 in the two papers).
Original Marik, Kory paper Fig3 is here: https://pubmed.ncbi.nlm.nih.gov/34375047/#&gid=article-figur...
Also, the Nature article is concerned that the underlying data isn't sound due to bad randomization and thus naive meta-analysis is inherently flawed in these cases.