Hack Your Way to Scientific Glory (2016)

projects.fivethirtyeight.com

11–12 of 12 posts

Re: Hack Your Way to Scientific Glory (2016)

#11

Post by nextos » Mon, Jul 01, 2024, 6:00 PM UTC

wjb3 wrote: post #10

nextos wrote: post #9

Earlier quoted context omitted.

eLife is very rigorous, but also very selective. Most submissions don't pass editorial consideration. I've found their reviews much more detailed and technical than Nature Medicine or Cell.

When you say that eLife is "very rigorous," could you elaborate on what specific aspects of their review process or criteria you find particularly stringent, and how those compare to the review processes of Nature/Cell?

eLife publishes reviews very prominently within articles, and they ask all reviewers to discuss their assessment after they have evaluated the article. I think this favors deep reviews.

In Nature or Cell, I have sometimes seen 2-liners sent to authors as a review, or reviews where the reviewer clearly read the article for just 10 minutes before rushing an opinion.

I wish eLife doubled down on this and disclosed the reviewer names after e.g. a 2-year embargo. This would make reviews even better IMHO, more skin in the game.

Re: Hack Your Way to Scientific Glory (2016)

#12

Post by kjkjadksj » Tue, Jul 02, 2024, 4:07 AM UTC

caddemon wrote: post #6

paulpauper wrote: post #4

Sigh. P hacking has long been a meme. Nicholas Cage and shark attacks for example. P values are useful though if the results seem at least plausible or can be explained by some underlying process. Like a low p value between calorie restriction and weight loss.

Shark attacks isn't really p-hacking though, that's about misinterpreting/misrepresenting a real correlation. P-hacking is finding a fake correlation due to chance because you tested a lot of different things in a smallish dataset, and then reporting that correlation as meaningful without mentioning your negative results or properly explaining your entire process.

This is what I don’t fully grasp about p hacking myself. Say you test variable abc and so on all the way to z. Variable z were found to be significant. Now imagine you had the same data set and just happened to test Z and stopped there, finding it significant.

According to the p hacking concept the former would be potentially unethical while the latter would be great science, however nothing about the underlying data or its association between variables has changed.

Post reply on HN