Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Scientific Paper is Obsolete (2018)

theatlantic.com

Post reply on HN
Print view View original

71–80 of 175 posts

Re: The Scientific Paper is Obsolete (2018)

#71

Post by Terr_ »

BeetleB wrote: post #58
multilogit39 wrote: post #43

As an academically employed scientist of 20 years, the notion that scientific communication suddenly needs better standards puzzles me. The core research curriculum of nearly every scientific field I’ve seen, STEM or otherwise, is that the data needed for replication are non-negotiable. A paper that doesn’t include it would be table rejected by any editor. Or one would hope. This is taught at the UNDERgraduate level,…

> The core research curriculum of nearly every scientific field I’ve seen, STEM or otherwise, is that the data needed for replication are non-negotiable. A paper that doesn’t include it would be table rejected by any editor. Or one would hope. This is taught at the UNDERgraduate level, for heaven’s sake. This may vary based on discipline, but in both the subdisciplines of experimental and theoretical physics I was in…

Sounds like there's a need for some sort of time-based escrow on "the full paper"... Of course, game-theory strikes again because nobody will want to pay for it.

Re: The Scientific Paper is Obsolete (2018)

#72

Post by aiphex »

BeetleB wrote: post #58
multilogit39 wrote: post #43

As an academically employed scientist of 20 years, the notion that scientific communication suddenly needs better standards puzzles me. The core research curriculum of nearly every scientific field I’ve seen, STEM or otherwise, is that the data needed for replication are non-negotiable. A paper that doesn’t include it would be table rejected by any editor. Or one would hope. This is taught at the UNDERgraduate level,…

> The core research curriculum of nearly every scientific field I’ve seen, STEM or otherwise, is that the data needed for replication are non-negotiable. A paper that doesn’t include it would be table rejected by any editor. Or one would hope. This is taught at the UNDERgraduate level, for heaven’s sake. This may vary based on discipline, but in both the subdisciplines of experimental and theoretical physics I was in…

If you are working in the same niche, you will likely know the tricks of the trade. You usually have an idea of what your peers are working on and it's often a race to see who gets a paper out first. This is what conferences are for as well, to try and figure out what your peers are up to.

Re: The Scientific Paper is Obsolete (2018)

#73

Post by AOsborn »

BeetleB wrote: post #58
multilogit39 wrote: post #43

As an academically employed scientist of 20 years, the notion that scientific communication suddenly needs better standards puzzles me. The core research curriculum of nearly every scientific field I’ve seen, STEM or otherwise, is that the data needed for replication are non-negotiable. A paper that doesn’t include it would be table rejected by any editor. Or one would hope. This is taught at the UNDERgraduate level,…

> The core research curriculum of nearly every scientific field I’ve seen, STEM or otherwise, is that the data needed for replication are non-negotiable. A paper that doesn’t include it would be table rejected by any editor. Or one would hope. This is taught at the UNDERgraduate level, for heaven’s sake. This may vary based on discipline, but in both the subdisciplines of experimental and theoretical physics I was in…

Serious question: if the experiment can't be reproduced to be verified, how does the paper provide more proof than a blanket 'trust me'?

From someone who isn't an academic, isn't this letting politics come before science?

Re: The Scientific Paper is Obsolete (2018)

#74

Post by haihaibye »

PhilipVinc wrote: post #2

Papers today are longer than ever and full of jargon and symbols. They depend on chains of computer programs that generate data, and clean up data, and plot data, and run statistical models on data. These programs tend to be both so sloppily written and so central to the results that it’s contributed to a replication crisis, or put another way, a failure of the paper to perform its most basic task: to report what you…

One of the first suggestions I have is to use source control and store the Git hash of code used to generate data. A few times I've heard back "we don't have time for that" - pretty easy to see how the replication crisis flows from processes like that.

Re: The Scientific Paper is Obsolete (2018)

#75

Post by [deleted] »

diognesofsinope wrote: post #69
williamkuszmaul wrote: post #67

In my field, at least, I think the problem is less about the medium, and more about the incentives. Researchers are incentivized to write papers that seem impressive (and intimidating) rather than clear and intuitive. To make matters worse, this is an evolved trait: researchers whose papers are intimidating are more likely to succeed, which means they're more likely to have future PhD students, which means that the s…

> Researchers are incentivized to write papers that seem impressive (and intimidating) rather than clear and intuitive. Ah, a fellow economist lol. Lack of clarity is a strategic advantage because (1) (as you said) it looks impressive and (2) it's hard to validate that it's correct. So many papers contain such elementary statistics mistakes such as survivorship bias, e.g. 'returns to education' is almost exclusively…

[deleted]

Re: The Scientific Paper is Obsolete (2018)

#76

Post by derekpankaew »

williamkuszmaul wrote: post #67

In my field, at least, I think the problem is less about the medium, and more about the incentives. Researchers are incentivized to write papers that seem impressive (and intimidating) rather than clear and intuitive. To make matters worse, this is an evolved trait: researchers whose papers are intimidating are more likely to succeed, which means they're more likely to have future PhD students, which means that the s…

I hope the "trend" of researchers writing for the public, via books & blog posts, can contribute some incentive towards being clear and intuitive.

I'm thinking about the success of Freakonomics, Thinking Fast and Slow, The Elegant Universe, etc. These are all academics, who've "translated" their research for the masses. That translation ended up being much more impactful - and prestigious - than an intimidating paper.

I hope this becomes more of a trend, and an incentive structure, in the future.

Re: The Scientific Paper is Obsolete (2018)

#77

Post by bloaf »

sam-2727 wrote: post #25

I think this is a great idea theoretically, but in reality for most papers I don't want to see the data/underlying code. While it would be great to publish data/code with the paper (in the field I've worked on the most, astronomy, most data is already published with the paper anyways), I don't want/need to look through a notebook with the underlying code of the paper in order to just read the intro/conclusions (and m…

You should always want to have the underlying code available. Without the exact procedures they used to process their data, the only kind of "using their conclusions" you can do is the superficial "take it at face value" kind. So many important details get hand-waved away in papers that say things like "we used the well known blahblahblah method to analyze the data."

If you do it right, the code should in no way interfere with your ability to read abstracts.

Re: The Scientific Paper is Obsolete (2018)

#78

Post by mattkrause »

AshamedCaptain wrote: post #17
queuebert wrote: post #11

Earlier quoted context omitted.

I do share code that way, but the traditional ivory tower standards by which I am judged require "refereed journal publications" in high impact factor traditional journals. I'm trying to fight back against that, largely unsuccessfully. What would help me is to have the old geezers consider GitHub issues, PRs, and commits as a type of citation and to have a better way of tracking when my code gets used by others that…

> What would help me is to have the old geezers consider GitHub issues, PRs, and commits as a type of citation As a geezer myself I am imagining what this type of request would look like in less than 5 years from now. "What would help me is to have the old geezers consider Tiktok videos and replies as a type of citation" i.e. if you open this particular can of worms for a very restricted subset of users (not only pro…

So what?

Papers qua papers aren't the goal. The idea is to advance our collective understanding of a field. Papers are certainly a means to that end, but other things can too, like code, videos, and blogposts, even if they don't fit into the "6000 words and 6 figures" box.

I get that citations and citation metrics feel objective, but they emphatically aren't great measures of research/researcher "quality".

Re: The Scientific Paper is Obsolete (2018)

#79

Post by queuebert »

Mezzie wrote: post #27
queuebert wrote: post #11

Earlier quoted context omitted.

I do share code that way, but the traditional ivory tower standards by which I am judged require "refereed journal publications" in high impact factor traditional journals. I'm trying to fight back against that, largely unsuccessfully. What would help me is to have the old geezers consider GitHub issues, PRs, and commits as a type of citation and to have a better way of tracking when my code gets used by others that…

> I also think citations of your work that find errors or correct things should count as a negative citation. Strong disagree. Given how much influence colleagues can have over one another's career prospects, how petty academic disagreements can get, admin focus on metrics like citation count, and how it's easier to prove someone else wrong than to do your own original work (both have value, one is just easier), it w…

You make a good point, and maybe there is a mathematical solution, like netting out mutually negative citations.

Re: The Scientific Paper is Obsolete (2018)

#80

Post by queuebert »

fennecfoxen wrote: post #23
AshamedCaptain wrote: post #17

Earlier quoted context omitted.

> What would help me is to have the old geezers consider GitHub issues, PRs, and commits as a type of citation As a geezer myself I am imagining what this type of request would look like in less than 5 years from now. "What would help me is to have the old geezers consider Tiktok videos and replies as a type of citation" i.e. if you open this particular can of worms for a very restricted subset of users (not only pro…

Tiktok videos are primarily used for entertainment, rather unlike Jupyter notebooks and source code repositories. Surely you have a more serious objection.

You don't read git repos for entertainment?!
Post reply on HN