Live data from Hacker News

The Scientific Paper is Obsolete (2018)

theatlantic.com

91–100 of 175 posts

Re: The Scientific Paper is Obsolete (2018)

#91
While I agree with the premise of this article - that the scientific paper is obsolete - the proposed alternative does not address the primary inefficiencies with our current system. Dynamic 'papers' with illustrative examples are surely an improvement over current manuscripts. However, a flashy paper should be well down the list of goals for a research program. At the top of the list should be to make an important discovery or breakthrough.

Currently there is way too much trivial shit being published in the ever-expanding number of scientific journals.

Our current model for career advancement in academia is partially to blame for this. Landing a tenure track job requires having a prolific publication record. And once you've landed that coveted ladder-rank position, the pressure to publish only heats up, with tenure on the line. And then even after you've secured tenure, career advancement still depends heavily on the ol' publication record. Pre-tenure the pressure to publish in high-impact journals is immense; it's the kind of pressure that drives otherwise honest people to consider partaking in fraud (and unfortunately those who stick to their morals often lose out to those who fabricate results to some degree). Post-tenure the pressure changes from securing high-impact papers to just getting on whatever papers you can. Review boards for associate professors would more readily give a promotion to someone with 20 meaningless papers over the last three years than someone with 2 papers in CNS journals over the same timeframe, even though a single paper in Cell/Nature/Science is typically more impactful than 100 papers in Frontiers or other similarly dogshit journals. So post-tenure pay raises are based on getting as many words in print as possible, using the least amount of effort to do so.

I almost forgot where I was going with this... right, so, in my opinion we need to switch from a publication-based mindset to a discovery-based mindset. We (the public) provide the NIH with $30 billion dollars per year, with the idea that such an investment will lead to medical breakthroughs, discoveries, and other innovations that can concretely improve our health and wellbeing. However, so much of that is wasted on the idiosyncrasies of career advancement inside the ivory tower.

If I were the director of the NIH my sole purpose would be to end this nonsense. And my first order of action would be no longer accepting grant applications from individual PIs. I would only entertain grant applications from a small force of scientists (4-8 lab equivalents) with thorough and cogent plans for making breakthroughs on cancer or heart disease etc. I would change the minimum R01 funding amount from $10 million dollars. I would change the grant renewal timelines from every year to every 5 years but require yearly progress updates to ensure the proposed experiment were soundly conducted. I would encourage the equal reporting of both positive and null findings. Performance would not in any way be based on positive findings, only that the experiments were carefully run. I would require that all raw data be deposited into publicly accessible repos (not at the end of the study, not yearly, but whenever data is generated it should be made accessible asap). I would encourage that research groups bother with drafting/submitting interim manuscripts (interim meaning prior to completion of the full 5-10 year study) only if they find something important, otherwise just provide a comprehensive writeup at the end of the study. This final writeup would not be submitted to a 3rd party journal. It would be posted directly on the NIH website I'd have created for such results reporting. Naturally this would also be publicly accessible. That would be a start...

Re: The Scientific Paper is Obsolete (2018)

#92
This is a little exaggerated. Most papers have to be somewhat readable to be accepted into journals and notable conferences. The fact that the layman cannot understand an advanced biology paper is nothing new. I'd wager the scientific paper "golden age" the author cites as having such readable papers were not very readable for the general public of the time. It's just we are taught those things in elementary school and so can see the concepts in those older papers much better than the people of the time can.

Re: The Scientific Paper is Obsolete (2018)

#93

Earlier quoted context omitted.

Yes but chances are it only appears to work because the analysis code has bugs. So first i want to check the code and that it works before i put effort into understanding the idea.

So let me clarify. You're saying that the majority of CS papers only appear to work because the analysis code has bugs? And that checking the code (presumably also the analysis code) is easier than "understanding the idea"? Neither of those ring true to me, but your mileage may vary.

Yes, a significant proportion of all scientific research is not reproducible e.g., "Over half of psychology studies fail reproducibility test" https://www.nature.com/articles/nature.2015.18248 Cancer research: "scientific findings were confirmed in only 6 (11%) cases." https://www.nature.com/articles/483531a

Re: The Scientific Paper is Obsolete (2018)

#94
>The earliest papers were in some ways more readable than papers are today. They were less specialized, more direct, shorter, and far less formal. Calculus had only just been invented. Entire data sets could fit in a table on a single page. What little “computation” contributed to the results was done by hand and could be verified in the same way.

This is not even close to true. Look up Tycho Brahe's observations, and he's only the earliest I can think of.

Re: The Scientific Paper is Obsolete (2018)

#95

Earlier quoted context omitted.

I think this almost every time I read the paper. It’s like Linus’ “show me the code.” I just want papers now to “show me the data and the code.” And include a discussion about why these results are important. I think it’s a great time for the scientific community to improve transparency on these fronts. Sincerely, someone who reads a lot of research but contributes none because I’m an amateur. Edit: when I say data,…

I agree that they should ideally come with raw data along with all code that was used to process it to produce the results as presented. > but contributes none because I’m an amateur I don't mean to be rude but it seems relevant to point out. Papers aren't written for the benefit of amateurs. They're written for experts who actively work in that specific field. I don't think there's anything wrong with that.

Yeah I agree, but I read papers mainly in domains I do have university level degrees in. So while I’m not as expert as a lifetime professor, I do know the fields relatively well.

And I don’t think it’s rude, that’s why I included that statement!

Re: The Scientific Paper is Obsolete (2018)

#97

Earlier quoted context omitted.

I do share code that way, but the traditional ivory tower standards by which I am judged require "refereed journal publications" in high impact factor traditional journals. I'm trying to fight back against that, largely unsuccessfully. What would help me is to have the old geezers consider GitHub issues, PRs, and commits as a type of citation and to have a better way of tracking when my code gets used by others that…

> What would help me is to have the old geezers consider GitHub issues, PRs, and commits as a type of citation As a geezer myself I am imagining what this type of request would look like in less than 5 years from now. "What would help me is to have the old geezers consider Tiktok videos and replies as a type of citation" i.e. if you open this particular can of worms for a very restricted subset of users (not only pro…

A good youtube video easily can represent more work than a bad (published) paper, so why not.

Re: The Scientific Paper is Obsolete (2018)

#98

> The earliest papers were in some ways more readable than papers are today. They were less specialized, more direct, shorter, and far less formal. Calculus had only just been invented. Entire data sets could fit in a table on a single page. What little “computation” contributed to the results was done by hand and could be verified in the same way. This is not even close to true. Look up Tycho Brahe's observations, a…

in another way, if you read the early article in biology, they are very easy to read and funny! e.g., Wallace's article looks like travel note. but use them in research just a nightmare...

Re: The Scientific Paper is Obsolete (2018)

#99

As a practicing scientist, I firmly believe the world would be much better off if we simply published version-controlled Jupyter notebooks on a free site, such as GitHub or ArXiv.

So they would suffer the same decay all other web resources do? Broken links, no longer maintained tools, their opinionated choice (or lack of flexibility - which text + mathematical notation do have - if a set of stacks and data formats gets standardized). Nobody is preventing scientists from publishing code and data in addition to & before the paper, which imho itself should be as conservative in format as possible…

> So they would suffer the same decay all other web resources do? Broken links, no longer maintained tools, their opinionated choice (or lack of flexibility - which text + mathematical notation do have - if a set of stacks and data formats gets standardized).

Tools like Zenodo [1] are meant to solve this exact problem, and ensure these kinds of data don't suffer web decay.

[1] https://zenodo.org/

Re: The Scientific Paper is Obsolete (2018)

#100

While I agree with the premise of this article - that the scientific paper is obsolete - the proposed alternative does not address the primary inefficiencies with our current system. Dynamic 'papers' with illustrative examples are surely an improvement over current manuscripts. However, a flashy paper should be well down the list of goals for a research program. At the top of the list should be to make an important d…

if you can magically predict result of research project, this will work!
Post reply on HN