Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redesigning the Scientific Paper

theatlantic.com

Post reply on HN
Print view View original

51–60 of 110 posts

Re: Redesigning the Scientific Paper

#51

Post by aplorbust »

"... the skill most in demand among physicists, biologists, chemists, geologists, even anthropologists and research psychologists, is facility with programming languages and "data science" packages."

If I wanted to prove to someone this statement was true, what would be the most effective way to do that?

Is author basing this conclusion on job postings somewhere?

Has he interviewed anyone working in these fields?

Has he worked in a lab or for a company doing R&D?

How does he know?

What evidence (cf. media hype) could I cite in order to convince someone he is right?

When I look at the other articles he has written, they seem focused on popularised notions about computers, but I do not see any articles about the academic disciplines he mentions.

Re: Redesigning the Scientific Paper

#52

Post by tomkinstinch »

JorgeGT wrote: post #39

> How to integrate billions of base pairs of genomic data, and 10 times that amount of proteomic data, and historical patient data, and the results of pharmacological screens into a coherent account of how somebody got sick and what to do to make them better? How to make actionable an endless stream of new temperature and precipitation data, and oceanographic and volcanic and seismic data? How to build, and make sens…

A group upstairs at the Broad Institute built out a system to use Jupyter notebooks for analysis of genomic data, with backend computation happening on a Spark cluster[1]. Science on large datasets can happen via interactive notebook. In a connection with a recent GWAS on a massive dataset from the UK Biobank, the researchers involved decided not to write a traditional scientific journal article (at least for now) since their analyses will continue to mature. Instead, they've been posting insights online in blog form, with associated code on GitHub[2]. It's a daring move toward publishing at the speed of research. Once their conclusions mature, traditional journal articles may follow to distill and preserve the key findings. In the mean time, those in the field can apply the same code to their data, replicate the analyses, and get an early look at the output of the research. This works partially because the methods (univariate GWAS) are understood in the field and the interpretation and rendering of a particular dataset is the science in this case, rather than a new method (which would still likely warrant a paper).

1. https://hail.is

2. http://www.nealelab.is/blog/2017/7/19/rapid-gwas-of-thousand...

Re: Redesigning the Scientific Paper

#53

Post by chr1 »

bjourne wrote: post #21
jfaucett wrote: post #9

> These programs tend to be both so sloppily written and so central to the results that it’s contributed to a replication crisis, or put another way, a failure of the paper to perform its most basic task: to report what you’ve actually discovered, clearly enough that someone else can discover it for themselves. This is the crux of the of the problem IMHO - at least for the fields I study (AI/ML). Replicating the resu…

I've had a paper peer reviewed. It was ultimately rejected but I can't help but suspect that by making all my code publicly available, I hurt my chances of publication. The reviewers comments were about my coding style, my choice of build tool (I didn't use make, but something else which is just as easy to use), the choice of C vs C++... It's like best practices for computer security -- always strive to minimize the…

Then do we even need peer review? In my experience it is always superficial, people just feel that they have to say something, so they say something about writing style, or similar trivia.

The way it should work is you put your stuff with code and all data on github. People interested in the field or working for journals read it, and rate it, journals collect links to paper repositories that are highly rated by scientists who have many highly rated papers in the field, and call that publication.

Re: Redesigning the Scientific Paper

#54

Post by chr1 »

mschuetz wrote: post #28
bjourne wrote: post #21

Earlier quoted context omitted.

I've had a paper peer reviewed. It was ultimately rejected but I can't help but suspect that by making all my code publicly available, I hurt my chances of publication. The reviewers comments were about my coding style, my choice of build tool (I didn't use make, but something else which is just as easy to use), the choice of C vs C++... It's like best practices for computer security -- always strive to minimize the…

> It's like best practices for computer security -- always strive to minimize the attack surface. I suspect that's also why some papers are unnecesarely verbose and describe simple things as complicated as possible. Can't criticize something that can't be understood.

It's unfair that your comment is downvoted because it's spot on.

Hiding code, obfuscating language, fudging data, all are symptoms of the same problem: of being interested in getting paper on cv instead of doing research.

There are many circumstances that can put even a good scientist in a situation where he/she has to do this but that's not a good argument for not sharing the code.

Re: Redesigning the Scientific Paper

#56

Post by RBerenguel »

chr1 wrote: post #53
bjourne wrote: post #21

Earlier quoted context omitted.

I've had a paper peer reviewed. It was ultimately rejected but I can't help but suspect that by making all my code publicly available, I hurt my chances of publication. The reviewers comments were about my coding style, my choice of build tool (I didn't use make, but something else which is just as easy to use), the choice of C vs C++... It's like best practices for computer security -- always strive to minimize the…

Then do we even need peer review? In my experience it is always superficial, people just feel that they have to say something, so they say something about writing style, or similar trivia. The way it should work is you put your stuff with code and all data on github. People interested in the field or working for journals read it, and rate it, journals collect links to paper repositories that are highly rated by scien…

I've been peer reviewed once (and waiting for the second) and it was very in depth, giving me a couple of pointers to improve my paper. Field was mathematics, though.

Re: Redesigning the Scientific Paper

#57

Post by RangerScience »

jfaucett wrote: post #9

> These programs tend to be both so sloppily written and so central to the results that it’s contributed to a replication crisis, or put another way, a failure of the paper to perform its most basic task: to report what you’ve actually discovered, clearly enough that someone else can discover it for themselves. This is the crux of the of the problem IMHO - at least for the fields I study (AI/ML). Replicating the resu…

Seems like this could pair well with the journal crisis and suggestions to implement a blockchain journal: Your paper cannot be accepted by the journal unless it includes executable code; the results of which are then injected into the "paper" view...?

So, basically - a paper consists of what it takes to replicate the paper, and the blockchain journal's first step is running the replication.

This would be problematic for papers that require expensive computation, however...

Re: Redesigning the Scientific Paper

#58

Post by JorgeGT »

tomkinstinch wrote: post #52
JorgeGT wrote: post #39

> How to integrate billions of base pairs of genomic data, and 10 times that amount of proteomic data, and historical patient data, and the results of pharmacological screens into a coherent account of how somebody got sick and what to do to make them better? How to make actionable an endless stream of new temperature and precipitation data, and oceanographic and volcanic and seismic data? How to build, and make sens…

A group upstairs at the Broad Institute built out a system to use Jupyter notebooks for analysis of genomic data, with backend computation happening on a Spark cluster[1]. Science on large datasets can happen via interactive notebook. In a connection with a recent GWAS on a massive dataset from the UK Biobank, the researchers involved decided not to write a traditional scientific journal article (at least for now) si…

> Science on large datasets can happen via interactive notebook.

I did not claim the opposite, just that it regularly happens without interactive notebooks. This seems like an interesting project though. Regarding the blog posts, it seems that there's a bug that makes all the entries appear as published on September 20, 2017?

Re: Redesigning the Scientific Paper

#59

Post by contravariant »

dsacco wrote: post #45
contravariant wrote: post #20

Earlier quoted context omitted.

I assume you don't just means using Github for version control of scientific papers, because that sounds pretty pointless to me. As far as tracking incremental improvements over time goes, I think it'll be hard to do better than our current method of including references to papers. It's impossible to track ideas the same way tracking code works (which itself is limited for similar reasons). It would be nice if you co…

> I assume you don't just means using Github for version control of scientific papers, because that sounds pretty pointless to me. Why? I recently wrote a paper on GitHub. I loved it.

I mean sure use whatever you like, but I'm just not sure how much use it would be for people interested in reading your paper.

Re: Redesigning the Scientific Paper

#60

Post by dsacco »

contravariant wrote: post #59
dsacco wrote: post #45

Earlier quoted context omitted.

> I assume you don't just means using Github for version control of scientific papers, because that sounds pretty pointless to me. Why? I recently wrote a paper on GitHub. I loved it.

I mean sure use whatever you like, but I'm just not sure how much use it would be for people interested in reading your paper.

Oh I follow. No, I meant I like using git for version control.
Post reply on HN