Live data from Hacker News

The Scientific Paper is Obsolete (2018)

theatlantic.com

141–150 of 175 posts

Re: The Scientific Paper is Obsolete (2018)

#141
post #123

Earlier quoted context omitted.

I think I can convince you otherwise. If I publish a paper saying I have an algorithm which can factor large composites, and in the paper publish the factors to all of the RSA numbers listed at https://en.wikipedia.org/wiki/RSA_Factoring_Challenge , then I think people will take it seriously, and not consider it at the superficial level. Even if I don't publish the algorithm. ("Because of the security implications of…

Yes, there are occasional exceptions where you don't have to repeat or replicate the experiments reported in a paper to verify them. But that is very much the exception. Generally you are expected to explain what you did in enough detail that the reader can replicate your experiment. If you're fitting a protein model to X-ray diffraction data, you aren't expected to include all the other protein models you considered…

>Yes, there are occasional exceptions where you don't have to repeat or replicate the experiments reported in a paper to verify them. But that is very much the exception.

And even in this exceptional case, the algorithm itself is interesting above and beyond the fact of its existence.

Re: The Scientific Paper is Obsolete (2018)

#142
The paper as way to publish information is rooted in a world that no longer exists. That world consisted of scientists accessing information in printed form through libraries that subscribed to relevant magazines, journals, and what not from publishers. That's still a thing as far as publishers are concerned but I rarely set foot in a library after Google became a thing last century.

The last thirty years have changed that game to basically no longer involve printing (other than for a very few select publications) and basically switching to a digital only publishing form.

I completed my phd, read thousands of papers (well skimmed mostly, it's a fine art to zoom in on the relevant stuff), all without visiting the library more than once or twice. I only printed the ones actually worth reading in detail. And these days I consume vast amounts of information on screen without ever using a printer. My printer is fifteen years old and I just installed the second toner cartridge I ever bought for it.

Yet we still pretend to have "journals" like we're in the 19th century. It's the equivalent to writing your friend a letter to inform them that your train is delayed by 5 minutes. Most sane people use some kind of instant messaging tool for that. Writing letters of course used to be a primary way to communicate for scientists. That too has stopped being a thing. People use email now.

The whole point of publishing is to convey information in a form that's convenient to the reader and to solicit endorsement from your peers (via peer review). Peer review used to be implied by virtue of an editor choosing to select a certain paper for publishing. That in turn implies they would have consulted a number of peers about the suitability of that paper. It's sort of the super tedious equivalent of a soliciting a thumbs up button in a social network.

If you publish on linkedin because you are some kind of wannabe influencer you basically need to get people to 1) read your stuff and 2) click the like or share button. Scientific publishing basically is not that different. You have wannabe scientist that want to get the attention of the influencers (reputable peers) so people will be convinced they know their shit. This ultimately translates into degrees, research funding, and tenure track positions. The whole process is kind of biased towards metrics because that's how universities choose to allocate their money.

An ambitious scientist behaves basically in a similar way as a linkedin influencer and will try to game the system by flooding the system with a lot of content and getting their buddies to sign off on it. There are a lot of mediocre articles that get published in obscure places with cliques of scientists basically doing each other favors by referencing each other's work; or worse self referencing. In linkedin terms, this would be the absolute drivel that nobody likes that gets re-shared by a few people that also don't manage to produce much content of interest.

So, here's a thought, maybe get this a bit more out in the open and give scientists some modern tools to endorse each other's work. The best endorsement is a reference. A link basically. Tracking links between bits of paper is super tedious. These papers need permanent URLs. And they need to be digitally signed by their authors so we can have some authenticity and prevent cheating. And scientists need a place where scientists can debate and exchange thoughts about these papers. That used to be a big tradition between scientist back when they still wrote letters to each other or used journals to criticize each other's work.

Curating and aggregating work by means of linking to it is a job that should not be reserved for fussy editors of non paper based journals that absolutely nobody ever reads cover to cover. HN for science; why not? Why not have a multitude of websites referring, editorializing and commenting on published work? How is that not a thing?

Re: The Scientific Paper is Obsolete (2018)

#143

Earlier quoted context omitted.

The code is there to show that the idea is worth the discussion around.

Would the idea be valuable without the code? Yes.

We can only possibly gain from publishing the code, and lose by not publishing.

It's not like it takes a whole lot of time to just dump your code in a github repo once you're done and link it somewhere on the paper (if you wrote code at all while working on the paper).

Sometimes I did just want to run my own experiments with different datasets, and those algorithms aren't always trivial to implement :|

Re: The Scientific Paper is Obsolete (2018)

#144
Computational notebooks are great, but only seeing what the author (or coder) wanted you to see is not enough to evaluate their work. In addition to data and code, we need to see the path they took, the exploration and experimentation that led to the final presentation of ideas in the notebook.

For this we can use cloud based environments controlled by funding agencies/universities that ensure every interaction with data is recorded from the very beginning.

Something like this would at least reduce the risk of p-hacking practices that would otherwise be there even if everyone used notebooks instead of papers.

Re: The Scientific Paper is Obsolete (2018)

#145

Earlier quoted context omitted.

The code is there to show that the idea is worth the discussion around.

Would the idea be valuable without the code? Yes.

Facts and proofs are more interesting that unproven (and sometimes unprovable) ideas.

Re: The Scientific Paper is Obsolete (2018)

#146

In my field, at least, I think the problem is less about the medium, and more about the incentives. Researchers are incentivized to write papers that seem impressive (and intimidating) rather than clear and intuitive. To make matters worse, this is an evolved trait: researchers whose papers are intimidating are more likely to succeed, which means they're more likely to have future PhD students, which means that the s…

> Researchers are incentivized to write papers that seem impressive (and intimidating) rather than clear and intuitive. Ah, a fellow economist lol. Lack of clarity is a strategic advantage because (1) (as you said) it looks impressive and (2) it's hard to validate that it's correct. So many papers contain such elementary statistics mistakes such as survivorship bias, e.g. 'returns to education' is almost exclusively…

> Ah, a fellow economist lol. Lack of clarity is a strategic advantage because (1) (as you said) it looks impressive and (2) it's hard to validate that it's correct.

Reminds me of the old "there are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies."

Re: The Scientific Paper is Obsolete (2018)

#147

Earlier quoted context omitted.

I think I can convince you otherwise. If I publish a paper saying I have an algorithm which can factor large composites, and in the paper publish the factors to all of the RSA numbers listed at https://en.wikipedia.org/wiki/RSA_Factoring_Challenge , then I think people will take it seriously, and not consider it at the superficial level. Even if I don't publish the algorithm. ("Because of the security implications of…

> If I publish a paper saying I have an algorithm which can factor large composites, and in the paper publish the factors to all of the RSA numbers If some factors of those numbers are also large composites, without access to a good algorithm, nobody can truly verify your claims. If not and you include all of those factors in an easily digestible way for computers to process (let's call that "code"), it will be easy…

It's a lot easier to check if something is a prime than to factor it.

Re: The Scientific Paper is Obsolete (2018)

#148

Earlier quoted context omitted.

> I just want papers now to “show me the data and the code.” But the code is secondary to the idea. The idea and the discussion around how it was arrived at and what it means is the key thing. The code is just there to implement it. You could code the same idea ten different ways.

I translated what you said in the context of mathematics: “But the proof is secondary to theorem. The theorem and the discussion around how it was arrived at and what it means is the key thing. The proof is just there to show it’s true. You could write a proof for the same theorem ten different ways.” Which is all true. But man it wastes so much time having to re-prove everything. Also some lemmas/theorems are so har…

Ok but can you understand other fields are different to mathematics? In other fields we argue more informally.

Re: The Scientific Paper is Obsolete (2018)

#149
post #145

Earlier quoted context omitted.

Would the idea be valuable without the code? Yes.

Facts and proofs are more interesting that unproven (and sometimes unprovable) ideas.

If nobody ever shared useful but unprovable ideas we'd worse off.

Re: The Scientific Paper is Obsolete (2018)

#150
post #66

I think this is a great idea theoretically, but in reality for most papers I don't want to see the data/underlying code. While it would be great to publish data/code with the paper (in the field I've worked on the most, astronomy, most data is already published with the paper anyways), I don't want/need to look through a notebook with the underlying code of the paper in order to just read the intro/conclusions (and m…

EMBL-EBI and others had some RDF-related effort to provide machine readable abstracts, which I thought was a really cool idea. IMHO, the biggest problem with papers is politics and reviews. In many top journals like Nature there's no double-blind review (actually in Nature it's now optional but big groups never use it). And even if there was double-blind review, referees have no skin in the game. So the usual outcome…

> referees have no skin in the game

That's a big problem. I just got a paper rejected. Reviewer 1 was just focalized on a single detail I mentioned somewhere, not central at all in the paper yet is basing most on this criticism around that. Reviewer 2 has difficulty understanding a table containing 2 columns and 3 rows, and what means N, V, ADJ and ADV in a paper about dictionary (not to mention the same abbreviation was used just before, and used a plain words numerous times in multiple paragraphs). Reviewer 3 is the only one saying a remotely nice thing and who seems to have grasped what the paper is about. There is of course some valid criticism raised in the reviews, but half of it is bullshit that would be dispelled in a more interactive process or/and if reviewers had incentives to actually put a minimal effort to understand the paper.

Post reply on HN