Live data from Hacker News

The Scientific Paper is Obsolete (2018)

theatlantic.com

61–70 of 175 posts

Re: The Scientific Paper is Obsolete (2018)

#61

I think this is a great idea theoretically, but in reality for most papers I don't want to see the data/underlying code. While it would be great to publish data/code with the paper (in the field I've worked on the most, astronomy, most data is already published with the paper anyways), I don't want/need to look through a notebook with the underlying code of the paper in order to just read the intro/conclusions (and m…

I think having the ability to focus on the things you care about the paper mostly is what would be more beneficial for all readers. You care more about an overview? You can easily find it (perhaps with graphics and walkthroughs), you care more about proofs? Then you can get them, what about code and experiments? And so on and so forth.

Readability and scalability is about making all this data available in the publication record, but easy to navigate for whoever is looking for whatever.

Re: The Scientific Paper is Obsolete (2018)

#62

I feel like the website paperswithcode.com addresses this very well, especially with their feature "quick start in Colab". For example, here's the top paper on the website as of now: https://paperswithcode.com/paper/towards-real-world-blind-fa... . Instead of going through the process of cloning a repo, initializing a fresh Anaconda environment from scratch, reading through nebulous, haphazard documentation about how…

It's really disappointing that technical societies like the ACM and IEEE haven't done this already. For many journals and conferences there isn't even a way to submit the code or other digital artifacts with the PDF. A few have badging for whether digital artifacts are provided and whether the results have been reproduced or repeated by others - steps in the right direction at least. As much as I intensely dislike th…

I don't know if they necessarily should. It's easy enough to post your own supporting materials online. But it doesn't fit the publisher's mission of keeping a timeless paper trail of ideas.

Re: The Scientific Paper is Obsolete (2018)

#63

As a practicing scientist, I firmly believe the world would be much better off if we simply published version-controlled Jupyter notebooks on a free site, such as GitHub or ArXiv.

So they would suffer the same decay all other web resources do? Broken links, no longer maintained tools, their opinionated choice (or lack of flexibility - which text + mathematical notation do have - if a set of stacks and data formats gets standardized).

Nobody is preventing scientists from publishing code and data in addition to & before the paper, which imho itself should be as conservative in format as possible to provide the most universal baseline for understanding, reproducibility, and reliability.

Re: The Scientific Paper is Obsolete (2018)

#64

Earlier quoted context omitted.

If I've learned anything in my career it is that no, ideas are not valuable. There are vastly more bad ideas than good ideas. What makes an idea valuable is validation. Papers aren't to present ideas: papers are to present ideas that have been validated. We proposed an idea, we went and ran some experiments or gathered data some other way, and we concluded the idea was valid (or not valid). The point of this discussi…

But there are so many valuable papers in CS that just presented an idea. If you ignored them you’d be ignorant of how to do 90% of modern engineering. Likely the tech stack you use is built on a tower of ‘just idea’ papers.

Really? I've read a lot of SIGGraph papers, and sure, they didn't provide all the code. But you know that the code exists. And certainly for those there's a lot of trust. I think we're talking here about something different. Not "hey, you can use quaternions for animation" but "If you factor in hippopotamus usage, young adults experience 22% more flange per square doughnut, and we got all this data, and we ran it through 25 separate computer programs written in lolcode, and look, proof!"

Re: The Scientific Paper is Obsolete (2018)

#65
post #30

Earlier quoted context omitted.

Yes. We should record all of this, and turn them into easily browsable graphs/hypertext to easily assemble sets of papers to read/look into. At the very least things like 'background reading', 'further reading', 'supporting evidence' and 'addressed arguments' would be useful. 'We' meaning the librarians and archivists. You guys actually researching have more than enough to do.

Actually I think that's an intriguing idea for how to improve citations. Instead of a single citation, have multiple citations that include a region of text as well as basic categorization of how the citation is being used in that instance. I'm not sure if it would prove feasible in practice. It seems like it would aid the writing process in some cases by helping the author keep track of details. But in other cases m…

[deleted]

Re: The Scientific Paper is Obsolete (2018)

#66

I think this is a great idea theoretically, but in reality for most papers I don't want to see the data/underlying code. While it would be great to publish data/code with the paper (in the field I've worked on the most, astronomy, most data is already published with the paper anyways), I don't want/need to look through a notebook with the underlying code of the paper in order to just read the intro/conclusions (and m…

EMBL-EBI and others had some RDF-related effort to provide machine readable abstracts, which I thought was a really cool idea.

IMHO, the biggest problem with papers is politics and reviews. In many top journals like Nature there's no double-blind review (actually in Nature it's now optional but big groups never use it). And even if there was double-blind review, referees have no skin in the game. So the usual outcome is to get reviewed by a big name in your field, who is actually interested in controlling research trends and killing "competitors".

This is hindering progress and hurting new ideas. For example, proponents of Alzheimer's disease being caused by an infection or dysbiosis have had a hard time to do research, get grants and publish articles during the last 2 decades. Despite their theory is able to explain the etiology quite well, unlike competing alternatives.

Another problem is that to publish in good journals you need cool results. Cool results are rare, but Nature, Science, Cell et al. are full of articles every month. So, most groups are overselling and misreporting things. Research fraud, p-value hacking and data manipulation are really common.

Re: The Scientific Paper is Obsolete (2018)

#67
In my field, at least, I think the problem is less about the medium, and more about the incentives. Researchers are incentivized to write papers that seem impressive (and intimidating) rather than clear and intuitive.

To make matters worse, this is an evolved trait: researchers whose papers are intimidating are more likely to succeed, which means they're more likely to have future PhD students, which means that the style of writing is more likely to get passed on.

I think the main way to address this is to change the incentives. In particular, by creating publication venues that value simplicity and clarity (one such conference is SOSA, which has had a lot of impact on theoretical computer science in the last few years).

Re: The Scientific Paper is Obsolete (2018)

#69

In my field, at least, I think the problem is less about the medium, and more about the incentives. Researchers are incentivized to write papers that seem impressive (and intimidating) rather than clear and intuitive. To make matters worse, this is an evolved trait: researchers whose papers are intimidating are more likely to succeed, which means they're more likely to have future PhD students, which means that the s…

> Researchers are incentivized to write papers that seem impressive (and intimidating) rather than clear and intuitive.

Ah, a fellow economist lol. Lack of clarity is a strategic advantage because (1) (as you said) it looks impressive and (2) it's hard to validate that it's correct.

So many papers contain such elementary statistics mistakes such as survivorship bias, e.g. 'returns to education' is almost exclusively measured by asking individuals who graduated (on average 50% of enrolled students don't) and respond back to surveys (good chance of bias).

Pubs are how you get jobs. It's not about science anymore, it's about navigating bureaucracy for an elite job.

Re: The Scientific Paper is Obsolete (2018)

#70
It's easy to say things are obsolete when you are your own publisher (Victor, Wolfram, Perez) or when can suggest your favorite (even if it's very cool) approach (Jupyter Notebooks) as a potential key solution. If you can't be your own publisher, it's a much more difficult proposition.

We're trying to figure out how to facilitate taxonomists publishing their own taxon pages, i.e. species descriptions, from a science 250+ years old. Our MVP use case is ~20k pages, one per species, for one project. There will be many of these projects, though maybe not many with 20k pages, and some with much greater than 20k pages. Updates are needed with as little latency as possible with data from from multiple sources. There has to be basically zero cost to serve these (I know, nothing is free). Sites must be trivially configurable (e.g. clone a GH template repo and edit a YAML file and some markdown). Even if we can get this in infrastructure in place we then have to figure out how to get the social structure in place to have this type of product recognized as equivalent to traditional on paper publishing, i.e. advance people's careers because they "published".

In my field, until we give people the power to publish on their own, I don't see traditional publishing go away. Many in the past have indeed published (traditionally) their own species descriptions on their own dime, meeting the rules within the various international codes of nomenclature. I also don't have a problem with dead wood- if we go digital too fast we will loose so much for any number of reasons associated with the ephemeral nature of electron-based infrastructures.

Post reply on HN