Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Scientific Paper is Obsolete (2018)

theatlantic.com

Post reply on HN
Print view View original

111–120 of 175 posts

Re: The Scientific Paper is Obsolete (2018)

#111

Post by ansgri »

nextos wrote: post #66
sam-2727 wrote: post #25

I think this is a great idea theoretically, but in reality for most papers I don't want to see the data/underlying code. While it would be great to publish data/code with the paper (in the field I've worked on the most, astronomy, most data is already published with the paper anyways), I don't want/need to look through a notebook with the underlying code of the paper in order to just read the intro/conclusions (and m…

EMBL-EBI and others had some RDF-related effort to provide machine readable abstracts, which I thought was a really cool idea. IMHO, the biggest problem with papers is politics and reviews. In many top journals like Nature there's no double-blind review (actually in Nature it's now optional but big groups never use it). And even if there was double-blind review, referees have no skin in the game. So the usual outcome…

It’s not really possible to conduct double-blind reviews in most cases: authors or at least the group can often be easily guessed from the list of references, “in our previous work…”, and research domain and approach in general.

Re: The Scientific Paper is Obsolete (2018)

#112

Post by _Microft »

PhilipVinc wrote: post #2

Papers today are longer than ever and full of jargon and symbols. They depend on chains of computer programs that generate data, and clean up data, and plot data, and run statistical models on data. These programs tend to be both so sloppily written and so central to the results that it’s contributed to a replication crisis, or put another way, a failure of the paper to perform its most basic task: to report what you…

Papers aren’t pop-sci articles, they do not target an audience that does not knows anything about the field yet. They are from experts for experts. If someone wants to familiarize themselves with the language, symbols and methods of a field, a textbook is a better thing to start with. Over time they will also learn the shared knowledge of the field that isn’t even mentioned in these articles.

Re: The Scientific Paper is Obsolete (2018)

#113

Post by ketozhang »

PhilipVinc wrote: post #2

Papers today are longer than ever and full of jargon and symbols. They depend on chains of computer programs that generate data, and clean up data, and plot data, and run statistical models on data. These programs tend to be both so sloppily written and so central to the results that it’s contributed to a replication crisis, or put another way, a failure of the paper to perform its most basic task: to report what you…

Certain scientific software packages (e.g., Tensorflow, pymc3, etc) do have frameworks that you follow to return pipeline and result objects that follow some data model that others can learn quickly (e.g., an arviz::InferenceData result object). I wish there was a more extensive framework where this is applied end-to-end from data input, to library components in a pipeline processes, to the result, and then to plot.

Re: The Scientific Paper is Obsolete (2018)

#114

Post by ketozhang »

Ice_cream_suit wrote: post #107

So I am speed reading this: DNA Methylation and Protein Markers of Chronic Inflammation and Their Associations With Brain and Cognitive Aging https://n.neurology.org/content/97/23/e2340.abstract I read the abstract, in reverse order. Discussion and Results first. If it seems plausible ( much research is trash, churned out to pad a CV ) and interesting, then I scan the Methods section. This heuristic helps classify 95…

> At no point do I want pretty visualisations made by wannbe PhD candidates

See Figure (5). Your argument doesn't really counter any part of the scientific notebook. A notebook will still have the abstract and conclusion (result & discussion). The tools mentioned in the article describes how to restructure the methods, data, and figures. You're note going to look at these anyways until the abstract and conclusion intrigues you.

Re: The Scientific Paper is Obsolete (2018)

#115

Post by RandomLensman »

sam-2727 wrote: post #25

I think this is a great idea theoretically, but in reality for most papers I don't want to see the data/underlying code. While it would be great to publish data/code with the paper (in the field I've worked on the most, astronomy, most data is already published with the paper anyways), I don't want/need to look through a notebook with the underlying code of the paper in order to just read the intro/conclusions (and m…

Moving the burden of assessing everything in a paper completely to the reader is an interesting idea but seems somewhat a step back when at the same time good and curated data gets ever more expensive. So the market for validated results is already not bad where those results "matter".

And not every paper has a lot of code or data associated with it. If you do experiments on organisms etc. then there is so much happening in the actual lab work - where would that go? Endless hours of video documentation?

Re: The Scientific Paper is Obsolete (2018)

#116

Post by atoav »

sam-2727 wrote: post #25

I think this is a great idea theoretically, but in reality for most papers I don't want to see the data/underlying code. While it would be great to publish data/code with the paper (in the field I've worked on the most, astronomy, most data is already published with the paper anyways), I don't want/need to look through a notebook with the underlying code of the paper in order to just read the intro/conclusions (and m…

In a ipython notebook you can fold away "blocks" of code, that means you can have everything there that produces the graphs and still be able to look under the hood if you like to.

Isn't that the practical part about digital technology? That you are not limited to one view?

Re: The Scientific Paper is Obsolete (2018)

#117

Post by janeroe »

agucova wrote: post #101
btheshoe wrote: post #81

Similar things can be said about the textbook and the lecture

> Similar things can be said about the textbook and the lecture If you haven't yet, maybe look at Andy Matuschak's "Why books donʼt work" [1] and " How can we develop transformative tools for thought?" [2] which connect these same ideas to education in general. [1] https://andymatuschak.org/books/ [2] https://numinous.productions/ttft/

> Why books donʼt work

They do. First, just like with everything else there are brilliant, good, mediocre, and outright poorly written books. Among those some may work for you, others miss the mark completely depending on your prior experience and background (as the author rightly notices, books are just a medium). Second, did the author expect to become a domain expert after finishing a single book? Clearly, his expectations are unrealistic then. You start somewhere, then use references to deepen your knowledge. That's a task requiring interest and dedication, but no "several lifetimes of research" (of mnemonics and learning methods) as he puts it, will replace that.

Re: The Scientific Paper is Obsolete (2018)

#118

Post by eesmith »

bloaf wrote: post #77
sam-2727 wrote: post #25

I think this is a great idea theoretically, but in reality for most papers I don't want to see the data/underlying code. While it would be great to publish data/code with the paper (in the field I've worked on the most, astronomy, most data is already published with the paper anyways), I don't want/need to look through a notebook with the underlying code of the paper in order to just read the intro/conclusions (and m…

You should always want to have the underlying code available. Without the exact procedures they used to process their data, the only kind of "using their conclusions" you can do is the superficial "take it at face value" kind. So many important details get hand-waved away in papers that say things like "we used the well known blahblahblah method to analyze the data." If you do it right, the code should in no way inte…

I think I can convince you otherwise.

If I publish a paper saying I have an algorithm which can factor large composites, and in the paper publish the factors to all of the RSA numbers listed at https://en.wikipedia.org/wiki/RSA_Factoring_Challenge , then I think people will take it seriously, and not consider it at the superficial level.

Even if I don't publish the algorithm. ("Because of the security implications of this work, I have decided to withhold publication for a year.")

Furthermore, some things are worth publishing even if the methods was "it came to me in a dream" à la Kekulé's snake. If you can demonstrate a sorting network of size 47 for n=14 input (which is the known lowest bound) then you can publish that exemplar, even without publishing the method used to generate it.

(If you used computer assistance then that method would likely also be publishable, but that's a different point. Newton famously used the calculus to solve problems, but published their proofs using more traditional approaches.)

If you can come up with a protein model that is a significantly better fit to the X-ray diffraction data, then that's publishable too, no matter how you came up with that model.

In all of these cases, there are ways to verify the validity of the results without reproducing the methods used to come up with the result.

Re: The Scientific Paper is Obsolete (2018)

#119

Post by [deleted] »

eesmith wrote: post #118
bloaf wrote: post #77

Earlier quoted context omitted.

You should always want to have the underlying code available. Without the exact procedures they used to process their data, the only kind of "using their conclusions" you can do is the superficial "take it at face value" kind. So many important details get hand-waved away in papers that say things like "we used the well known blahblahblah method to analyze the data." If you do it right, the code should in no way inte…

I think I can convince you otherwise. If I publish a paper saying I have an algorithm which can factor large composites, and in the paper publish the factors to all of the RSA numbers listed at https://en.wikipedia.org/wiki/RSA_Factoring_Challenge , then I think people will take it seriously, and not consider it at the superficial level. Even if I don't publish the algorithm. ("Because of the security implications of…

[deleted]

Re: The Scientific Paper is Obsolete (2018)

#120

Post by lonesword »

eesmith wrote: post #118
bloaf wrote: post #77

Earlier quoted context omitted.

You should always want to have the underlying code available. Without the exact procedures they used to process their data, the only kind of "using their conclusions" you can do is the superficial "take it at face value" kind. So many important details get hand-waved away in papers that say things like "we used the well known blahblahblah method to analyze the data." If you do it right, the code should in no way inte…

I think I can convince you otherwise. If I publish a paper saying I have an algorithm which can factor large composites, and in the paper publish the factors to all of the RSA numbers listed at https://en.wikipedia.org/wiki/RSA_Factoring_Challenge , then I think people will take it seriously, and not consider it at the superficial level. Even if I don't publish the algorithm. ("Because of the security implications of…

This won’t work for empirical research. I vividly recall weeks spent trying to reproduce a paper on information retrieval (a deep learning model). What saved me is skimming through the author’s codebase and chancing upon an undocumented sampling step. They were only using the first and last passage in a document as training data and uniformly sampling from 10% of the remaining passages, and the paper didn't mention this. I adopted their sampling strategy, and i was able to obtain their results.

My argument is that there are nuances and subtleties that are often omitted in a paper (accidentally or otherwise), but are nevertheless required to reproduce the research.

Post reply on HN