Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Scientific Paper is Obsolete (2018)

theatlantic.com

Post reply on HN
Print view View original

151–160 of 175 posts

Re: The Scientific Paper is Obsolete (2018)

#151

Post by Mezzie »

fho wrote: post #130
Mezzie wrote: post #27

Earlier quoted context omitted.

> I also think citations of your work that find errors or correct things should count as a negative citation. Strong disagree. Given how much influence colleagues can have over one another's career prospects, how petty academic disagreements can get, admin focus on metrics like citation count, and how it's easier to prove someone else wrong than to do your own original work (both have value, one is just easier), it w…

Which is basically the reddit up- & down-vote versus the HN only upvotes story just on a broader scale, is it?

Superficially yes, but in actuality it would be very different due to the context surrounding academic papers vs. Reddit.

Organizationally speaking, Reddit is a dumpster fire; check out the 'search' function (I'm just speaking on a taxonomical/categorization perspective, I can't speak to their dev practices).

Academic papers aren't. (They're a dumpster fire in their own ways: The replication crisis and the lack of publishing negative results comes to mind, but damn if they aren't all organized!)

There's two key differences:

1.) Academic papers have other supporting metadata that could combine with the more in-depth citation information to offer clear improvements to the discovery process. Imagine being able to click on a MeSH term and then see, in order, what every paper published on that topic in the past year recommends you read. I also think improving citation information would do a lot to make research more accessible for students.

2.) Reddit's system lets anybody with an account upvote or downvote. Given you don't even need an email address to make a Reddit account, there's functionally zero quality control for expressing an opinion. For academic publications, there is a quality control process (albeit an imperfect one). If only 5 people in the world understand a given topic, it's really helpful to be able to see THEIR votes: If they all 'downvote' a paper that would suggest it's wrong.

Re: The Scientific Paper is Obsolete (2018)

#152

Post by Mezzie »

queuebert wrote: post #79
Mezzie wrote: post #27

Earlier quoted context omitted.

> I also think citations of your work that find errors or correct things should count as a negative citation. Strong disagree. Given how much influence colleagues can have over one another's career prospects, how petty academic disagreements can get, admin focus on metrics like citation count, and how it's easier to prove someone else wrong than to do your own original work (both have value, one is just easier), it w…

You make a good point, and maybe there is a mathematical solution, like netting out mutually negative citations.

Maybe! Probably not that solution since it would penalize academics who are having a long-standing but intellectually productive disagreement.

I think it'd be hard to math out right now since the skill doesn't exist in the departments who'd be doing the work, but in 20 years who knows?

Re: The Scientific Paper is Obsolete (2018)

#153

Post by eesmith »

tsbischof wrote: post #124
eesmith wrote: post #121

Earlier quoted context omitted.

My example of a protein model is an example of empirical research, yes? My understanding is the X-ray gives you a diffraction pattern which is hard to invert to a structure, while if you have the structure the diffraction pattern is easy to compute. The diffraction pattern therefore gives you a way to verify that one model is a better fit than another model. It may not be perfect, certainly not. It might not even be…

What do you see as the fundamental point of scientific communication? In your counterpoints you narrow in on papers being a means of communicating concepts or proof of work. In this view, showing the process itself is pointless or at least irrelevant to the main axiom. However, others (myself included) see the the communication of methods as a primary function of the literature, because this is what enables others to…

There is no single fundamental point.

If you want to be that broad about it, science journals publish a lot more than just method development, including obituaries and opinion pieces on where funding should be directed.

Here's a famous paper showing that "Euler's conjecture on sums of like powers" is incorrect - https://www.ams.org/journals/bull/1966-72-06/S0002-9904-1966... . I will repeat the body in full:

> A direct search on the CDC 6600 yielded 27⁵ + 84⁵ + 110⁵ + 133⁵ = 144⁵ as the smallest instance in which four fifth powers sum to a fifth power. This is a counterexample to a conjecture by Euler [l] that at least n nth powers are required to sum to an nth power, n>2.

Do I need to know how the direct search was carried out to confirm Euler's conjecture was false?

No.

  >>> 27**5 + 84**5 + 110**5 + 133**5 == 144**5
  True
And now that you know it isn't true, you might adjust which project areas to spend your time on. Which is part of what we get from scientific publications.

Just because you prefer one sort of scientific research doesn't mean other forms aren't science.

Again, is Kekulé's model of the benzene ring less scientific because it came to him in a daydream?

We accept Newton's publications where he secretly used the calculus, even though he didn't publish the calculus, because they could be proved through other more laborious means.

Why is it not scientific to write publications which use secret software, so long as we can verify the results?

Re: The Scientific Paper is Obsolete (2018)

#154

Post by eesmith »

kragen wrote: post #123
eesmith wrote: post #118

Earlier quoted context omitted.

I think I can convince you otherwise. If I publish a paper saying I have an algorithm which can factor large composites, and in the paper publish the factors to all of the RSA numbers listed at https://en.wikipedia.org/wiki/RSA_Factoring_Challenge , then I think people will take it seriously, and not consider it at the superficial level. Even if I don't publish the algorithm. ("Because of the security implications of…

Yes, there are occasional exceptions where you don't have to repeat or replicate the experiments reported in a paper to verify them. But that is very much the exception. Generally you are expected to explain what you did in enough detail that the reader can replicate your experiment. If you're fitting a protein model to X-ray diffraction data, you aren't expected to include all the other protein models you considered…

Sure, but "I found the structure after 5 months playing around with it in Foldit" isn't that reproducible or informative either.

The result is still the same - a novel fold which is a significantly better fit than existing modules, based on measured vs. predicted x-ray diffraction patterns and whatever other data you might have.

Which is publishable, yes?

When the Wikipedia entry at https://en.wikipedia.org/wiki/Foldit says "Foldit players reengineered the enzyme by adding 13 amino acids, increasing its activity by more than 18 times", how is that much different than "A magical wizard added 13 amino acids, increasing its activity by more than 18 times"?

Or "secret software".

What's publishable is that the result is novel (and hopefully interesting), and can be verified. The publication does not require that all step can be repeated.

Re: The Scientific Paper is Obsolete (2018)

#155

Post by [deleted] »

chrisseaton wrote: post #148
laichzeit0 wrote: post #90

Earlier quoted context omitted.

I translated what you said in the context of mathematics: “But the proof is secondary to theorem. The theorem and the discussion around how it was arrived at and what it means is the key thing. The proof is just there to show it’s true. You could write a proof for the same theorem ten different ways.” Which is all true. But man it wastes so much time having to re-prove everything. Also some lemmas/theorems are so har…

Ok but can you understand other fields are different to mathematics? In other fields we argue more informally.

[deleted]

Re: The Scientific Paper is Obsolete (2018)

#156

Post by BeetleB »

seoaeu wrote: post #132
BeetleB wrote: post #56

Earlier quoted context omitted.

> It forces the author to sit down and work through things in an orderly manner and they're still often difficult to read. As a former academian: Papers are difficult to read primarily because the academic community does not value making them easier to read - no other reason. You may hear things like "papers should be written for other experts", but even that doesn't hold up to scrutiny. They typically spend 99% of t…

> They typically spend 99% of their research time on the actual research, and less than 1% This isn’t anywhere close to my experience. 2-3 days of writing per year seems like a wild underestimate for any academic I know. I’d maybe believe only 20% of time spent writing, but for some folks even that’s probably way too low

Are they writing papers or grant applications? I'm speaking specifically about papers.

And fine, even if 1% is an exaggeration, I doubt they spend more than 5% (about 18 days of the year).

Re: The Scientific Paper is Obsolete (2018)

#157

Post by BeetleB »

ciphol wrote: post #125
BeetleB wrote: post #56

Earlier quoted context omitted.

> It forces the author to sit down and work through things in an orderly manner and they're still often difficult to read. As a former academian: Papers are difficult to read primarily because the academic community does not value making them easier to read - no other reason. You may hear things like "papers should be written for other experts", but even that doesn't hold up to scrutiny. They typically spend 99% of t…

As a former academian, I mostly disagree. In my experience, papers are difficult to read because they are as concise as possible, striving to refer to previous work for anything that's not original, and only elaborating on original things. This is done to make them as quick as possible to read by experts, which is pretty important given the immense volume of papers that appear in many fields. I do think papers nowada…

> I mostly disagree. In my experience, papers are difficult to read because they are as concise as possible, striving to refer to previous work for anything that's not original, and only elaborating on original things. This is done to make them as quick as possible to read by experts, which is pretty important given the immense volume of papers that appear in many fields.

This is consistent with what I said: Papers are difficult to read by choice. Yes, they strive to make it as concise as possible, which translates to making them harder to read.

Where I would disagree is the claim that it is done to make it as quick as possible to read by experts. In my field, the experts would skim papers quickly to get an idea, but if they then honed in on a paper to actually extract the meat of it so they can use it in their own work, it would take a lot of time, and was a pain.

I've heard from math professors that it takes about a day to read and digest one page of a journal article in their field.

Also disagree on it merely being a matter of consulting references. My work was theoretical/computational. It was common to see the final equations without any derivation, and many experts would not be able to reproduce it. There are lots of tricks that go into the derivation, but they are not provided under the pretext that any expert should be able to solve the equations and derive them.

And in the day of digital media, it's quite trivial to write a paper the way you suggest, and then put all the extra details in appendices. I guarantee that they will be read by most people who want to read the paper in detail, as opposed to merely skimming it.

Re: The Scientific Paper is Obsolete (2018)

#158

Post by tsbischof »

eesmith wrote: post #153
tsbischof wrote: post #124

Earlier quoted context omitted.

What do you see as the fundamental point of scientific communication? In your counterpoints you narrow in on papers being a means of communicating concepts or proof of work. In this view, showing the process itself is pointless or at least irrelevant to the main axiom. However, others (myself included) see the the communication of methods as a primary function of the literature, because this is what enables others to…

There is no single fundamental point. If you want to be that broad about it, science journals publish a lot more than just method development, including obituaries and opinion pieces on where funding should be directed. Here's a famous paper showing that "Euler's conjecture on sums of like powers" is incorrect - https://www.ams.org/journals/bull/1966-72-06/S0002-9904-1966... . I will repeat the body in full: > A dire…

For the article on Euler's conjecture, I am aware of this paper and that it serves as a sufficient proof of work for publication. It includes context for its purpose by citation, and the methods for verification were well-established. There is a class of literature where this type of structure works.

For the Kekule paper [1] there is a significant amount of information about the context and reasoning for the claim. This is not an isolated concept and he wrote at length as to why the idea might be plausible given the current evidence. He also could have written solely about the dream without context, but that lacks a grounding in the reality he was attempting to describe.

If it is possible to write a paper where the result is possible to verify using already-known methods, then by all means write in that style. But this is a subset of the useful papers to be written, and in my experience a small one.

[1] https://gallica.bnf.fr/ark:/12148/bpt6k281952v/f102.item

Re: The Scientific Paper is Obsolete (2018)

#159

Post by kragen »

thfuran wrote: post #141
kragen wrote: post #123

Earlier quoted context omitted.

Yes, there are occasional exceptions where you don't have to repeat or replicate the experiments reported in a paper to verify them. But that is very much the exception. Generally you are expected to explain what you did in enough detail that the reader can replicate your experiment. If you're fitting a protein model to X-ray diffraction data, you aren't expected to include all the other protein models you considered…

>Yes, there are occasional exceptions where you don't have to repeat or replicate the experiments reported in a paper to verify them. But that is very much the exception. And even in this exceptional case, the algorithm itself is interesting above and beyond the fact of its existence.

It is, but if the algorithm produces a result such as a protein structure or a sorting network that is itself novel and verifiable, you can very reasonably publish that result separately. As long as it doesn't require knowing the search algorithm to replicate your result that the sorting network sorts correctly, which it wouldn't.

Re: The Scientific Paper is Obsolete (2018)

#160

Post by kragen »

eesmith wrote: post #154
kragen wrote: post #123

Earlier quoted context omitted.

Yes, there are occasional exceptions where you don't have to repeat or replicate the experiments reported in a paper to verify them. But that is very much the exception. Generally you are expected to explain what you did in enough detail that the reader can replicate your experiment. If you're fitting a protein model to X-ray diffraction data, you aren't expected to include all the other protein models you considered…

Sure, but "I found the structure after 5 months playing around with it in Foldit" isn't that reproducible or informative either. The result is still the same - a novel fold which is a significantly better fit than existing modules, based on measured vs. predicted x-ray diffraction patterns and whatever other data you might have. Which is publishable, yes? When the Wikipedia entry at https://en.wikipedia.org/wiki/Fold…

I agree!

Unfortunately we have a long way to go to make it easy to repeat the calculation that a novel structure is "a significantly better fit than existing modules, based on measured vs. predicted x-ray diffraction patterns". (If I run STEREOPOLE and it says the diffraction pattern from your new structure is a worse fit, is that because I'm running a different version of IDL? Maybe there's a bug in my FPU? Or the version of BLAS my copy of IDL is linked with? Or you're using a copy of STEREOPOLE that a previous grad student fixed a bug in, while my copy still has the bug? And stochastic software like GAtor is potentially even worse.)

This is something we could and should completely automate. There's been work on this by people like Konrad Hinsen, Yihui Xie, Jeremiah Orians, Eelco Dolstra, Ludovic Courtès, Shriram Krishnamurthi, Ricardo Wurmus, and Sam Tobin-Hochstadt, but there's a long way to go.

Post reply on HN