Peer rejection in science • Hacker News Board

Re: Peer rejection in science

#11

Post by woliveirajr » Sat, Dec 05, 2020, 9:45 AM UTC

meowface wrote: post #6

FooBarBizBazz wrote: post #2

Wait. What? -- > [T]he fantastic takeaway of McConnell’s study [was] that cells other than neurons could store information [...]. [...] Widely known as "the cannibalism experiment," the study [...] fed bits of trained flatworms to their untrained brethren. As a result, McConnell claimed, the untrained flatworms performed behaviors that the trained flatworms had previously learned. In short, the dead flatworms’ memori…

It does make you wonder though... huge ethical and health issues aside, is there even the tiniest chance that eating certain parts of another human could be advantageous or even partly grant you some of the positive traits that other human possesses? There's growing interest in infusion of young people's plasma to counter aging, for example. Fecal transplants (more like gut biome transplants) also come to mind. And C…

What about transplants? You don't need to digest everything to be incorporated to you.

I don't know if there are research about one's behavior changing after receiving an organ (but there are anecdotes about it). Considering that many donors are people who suffered transit accidents, some of those caused by impetuous driving...

Re: Peer rejection in science

#12

Post by scscsc » Sat, Dec 05, 2020, 9:50 AM UTC

acenes wrote: post #10

Let's not jump to conclusions. Yes, there have been numerous cases of "misses." But to evaluate the value of peer review for science, we need to know the "correct rejection" rates as well. Also note that we now know of these cases because science self-corrected, presumably via peer-reviewed replication / validation.

To have a truly fair evaluation, you would also need to know all the cases where science has not self-corrected yet.

Re: Peer rejection in science

#13

Post by marcus_holmes » Sat, Dec 05, 2020, 9:58 AM UTC

scscsc wrote: post #12

acenes wrote: post #10

Let's not jump to conclusions. Yes, there have been numerous cases of "misses." But to evaluate the value of peer review for science, we need to know the "correct rejection" rates as well. Also note that we now know of these cases because science self-corrected, presumably via peer-reviewed replication / validation.

To have a truly fair evaluation, you would also need to know all the cases where science has not self-corrected yet.

This always occurs to me. How many great ideas are out there, that our future will accept as totally normal, but are now just "crackpot" ideas being pursued by weird people. (and a shout out to all the weird people pursuing crackpot ideas: I have no idea how you handle this much rejection, but I'm glad you can).

And how many never got accepted, despite being right? How many times has Academia failed the scientific process and rejected a correct hypothesis?

Re: Peer rejection in science

#14

Post by veddox » Sat, Dec 05, 2020, 10:30 AM UTC

  > From many of these examples, a common thread is the following or a combination thereof:

    - Someone is seen as an outsider within a field

    - The data that supports or undermines claims is noisy. This I think is the core underlying reason, the root of most evil. In turn, the reason for this is lack of tooling to produce precise observations,  or funding to increase sample sizes.

    - Being right, for the wrong reasons. The reasons may be shot down. Rejecting "A->B" does not mean "B" is false, it just means "A"was not a good reason for B.

I think it's important to note that these are actually pretty valid reasons to be sceptical of a claim. (Well, at least the second and third are, but even the first is generally a useful heuristic.)

So I think the issue is not so much that peer-review is broken and needs to be fixed, as that it is by nature conservative and could do with a supplementary process to reduce the false-negatives.

Re: Peer rejection in science

#15

Post by veddox » Sat, Dec 05, 2020, 10:55 AM UTC

veddox wrote: post #14

> From many of these examples, a common thread is the following or a combination thereof: - Someone is seen as an outsider within a field - The data that supports or undermines claims is noisy. This I think is the core underlying reason, the root of most evil. In turn, the reason for this is lack of tooling to produce precise observations, or funding to increase sample sizes. - Being right, for the wrong reasons. The…

Interestingly, it has been argued that the Galileo affair was actually a case of him "being right for the wrong reasons". Obviously there was also a lot of politics involved, but the Vatican did have some of the best astronomers of the time on their pay roll - and Galileo's arguments failed to convince them on a scientific level, not primarily a religious one.

Re: Peer rejection in science

#16

Post by jpkotyla » Sat, Dec 05, 2020, 11:58 AM UTC

veddox wrote: post #14

> From many of these examples, a common thread is the following or a combination thereof: - Someone is seen as an outsider within a field - The data that supports or undermines claims is noisy. This I think is the core underlying reason, the root of most evil. In turn, the reason for this is lack of tooling to produce precise observations, or funding to increase sample sizes. - Being right, for the wrong reasons. The…

Perhaps some non-binary classification could be of use. You could imagine one or two additional categories of acceptance/rejection, something like plausible and implausible. Journals could publish articles falling into these classifications making sure that it is abundantly clear that they have not been fully approved/ disapproved.

Having such options should allow for promotion/demotion of ideas as they garner more/less support.

The concept is similar to that of working papers and preprints, in the belief that getting more eyes (both expert and amateur) on a paper is better for truth discovery than if only a few experts evaluate.

Re: Peer rejection in science

#17

Post by derbOac » Sat, Dec 05, 2020, 1:51 PM UTC

marcus_holmes wrote: post #13

scscsc wrote: post #12

Earlier quoted context omitted.

To have a truly fair evaluation, you would also need to know all the cases where science has not self-corrected yet.

This always occurs to me. How many great ideas are out there, that our future will accept as totally normal, but are now just "crackpot" ideas being pursued by weird people. (and a shout out to all the weird people pursuing crackpot ideas: I have no idea how you handle this much rejection, but I'm glad you can). And how many never got accepted, despite being right? How many times has Academia failed the scientific pr…

That's not even the full content of the frequency cell. What you also need to know are the people who left or were dissuaded because of what they ran into. Eg the mRNA researcher made it clear she almost left; you can be guaranteed if she didn't, others did. It's not the papers that were rejected, it's the ones never written.

We still have this sort of just world narrative about science that I no longer see as warranted. Maybe it was true in the past but not anymore.

Re: Peer rejection in science

#18

Post by marcus_holmes » Sat, Dec 05, 2020, 2:46 PM UTC

derbOac wrote: post #17

marcus_holmes wrote: post #13

Earlier quoted context omitted.

This always occurs to me. How many great ideas are out there, that our future will accept as totally normal, but are now just "crackpot" ideas being pursued by weird people. (and a shout out to all the weird people pursuing crackpot ideas: I have no idea how you handle this much rejection, but I'm glad you can). And how many never got accepted, despite being right? How many times has Academia failed the scientific pr…

That's not even the full content of the frequency cell. What you also need to know are the people who left or were dissuaded because of what they ran into. Eg the mRNA researcher made it clear she almost left; you can be guaranteed if she didn't, others did. It's not the papers that were rejected, it's the ones never written. We still have this sort of just world narrative about science that I no longer see as warran…

Agree completely. Unfortunately expressing any doubts about "Science" these days gets you lumped in with the deniers and anti-vaxxers. Trying to explain "no, I'm totally with the Scientific Method, it's just I don't think Academia does a good job of following it" doesn't seem to cut through that.