Live data from Hacker News

We Should Not Accept Scientific Results That Have Not Been Repeated

nautil.us

261–270 of 282 posts

Re: We Should Not Accept Scientific Results That Have Not Been Repeated

#261
post #248

Earlier quoted context omitted.

The contents of the article are the only thing that ever matters. The title is picked by editors to get the most people to click on it.

I think the title is all that matters. I'm only here for the discussion, and the title defines the topic.

Your approach will lead you to having an incorrect take quite often. You should reconsider unless you enjoy being wrong for no particular reason.

Re: We Should Not Accept Scientific Results That Have Not Been Repeated

#262

Earlier quoted context omitted.

Oh I was not commenting on what you do. I was commenting that many who hold that view of non repetition have leanings that are favored by soft sciences. They dont feel the "need" for repetition even in hard science. It is a personal bias thing that influences their need. It can happen when emotion overcomes hard logic and science.

> many who hold that view of non repetition have leanings that are favored by soft sciences. They dont feel the "need" for repetition even in hard science. Really? You're suggesting that psychologists (to arbitrarily pick a softer science) would deny physicists (to arbitrarily pick a harder science) should reproduce their studies where possible? That seems remarkable to me, perhaps I've missed these discussions. Can…

You're clearly reading far too much into the parent. Do you have an emotional attachment to "soft" sciences that is causing you to be defensive in the face of no attack at all?

Re: We Should Not Accept Scientific Results That Have Not Been Repeated

#263

Earlier quoted context omitted.

I think you've got my argument completely backwards - I'm pointing out that if repetition as implied by the article is everything, then results from e.g. the LHC are discounted. Similarly, I am not saying that you don't need to repeat - just that it isn't the be all and end all of what defines 'science'. Supporting this interpretation is that I mentioned DAMA. Nobody is accusing them of not taking care, but nobody re…

I read the article carefully, and was unable to find where it says what you say it says. In particular, you're making super-strong statements like "be all and end all of what defines 'science'" -- that's not what I see in the article. The article is reasonably pointing out that we have a repetition crisis and that more emphasis should be placed on it. That's not the various super-strong things you're claiming you see…

2nd paragraph.

Re: We Should Not Accept Scientific Results That Have Not Been Repeated

#264
post #46
post #42

I think with the increased visibility of scientific research to the general public, it's less that science needs to stop accepting unrepeated results, but instead the paper process needs to be updated to reflect the new level of availability, and journal databases need better relationship views between papers and repeated tests. As an outsider looking in on the Scientific process, I am not really sure how applicable…

Those are both good but the key here is the media needs to understand that scientific papers that have not been independently verified are in a "maybe" state. Of course, they probably do know this and just choose to ignore it because "Unverified Study that MIGHT Point to M&M's Being Good For You" won't get as many clicks as "M&M's Are Good For You Says New Study!"

This is sort of why I think having it stated explicitly within the paper, not just an aside but part of the actual process. It's to pit less scrupulous journalists against one another, in an "honor among thieves" sort of way I guess. If someone wants to go ahead and write clickbait, they can, but it leaves them open to someone else looking to discredit them going "well, did you even read the paper? they told you not to write that."

it's not so much checking for the public purpose, it's for others.

Re: We Should Not Accept Scientific Results That Have Not Been Repeated

#265

Earlier quoted context omitted.

> many who hold that view of non repetition have leanings that are favored by soft sciences. They dont feel the "need" for repetition even in hard science. Really? You're suggesting that psychologists (to arbitrarily pick a softer science) would deny physicists (to arbitrarily pick a harder science) should reproduce their studies where possible? That seems remarkable to me, perhaps I've missed these discussions. Can…

You're clearly reading far too much into the parent. Do you have an emotional attachment to "soft" sciences that is causing you to be defensive in the face of no attack at all?

Project much? Looks like you're the one getting all heated and angry at someone defending the social sciences and are trying to attack them. It's better not to bring emotion into discussions of science.

Re: We Should Not Accept Scientific Results That Have Not Been Repeated

#266

Earlier quoted context omitted.

I think the title is all that matters. I'm only here for the discussion, and the title defines the topic.

Your approach will lead you to having an incorrect take quite often. You should reconsider unless you enjoy being wrong for no particular reason.

Well it hasnt, and Ive had this "approach" for years.

Re: We Should Not Accept Scientific Results That Have Not Been Repeated

#267

Earlier quoted context omitted.

Your approach will lead you to having an incorrect take quite often. You should reconsider unless you enjoy being wrong for no particular reason.

Well it hasnt, and Ive had this "approach" for years.

That's like going to a book club without having read the book.

You miss all nuance.

It's becomes very quickly apparent to those who've read the article when others comment without having read beyond the title.

Re: We Should Not Accept Scientific Results That Have Not Been Repeated

#268
So, we should have a structured way to represent that one study reproduces another? (e.g. that, with similar controls, the relation between the independent and dependent variables was sufficiently similar)

- RDF is the best way to do this. RDF can be represented as RDFa (RDF in HTML) and as JSON-LD (JSON LinkedData).

... " #LinkedReproducibility "

https://twitter.com/search?q=%23LinkedReproducibility

It isn't/wouldn't be sufficient to, with one triple, say (example.org/studyX, 'reproduces', example.org/studyY); there is a reified relation (an EdgeClass) containing metadata like who asserts that studyX reproduces studyY, when they assert that, and why (similar controls, similar outcome).

Today, we have to compare PDFs of studies and dig through them for links to the actual datasets from which the summary statistics were derived; so specifying who is asserting that studyX reproduces studyY is very relevant.

Ideally, it should be possible to publish a study with structured premises which lead to a conclusion (probably with formats like RDFa and JSON-LD, and a comprehensive schema for logical argumentation which does not yet exist). ("#StructuredPremises")

Most simply, we should be able to say "the study control type URIs match", "the tabular column URIs match", "the samples were representative", and the identified relations were sufficiently within tolerances to say that studyX reproduces studyY.

Doing so in prosaic, parenthetical two-column PDFs is wasteful and shortsighted.

An individual researcher then, builds a set of beliefs about relations between factors in the world from a graph of studies ("#StudyGraph") with various quantitative and qualitative metadata attributes.

As fields, we would then expect our aggregate #StudyGraphs to indicate which relations between dependent and independent variables are relevant to prediction and actionable decision making (e.g. policy, research funding).

Re: We Should Not Accept Scientific Results That Have Not Been Repeated

#269

Earlier quoted context omitted.

Perhaps those who are skeptical of the first research (and are losing money from it) should fund the replication research. Incentive for corrupting the data seems high, however.

This works for some "controversial" researches, which can be dismantled. What about successful (or apparently) ones which may lead to tons of $$$ in return? Is there any value to pay for a research which is going to prove someone else was right ending in making their wallets fatter? Sorry for the informal language, but makes things a little bit more salty.

If someone is making money from it, someone else is (most likely) losing money from it as well.

When the lightbulb was invented, Edison made lots of money, but I am sure candlemakers had plenty of incentive to fund research that hypothesized that lightbulbs emitted toxins.

Re: We Should Not Accept Scientific Results That Have Not Been Repeated

#270
post #267

Earlier quoted context omitted.

Well it hasnt, and Ive had this "approach" for years.

That's like going to a book club without having read the book. You miss all nuance. It's becomes very quickly apparent to those who've read the article when others comment without having read beyond the title.

Youre wrong
Post reply on HN