Live data from Hacker News

To an alarming degree, science is not self-correcting

economist.com

31–40 of 145 posts

Re: To an alarming degree, science is not self-correcting

#31
post #24

What a weird article. I agree with nearly every individual fact within it, but after reading it, I still don't know the point of the piece. I don't want to jump to conclusions, but given that this piece is mostly criticism of science the part of me that associates "conservatives" with "anti-intellectual reactionaries" suspects a hidden agenda to undermine science or give climate change "skeptics" something to point a…

Where did this article mention climate change?

If you agree with every fact in the article, why would you look for a hidden agenda?

Your comment makes you seem a bit paranoid.

Re: To an alarming degree, science is not self-correcting

#32
post #13

In my experience at the university performing graduate research - * Newness is prized * Replicating old science is not prized * Funding and papers happens for breakthroughs In software engineering - * many bugs in code creep through exacting peer reviews. Recapping the short list: new things make you money, reviewing things is error prone, replicating old things doesn't make you money until someone wants to rely on t…

As an experimental physicist, I want to point out that negative results do get published -- for example, much of the experimental work on gravity amounts to the showing of no measurable difference from theoretical expectation, to a high degree of accuracy [1] -- but writing the papers for them is much harder than writing up positive results. The reason is simple: you have to convincingly show that your failure to show a result is not just due to your mistakes.

Here is an analogy to a trivial situation. Which is more convincing?

Positive result: By following the steps in the documentation, I installed MS Word on my computer. Therefore, MS Word can be installed on my computer.

Negative result: I followed the steps in the documentation, but MS Word still doesn't work on my computer. Therefore, MS Word cannot be installed on my computer.

If you're like me, you barely pause after reading the positive result, but the negative one brings to mind piles of questions: did you have the right version for your OS? Do you have enough disk space? Does your computer work properly in other respects? A really tricky problem could take ages to figure out. At some point it's probably going to seem wiser just to abandon the problem, and get a different computer or a different program.

So, as a scientist, when you're faced with a negative result, you know you have a battle ahead of you. If you don't think anyone will care very much about your result, moving on to the next project may seem to be the right choice. It's a tough situation, and I sympathize with anyone facing it. And is it the right choice for science as a whole? Sometimes it isn't, but sometimes it is.

[1] For one such example, check out the Eotvos experiment and its many descendants. http://en.wikipedia.org/wiki/Eötvös_experiment

Re: To an alarming degree, science is not self-correcting

#33
post #24

What a weird article. I agree with nearly every individual fact within it, but after reading it, I still don't know the point of the piece. I don't want to jump to conclusions, but given that this piece is mostly criticism of science the part of me that associates "conservatives" with "anti-intellectual reactionaries" suspects a hidden agenda to undermine science or give climate change "skeptics" something to point a…

Where did this article mention climate change? If you agree with every fact in the article, why would you look for a hidden agenda? Your comment makes you seem a bit paranoid.

It didn't mention climate change. I know from past experience that this line of argument -- indeed, any that casts doubt on scientific research -- is a favorite of the "climate skeptic" world. Moreover, I know that the Economist has a center-right political bias.

Maybe they're advocating for something specific here (it's honestly unclear), but given that the Economist almost never writes anything of substance about science, and given their political bias, it isn't at all paranoid to wonder if this article has a subtext. It's called "being an intelligent consumer of media".

Re: To an alarming degree, science is not self-correcting

#34
post #15

The truth machine is broken. How do we fix it? And why is it that whenever I see a list of the "top 10 most important problems" to solve, this isn't on it? Most educated people take the veracity of published science as given, and we clearly know that's a false assumption. I know we need more transparency in science - sharing of data and code, and negative results. But institutionally, I don't know how we get there wi…

So there's an interesting topic hidden in your comment. What is truth? I don't mean to ask that in a flippant way. I mean to ask it in a way that has an answer - or at least a reliable set of answers. Of course this is a philosophical debate of long standing, but it has a lot of relevance for today. What do you mean by truth? How do you attain to truth? Is truth what is objectively perceived? If so, how do you determ…

Truth is discovered. The process of arriving at the discovery of truth is where the perceptions and subjectivity play a role.

For example, water has certain properties based on the conditions surrounding it. These properties exist whether you know about them or not. If you can, through observation and/or measurement, discover these properties, you have discovered truth. If you never discover these properties, it does not affect the existence of the properties.

Re: To an alarming degree, science is not self-correcting

#35

This is an excellent overview article on an important topic. It mentions many of the most influential authors on the topic of accuracy of scientific publications. Hacker News readers may enjoy "Warning Signs in Experimental Design and Interpretation"[1] by Peter Norvig, a LISP hacker who is now director of research at Google, on how to interpret scientific research. Norvig's essay is my all-time favorite link to shar…

Norving missed an excellent chance to title this paper "The structure and interpretation of science experiments"

Re: To an alarming degree, science is not self-correcting

#36
post #24

What a weird article. I agree with nearly every individual fact within it, but after reading it, I still don't know the point of the piece. I don't want to jump to conclusions, but given that this piece is mostly criticism of science the part of me that associates "conservatives" with "anti-intellectual reactionaries" suspects a hidden agenda to undermine science or give climate change "skeptics" something to point a…

Where did this article mention climate change? If you agree with every fact in the article, why would you look for a hidden agenda? Your comment makes you seem a bit paranoid.

He's saying that he wonders if the point of the article is to give those that deny scientific studies in a non-scientific manner ammo.

Something they can point to to rebuke scientific studies because they don't believe the scientific study is valid.

I too didn't understand the point of this article. Nor how it showed that science is not self-correcting. To me it just talked about some issues with scientific research but didn't support it's headline at all. I've never heard anyone claim that science was self-correcting in the short term, these things take decades and sometimes even centuries to self-correct.

And honestly that's ok. Science is all about having the best answer we can have, not the right answer. It's impossible to know we are right beyond the shadow of a doubt. We need to scrutinize how we research and improve our methods as we figure out how to but that doesn't mean science has some major issue. That's jut how it works and how it has historically worked as far as I know.

Re: To an alarming degree, science is not self-correcting

#37
post #24

What a weird article. I agree with nearly every individual fact within it, but after reading it, I still don't know the point of the piece. I don't want to jump to conclusions, but given that this piece is mostly criticism of science the part of me that associates "conservatives" with "anti-intellectual reactionaries" suspects a hidden agenda to undermine science or give climate change "skeptics" something to point a…

It sort of amuses me that there are HN climate change denialists, but there aren't really any HN creationists.

Re: To an alarming degree, science is not self-correcting

#38

This is an excellent overview article on an important topic. It mentions many of the most influential authors on the topic of accuracy of scientific publications. Hacker News readers may enjoy "Warning Signs in Experimental Design and Interpretation"[1] by Peter Norvig, a LISP hacker who is now director of research at Google, on how to interpret scientific research. Norvig's essay is my all-time favorite link to shar…

This beautiful (parent) comment, in contrast to the main article, actually contains links and references to the items it refers to. Is there some compelling reason I don't understand why _so_ many magazines / websites, even high profile reputable ones, cannot manage to link or cite?

Re: To an alarming degree, science is not self-correcting

#39
post #24

What a weird article. I agree with nearly every individual fact within it, but after reading it, I still don't know the point of the piece. I don't want to jump to conclusions, but given that this piece is mostly criticism of science the part of me that associates "conservatives" with "anti-intellectual reactionaries" suspects a hidden agenda to undermine science or give climate change "skeptics" something to point a…

Where did this article mention climate change? If you agree with every fact in the article, why would you look for a hidden agenda? Your comment makes you seem a bit paranoid.

I also think the linked article was somewhat strange. Or maybe just linkbaity. The title can be paraphrased as "To an alarming degree, science is not self-correcting." However, the article actually describes the way science is supposed to work, which is self-correcting.

The main criticism in the article seems to be that, in much of current scientific practice, the self-correction doesn't happen immediately. Well, it would be helpful to know: (1) how does current state of self-correction compare to different times in the past, 50 years ago, 100 years ago, 200 years ago?, (2) what happens over the long term to papers that aren't refuted immediately but which later turn out to be obviously wrong? do they hang around and have a negative impact on progress of science? I'm sure there are more questions.

Another strangeness in the article: The article begins by touting a concern over a 1998 "priming" study. In the third paragraph, the article states that there have been nine (9!) subsequent studies that have failed to replicate the 1998 study's result. Is that not an example of the scientific self-correction working quite well? The article complains, "Either the original research is flawed (as the replicators claim) or the replications are (as many of the original researchers on priming contend). Either way, something is awry." Yes, something is awry. But it's hard to see how it's the scientific method/process itself, which seems to be doing exactly what it's supposed to: calling the 1998 paper into question. (Is the main concern here just that more bad or questionable science is being published than in the past? I'm sure that's true, but there are also many more studies being published in this era and it seems not surprising that overall quality would decline.)

Like some others at HN, I get concerned when I see pieces like this that the science-haters will latch on to and use to discredit the scientific method itself. The article itself _should not_ be taken that way, by anyone who understands science. But there are many who don't understand science, who are anti-science, etc.

Re: To an alarming degree, science is not self-correcting

#40
post #18

Earlier quoted context omitted.

I'm not a fan of Kuhn, largely because he can be read as saying that, indeed, scientific models that get "overthrown" in a paradigm shift are therefore "invalid". (In the extreme, this leads to complete relativism, the idea that there is no such thing as a "valid" model at all. Kuhn himself has waffled on this point: sometimes he says he didn't really mean to take that extreme view, but other times he says things tha…

I was under the impression that models are thrown out in a paradigm shift because they aren't robust enough to account for edge cases that start out as weird (and ignored) outliers to the original model. These outliers eventually become glaringly apparent to the mainstream. Ideally, the new model resulting from a paradigm shift supercedes the old model while modeling the edge cases that the older model was unable to…

I was under the impression that models are thrown out in a paradigm shift

But they aren't always thrown out in a paradigm shift; that's the point. Newtonian physics was not thrown out in the paradigm shift to relativity. Our understanding of why Newtonian physics works as well as it does within its domain of validity changed; but the fact that it does work within its domain of validity did not change.

Kuhn picks a number of examples where that was not the case--where the old model did get thrown out (for example, the paradigm shift from Aristotelian physics to Galilean/Newtonian physics, and the paradigm shift from the Ptolemaic to the Copernican model of the Solar System). But he tried to apply particular features of those examples to all paradigm shifts, which doesn't work.

Ideally, the new model resulting from a paradigm shift supercedes the old model while modeling the edge cases that the older model was unable to account for.

In the case of relativity, it did account for edge cases that Newtonian physics couldn't; but it didn't supersede Newtonian physics, as I noted above. We still use Newtonian physics where its predictions are accurate enough for our purposes, which is most of the time. It's only in particular domains (such as GPS, to pick an example ordinary people are familiar with) that we need relativity to get accurate enough results.

Post reply on HN