The Structure of Scientific Revolutions by Thomas Kuhn has some interesting commentary on how invalid scientific models of our world are eventually (and painfully) discharged. It's the book that coined the term "paradigm shift". This article tends to focus more on bad science, but I think the book is still at least partially relevant.
I'm not a fan of Kuhn, largely because he can be read as saying that, indeed, scientific models that get "overthrown" in a paradigm shift are therefore "invalid". (In the extreme, this leads to complete relativism, the idea that there is no such thing as a "valid" model at all. Kuhn himself has waffled on this point: sometimes he says he didn't really mean to take that extreme view, but other times he says things tha…
To an alarming degree, science is not self-correcting
21–30 of 145 posts
Re: To an alarming degree, science is not self-correcting
#22"Through multiplication upon multiplication of facts, information, theories and hypotheses it is science itself that is leading mankind from single absolute truths to multiple, indeterminate, relative ones. The major producer of the social chaos, the indeterminacy of thoughts and values that rational knowledge is supposed to eliminate, is none other than science itself."
I used to think (10 years ago) he was wrong or concerned about something that would happen far in the future beyond my lifespan.
Re: To an alarming degree, science is not self-correcting
#23Any number of things stand in the way of this: Funding and funding restrictions, counter-productive IP laws, "cult of celebrity" (to the detriment of those quiet souls who "quiety grind away"), etc.
I've been screwed over more than once by "medical science" and medical "best practices", so I have been somewhat sensitized to incongruities in this area. A recent, emerging big one: Statins, and more so the premise upon which they became ubiquitous, stand upon increasingly shaky ground -- at least with regard to their current, widespread prescription and use.
E.g. http://www.peoplespharmacy.com/2013/03/09/895-the-great-chol...
Re: To an alarming degree, science is not self-correcting
#24Do scientists get funding to replicate research? No, not usually.
Does replication still happen? Yes replication of a previous work is often step 0 of new research. It happens all the time.
Do replication results get published? No, not usually. Unless it's a clear rebuttal of a famous paper.
Does this matter? Debatable. Science proceeds slowly, and bad papers tend to be forgotten unless they're easily replicable. It just takes time.
Does this mean that we can't trust science? Absolutely not. You just can't trust individual papers, prima facie. This is something that scientists are supposed to learn as undergrads. Just because it's in a journal (even a "good" journal, like Science or Nature -- some would say especially if it's in one of those journals) doesn't mean it's right.
What does this tell us about armchair science? It's utterly useless. The reason that you have to be a professional scientist is not because professional scientists are smarter, but because they have a huge depth of knowledge about a particular field. Any working scientist can point out published, cited papers in their field that are total crap. But you can't tell that a paper might be crap unless you've spent years reading all of the literature in the field. Yet people on the internet persist in thinking that they can cherry-pick a single paper from arXiv or PubMed, and make sweeping conclusions about a field.
[1] indeed, the usual HN climate-change critics are in this thread, pointing at and yelling about how science is "broken".
Re: To an alarming degree, science is not self-correcting
#25This is an excellent overview article on an important topic. It mentions many of the most influential authors on the topic of accuracy of scientific publications. Hacker News readers may enjoy "Warning Signs in Experimental Design and Interpretation"[1] by Peter Norvig, a LISP hacker who is now director of research at Google, on how to interpret scientific research. Norvig's essay is my all-time favorite link to shar…
Re: To an alarming degree, science is not self-correcting
#26Earlier quoted context omitted.
So there's an interesting topic hidden in your comment. What is truth? I don't mean to ask that in a flippant way. I mean to ask it in a way that has an answer - or at least a reliable set of answers. Of course this is a philosophical debate of long standing, but it has a lot of relevance for today. What do you mean by truth? How do you attain to truth? Is truth what is objectively perceived? If so, how do you determ…
In my experience, when most engineering-oriented people use the word truth, they (wittingly or unwittling) mean the pragmatic definition of "allows us to make stronger-than-previous predictive models about something". Most of the argument in philosophy of science circles strikes me as meta-interesting, not eminently relevant to the practice of science. Especially when dealing with psychology, a lot of those "what, ex…
A historian could speak more pertinently to this, but my understanding is that in 50s-60s "middle" USA, scientists were pretty much treated as oracles of divinity by a great deal of the common population.
Re: To an alarming degree, science is not self-correcting
#27Re: To an alarming degree, science is not self-correcting
#28The truth machine is broken. How do we fix it? And why is it that whenever I see a list of the "top 10 most important problems" to solve, this isn't on it? Most educated people take the veracity of published science as given, and we clearly know that's a false assumption. I know we need more transparency in science - sharing of data and code, and negative results. But institutionally, I don't know how we get there wi…
So there's an interesting topic hidden in your comment. What is truth? I don't mean to ask that in a flippant way. I mean to ask it in a way that has an answer - or at least a reliable set of answers. Of course this is a philosophical debate of long standing, but it has a lot of relevance for today. What do you mean by truth? How do you attain to truth? Is truth what is objectively perceived? If so, how do you determ…
In fact, I'm personally not convinced it's the best standard, but, nevertheless, it is a good start to hold journal papers to the standards they proclaim.
Worry about what the best theory is when they show some ability to implement any theory!
Re: To an alarming degree, science is not self-correcting
#29The truth machine is broken. How do we fix it? And why is it that whenever I see a list of the "top 10 most important problems" to solve, this isn't on it? Most educated people take the veracity of published science as given, and we clearly know that's a false assumption. I know we need more transparency in science - sharing of data and code, and negative results. But institutionally, I don't know how we get there wi…
Re: To an alarming degree, science is not self-correcting
#30The Structure of Scientific Revolutions by Thomas Kuhn has some interesting commentary on how invalid scientific models of our world are eventually (and painfully) discharged. It's the book that coined the term "paradigm shift". This article tends to focus more on bad science, but I think the book is still at least partially relevant.
I'm not a fan of Kuhn, largely because he can be read as saying that, indeed, scientific models that get "overthrown" in a paradigm shift are therefore "invalid". (In the extreme, this leads to complete relativism, the idea that there is no such thing as a "valid" model at all. Kuhn himself has waffled on this point: sometimes he says he didn't really mean to take that extreme view, but other times he says things tha…