Live data from Hacker News

A researcher's response to people pointing out 150 errors in four of his papers

andrewgelman.com

31–40 of 105 posts

Re: A researcher's response to people pointing out 150 errors in four of his papers

#31
post #17

It's weird. If conservatives don't believe scientists, it's because the conservatives are labelled "morons". Could it be that conservatives just think there's a lot more of this crappy "science" out there, and that a situation like this is the canary in the coal mine? When presented with examples like this, it's actually _irrational_ to think that there aren't other egregious papers out there. Possibly just a few, bu…

It should be noted that some conservatives believe in man made climate change. (although they may differ on an appropriate response).

And presumably (although I've never seen it) there is at least one liberal who doesn't.

Re: A researcher's response to people pointing out 150 errors in four of his papers

#33
I went to high school with Brian Wansink. He is brilliant, funny, and affable. I could hear his voice speaking his responses to the objections raised, and snickered a little bit.

I'm sure that Wansink cares about his research and its quality. He's that sort of person. I don't know what happened with these papers, but it seems likely that answering bloggers publicly would not serve the greater good.

It seems to me that all the hand-wringing about these four papers misses the bigger picture: What can we do to improve the practice of science across the board? Why is there almost no field of study which isn't tainted by the politics of left vs. right?

My suggestion is that it began when people began to believe the idea that all political and social questions are best served with some sort of applied scientific analysis. I counter that, in fact, scientists are some of the worst people to ask for solutions to political and social problems, since most of them have almost no understanding of the relevant fields of study, which mainly include history and philosophy, and perhaps psychology. Even fields like theology, linguistics, and literature and the arts have more to say to most political problems than, say, physics or chemistry or meteorology, because they speak to human behavior rather than the behavior of inanimate substances.

Please understand that I say all this not to provoke an argument, necessarily. But as someone whose formal studies were mostly in political history, in general, listening to HN talk about politics is a lot like what I must imagine it is for most on HN to listen to people on TV news shows talk about tech in general and software in particular: they sort of get it, but not really, and certainly are not as expert as they imagine themselves to be.

Re: A researcher's response to people pointing out 150 errors in four of his papers

#34
Wansink applied to the phd program I'm in for graduate school, and didn't get it, and then he applied for a professor position at my school and didn't get it. One of the faculty members told me she thought it was "a missed opportunity."

One thing I have trouble figuring out: is this indicative of a long-term trend? Or did his success create an expectation he couldn't keep up with and so felt a need to rush (or cheat)?

Incidentally, there is another professor at my school who has built a career out of exposing similar problems in research (especially p-hacking). It takes a brave person to inhabit that niche. It is a service to the research community, but I'll bet he sits alone at conferences.

Re: A researcher's response to people pointing out 150 errors in four of his papers

#35
post #25
post #3

Earlier quoted context omitted.

I wouldn't be certain. A lot of researchers really are clueless about anything to do with statistics. It takes much more work/time to do real research, so the funding system has actually been selecting for people who can remain ignorant (so it is not fraud) and just produce p-hacked "results". In many areas, this has been going on for multiple generations now and you are trained to do it as a grad student.

> I wouldn't be certain. A lot of researchers really are clueless about anything to do with statistics. This is definitely true. I once saw a book with a title similar to "Statistics for Dummies" in a professors office. He had plenty of access to staticians at the university too. Unfortunately if a given field involves many people with ignorance of statistics, these problems may not be called out during the peer revi…

>"Unfortunately if a given field involves many people with ignorance of statistics, these problems may not be called out during the peer review process."

As I said, poor understanding of statistics has been a beneficial trait in many areas of research for decades now. Universities and labs are rewarded with funding because they are able to (honestly, but incorrectly) pump out more "results" than if the researchers were properly trained.

So the issue goes well beyond the problems "not being called out". The problems are institutionalized. Editors and reviewers will try to force you to commit the errors in order to publish (and hence have a career). That is what they have been trained to consider science.

Re: A researcher's response to people pointing out 150 errors in four of his papers

#36
post #25
post #3

Earlier quoted context omitted.

I wouldn't be certain. A lot of researchers really are clueless about anything to do with statistics. It takes much more work/time to do real research, so the funding system has actually been selecting for people who can remain ignorant (so it is not fraud) and just produce p-hacked "results". In many areas, this has been going on for multiple generations now and you are trained to do it as a grad student.

> I wouldn't be certain. A lot of researchers really are clueless about anything to do with statistics. This is definitely true. I once saw a book with a title similar to "Statistics for Dummies" in a professors office. He had plenty of access to staticians at the university too. Unfortunately if a given field involves many people with ignorance of statistics, these problems may not be called out during the peer revi…

>I once saw a book with a title similar to "Statistics for Dummies" in a professors office. //

Could easily have been a handy reference text for lending or showing to erring students/faculty.

Re: A researcher's response to people pointing out 150 errors in four of his papers

#37
post #17

It's weird. If conservatives don't believe scientists, it's because the conservatives are labelled "morons". Could it be that conservatives just think there's a lot more of this crappy "science" out there, and that a situation like this is the canary in the coal mine? When presented with examples like this, it's actually _irrational_ to think that there aren't other egregious papers out there. Possibly just a few, bu…

Some areas of science are mostly quackery. Nutrition for one. Some areas are resistant to quackery like physics and chemistry. And most are a mix like medicine.

In school we're taught that every field is as self correcting as physics. Reality seems to be different.

Re: A researcher's response to people pointing out 150 errors in four of his papers

#38
post #33

I went to high school with Brian Wansink. He is brilliant, funny, and affable. I could hear his voice speaking his responses to the objections raised, and snickered a little bit. I'm sure that Wansink cares about his research and its quality. He's that sort of person. I don't know what happened with these papers, but it seems likely that answering bloggers publicly would not serve the greater good. It seems to me tha…

What does any of this have to do with politics?

Wansink doesn't appear to be troubled by his prodigious contributions of unreliable results to the literature, nor does he appear inclined to correct them. All of science and discovery is a joke to him.

Left or right, the troubled extremes of politics start where respect for evidence and truth ends. Scientists are, nominally, people who search for truths about our universe with some degree of rigor, some respect for evidence, some appetite for truth (and usually an insanely competitive nature, but I digress). Absent that, what's the point?

This isn't about society, in my opinion. It's about having some respect for evidence and truth. Without these, science isn't science. And what wansink is doing isn't science. Cold fusion had more rigor. Yet he will probably be rewarded with grants of taxpayer money, taken from fellow citizens as taxes, under threat of force. Meanwhile pediatric bone marrow transplanters will shut down their labs for "lack of funding". Or, more accurately, lack of standards in science.

Yes, it affects people when dishonesty is accepted.

Re: A researcher's response to people pointing out 150 errors in four of his papers

#39
post #25

Earlier quoted context omitted.

> I wouldn't be certain. A lot of researchers really are clueless about anything to do with statistics. This is definitely true. I once saw a book with a title similar to "Statistics for Dummies" in a professors office. He had plenty of access to staticians at the university too. Unfortunately if a given field involves many people with ignorance of statistics, these problems may not be called out during the peer revi…

>I once saw a book with a title similar to "Statistics for Dummies" in a professors office. // Could easily have been a handy reference text for lending or showing to erring students/faculty.

I really hope so. It was a stat strong department (but not statistics).
Post reply on HN