Live data from Hacker News

A Big Study About Honesty Turns Out to Be Based on Fake Data

buzzfeednews.com

81–90 of 98 posts

Re: A Big Study About Honesty Turns Out to Be Based on Fake Data

#81
post #32

Earlier quoted context omitted.

The 95% effectiveness was based on the version of COVID that was circulating at the time. No one said it would be 95% effective against every possible mutation. Claiming the results of those studies are wrong instead of a result of a changing environment is itself a misunderstanding of science.

Claiming that the vaccines were necessary to "end the pandemic" without factoring in the possibility of variants for which the vaccines were not as effective was not scientific thinking.

Maybe they expected 99%+ to get vaccinated before a vaccine-avoiding variant got out.

Re: A Big Study About Honesty Turns Out to Be Based on Fake Data

#82
post #32

Earlier quoted context omitted.

The 95% effectiveness was based on the version of COVID that was circulating at the time. No one said it would be 95% effective against every possible mutation. Claiming the results of those studies are wrong instead of a result of a changing environment is itself a misunderstanding of science.

Claiming that the vaccines were necessary to "end the pandemic" without factoring in the possibility of variants for which the vaccines were not as effective was not scientific thinking.

Vaccines are still necessary, they just aren't sufficient.

Re: A Big Study About Honesty Turns Out to Be Based on Fake Data

#83
post #46

> In the first sign of something amiss, the 13,488 drivers in the study reported equally distributed levels of driving over the period of time covered in the study. In other words, just as many people racked up 500 miles as those who drove 10,000 miles as 40,000-milers. Also, not a single one went over 50,000. I have a hard time believing that Dan Ariely didn’t know about this. The uniform distribution of mileage mak…

It’s not damning per se. That depends on the sampling (or otherwise it would be a tiny, tiny insurer). I’ve done studies where I sampled an insurance population to get equal group size on a few key parameters because we then did follow up questionnaires and I needed to account for non response. No point in random sampling then because all I probably would get was data on the largest groups in the population. As it tu…

The sample size is in the article:

> Nearly 13,500 drivers were randomly sent one of two policy review forms to sign…

The distribution referenced is of mileage, which you’d expect to have some kind of right-skewed, continuous distribution.

Re: A Big Study About Honesty Turns Out to Be Based on Fake Data

#84
post #7

Science is a process, a way to find truth. It is not in itself truth. It is subject to all the same human flaws and venality as any other human endeavor. Just because something is published in a paper doesn’t make it true. Just because the paper was peer reviewed doesn’t make it true. Just because there is a “consensus” doesn’t make it true. I’m sick and tired of people saying “the science is settled” or “trust the s…

> I’m sick and tired of people saying “the science is settled” or “trust the science”. You’re right, but also, there’s more to it. There is a difference between what people say and what people mean and context is important. When I’ve heard people say “trust the science” it is always directed to people who have demonstrated that they do not understand the scientific method and the meaning of the word “healthy skeptici…

I say “follow the science” about Covid and Global Warming, for instance. Yes, that is a political statement. It’s also rational and reasonable.

I am a tester. Science is never complete, just as testing is never complete. But stupid people think they can ignore bad bugs and get away with it; and stupid people spin fantasies that maybe all those experts are completely wrong about Ivermectin or vaccines.

Trust science is just a way of saying “hey fella, there are people who have devoted their careers to knowing this. Let’s hear from them and take them seriously.”

Re: A Big Study About Honesty Turns Out to Be Based on Fake Data

#85
post #10

"Now some are questioning whether the scientist himself is being dishonest" kinda undersells it. The debunking offers numerous pieces of stone cold proof that the paper is straight-up academic fraud: https://datacolada.org/98 And since the post says "the fourth author has made it clear to us that he was the only author in touch with the insurance company", it seems clear that Ariely personally fabricated the data.

I don't think that's "clear" at all. Even the blog link doesn't come to that conclusion.

The trail of the data goes cold at Ariely. He is refusing to tell who supposedly gave him the data, and can't keep his story straight. He is also marked as the creator of the Excel file in the metadata [0]. If he received the data in Excel format, he wouldn't be the creator. If he received it in some other format.

It's obvious that Ariely is dirty, and I'd bet we'll hear of other similar cases soon enough from other studies that he was involved in. Cheaters don't cheat just once.

[0] https://datacolada.org/98#footnote_13_6142

Re: A Big Study About Honesty Turns Out to Be Based on Fake Data

#86
It's like a set of code, you are liable to hear all kinds of claims about the code base, what it can do, what it can't, but the only way to be sure is to go and examine it for yourself. And every time I do that I find surprises. I imagine the world of scientific experimentation could be the similar in that respect. The synopsis brought to us by scientists can range from slightly misleading to downright fraudulent. It's valuable but unless you get into the weeds of it expect your knowledge to be flawed

Re: A Big Study About Honesty Turns Out to Be Based on Fake Data

#87
post #7

Science is a process, a way to find truth. It is not in itself truth. It is subject to all the same human flaws and venality as any other human endeavor. Just because something is published in a paper doesn’t make it true. Just because the paper was peer reviewed doesn’t make it true. Just because there is a “consensus” doesn’t make it true. I’m sick and tired of people saying “the science is settled” or “trust the s…

This seems like a generalization that needs to limited in scope. I think the science is settled about Newton’s laws, Maxwell’s equations, Avogadro’s number, and DNA as the genetic material in bacteria, archaea, and eukaryotes. For pretty much anything taught in a 100 level science course, the science is largely settled. There are controversies in science, and those controversies make for good popular science press, b…

> I think the science is settled about Newton’s laws

Except it isn't. These laws have been extended once by general relativity already and there's a non-zero chance that the dark matter problematic will prove them insufficient once again. I don't know enough about the other fields, but it's not unlikely that they have similar problems.

The point is, things are really complex and surg edge cases. And if you make final statements like this to someone who is doubting the current consensus anyway, you'll only make yourself vulnerable without convincing them.

Re: A Big Study About Honesty Turns Out to Be Based on Fake Data

#89
post #46

Earlier quoted context omitted.

It’s not damning per se. That depends on the sampling (or otherwise it would be a tiny, tiny insurer). I’ve done studies where I sampled an insurance population to get equal group size on a few key parameters because we then did follow up questionnaires and I needed to account for non response. No point in random sampling then because all I probably would get was data on the largest groups in the population. As it tu…

Sorry, but you seem to be suggesting that non-response bias would be responsible for the uniform distribution seen in the Update mileage digits as well as the uniform miles driven distribution?

Theoretically, yes. I’ve used stratified sampling in this kind of research, since the underlying portfolio is so skewed.

Re: A Big Study About Honesty Turns Out to Be Based on Fake Data

#90

New info from the Buzzfeed article: > “I can see why it is tempting to think that I had something to do with creating the data in a fraudulent way,” [Ariely] told BuzzFeed News. “I can see why it would be tempting to jump to that conclusion, but I didn’t.” He added, “If I knew that the data was fraudulent, I would have never posted it.” [..] he said that all his contacts at the insurer had left and that none of them…

>“I am willing to do a lie detection test on that.”

For a scientist researching honesty, I'd say that referring to a polygraph as a lie detector test raises a few questions.

Post reply on HN