Nothing is true, everything is Excel...
A Big Study About Honesty Turns Out to Be Based on Fake Data
41–50 of 98 posts
Re: A Big Study About Honesty Turns Out to Be Based on Fake Data
#42Science is a process, a way to find truth. It is not in itself truth. It is subject to all the same human flaws and venality as any other human endeavor. Just because something is published in a paper doesn’t make it true. Just because the paper was peer reviewed doesn’t make it true. Just because there is a “consensus” doesn’t make it true. I’m sick and tired of people saying “the science is settled” or “trust the s…
While you’re right in a broad sense your points are irrelevant here. This is about fraudulent data (apparently created with a random number generator) being used. This is not and should not be something we expect from scientists. If this story was solely about the replication crisis in behavioral economics, then your comment would be relevant.
There are other ways in which a study can be worthless than simply making up the data.
Re: A Big Study About Honesty Turns Out to Be Based on Fake Data
#43Earlier quoted context omitted.
> I’m sick and tired of people saying “the science is settled” or “trust the science”. You’re right, but also, there’s more to it. There is a difference between what people say and what people mean and context is important. When I’ve heard people say “trust the science” it is always directed to people who have demonstrated that they do not understand the scientific method and the meaning of the word “healthy skeptici…
When I hear people say "trust the science" the speaker is often a non-scientist who is selectively picking what science fits their agenda and ignoring the rest. It's the same thing when somebody says "trust me, I'm a XXXXX", it's time to be very skeptical.
Especially when it's only a single study with a surprising result.
Those go viral easily, but are usually wrong.
Re: A Big Study About Honesty Turns Out to Be Based on Fake Data
#44Earlier quoted context omitted.
> I’m sick and tired of people saying “the science is settled” or “trust the science”. You’re right, but also, there’s more to it. There is a difference between what people say and what people mean and context is important. When I’ve heard people say “trust the science” it is always directed to people who have demonstrated that they do not understand the scientific method and the meaning of the word “healthy skeptici…
When I hear people say "trust the science" the speaker is often a non-scientist who is selectively picking what science fits their agenda and ignoring the rest. It's the same thing when somebody says "trust me, I'm a XXXXX", it's time to be very skeptical.
Bingo.
"Science is the belief in the ignorance of the Experts"
Re: A Big Study About Honesty Turns Out to Be Based on Fake Data
#45Earlier quoted context omitted.
I don't think that's "clear" at all. Even the blog link doesn't come to that conclusion.
I think "We ran 1 million simulations to determine how often this level of similarity could emerge just by chance. Under the most generous assumptions imaginable, it didn’t happen once." is pretty damning.
Oh, somebody made it up. It’s just not proven that he was the one who made it up. It could be someone at the insurance company, or some student paid to type it in. Less likely perhaps, but not impossible.
Re: A Big Study About Honesty Turns Out to Be Based on Fake Data
#46> In the first sign of something amiss, the 13,488 drivers in the study reported equally distributed levels of driving over the period of time covered in the study. In other words, just as many people racked up 500 miles as those who drove 10,000 miles as 40,000-milers. Also, not a single one went over 50,000. I have a hard time believing that Dan Ariely didn’t know about this. The uniform distribution of mileage mak…
It’s not damning per se. That depends on the sampling (or otherwise it would be a tiny, tiny insurer). I’ve done studies where I sampled an insurance population to get equal group size on a few key parameters because we then did follow up questionnaires and I needed to account for non response. No point in random sampling then because all I probably would get was data on the largest groups in the population. As it turned out people loved the subject of our questionnaire (effect of preventive measures by home owners on incidence of a whole range of common claims) and we got about a 70% response rate (that’s crazy high for cold questionnaires to customers) so the study ended up quite overpowered.
Not knowing the sampling, not documenting, not having the emails or at least a zip containing the work (over 4 authors)… that’s a different ballpark.
Re: A Big Study About Honesty Turns Out to Be Based on Fake Data
#47Science is a process, a way to find truth. It is not in itself truth. It is subject to all the same human flaws and venality as any other human endeavor. Just because something is published in a paper doesn’t make it true. Just because the paper was peer reviewed doesn’t make it true. Just because there is a “consensus” doesn’t make it true. I’m sick and tired of people saying “the science is settled” or “trust the s…
We need some new language over this because some science is 'settled' or at least 'strongly indicative'.
I wonder if 'Social Sciences' and 'Psychology' - sine we know so little about it and don't necessarily have a foundation from which to work on ... if they should be called 'Social Philosophy' that happens to use applied scientific methods.
And when the news talks about published papers and scientific results, we can agree on a language like simply the term 'Unverified' or 'Not Fully Verified' to effectively mean 'Not peer reviewed or duplicated'.
That way, when the first Ivermectin trial comes out we can say 'Unverified Ivermectin Study' and that 'Verification' is in progress etc..
Re: A Big Study About Honesty Turns Out to Be Based on Fake Data
#48Science is a process, a way to find truth. It is not in itself truth. It is subject to all the same human flaws and venality as any other human endeavor. Just because something is published in a paper doesn’t make it true. Just because the paper was peer reviewed doesn’t make it true. Just because there is a “consensus” doesn’t make it true. I’m sick and tired of people saying “the science is settled” or “trust the s…
> people saying “the science is settled” or “trust the science” These two are not of the same level of severity. The first statement should usually raise eyebrows, as science is almost never "settled". The first and second laws of thermodynamics are pretty solid. For most other things we can entertain some skepticism. But the second statement raises the question: trust relative to what? If your options are to trust s…
> The first statement should usually raise eyebrows, as science is almost never "settled". The first and second laws of thermodynamics are pretty solid. For most other things we can entertain some skepticism.
One of my good friends is a particle physicist and he assures me that he and his peers very much hope the science isn't settled, despite agreeing pretty well with experiment, because the Standard Model is a hideous kludge.
The heuristic I use is judging based on my estimate of the statement's Shannon entropy. And most of the time the "follow the science" crowd's statements contain absolutely no entropy. I've already heard it verbatim from CNN or some other usually low expertise source. Furthermore, it's annoying to have people lecture me about PCR tests when they don't even know what a restriction endonuclease is. Note that high entropy doesn't mean correct, but to me it does mean more interesting. Now there are plenty of people out there who demonstrably do not want me to have access to high entropy sources, because they believe them to be wrong. They're certainly entitled to their beliefs, but I dislike the idea of someone else deciding what I'm allowed to read.
On the subject of Shannon entropy, I've observed that on technical subjects that readers here have expertise in, like programming, the high entropy comments tend to get upvoted. On the other hand any topic where the crowd here believes "the science is settled" you see the exact opposite effect: high entropy comments are consistently massively disapproved of, while clever restatements of the conventional wisdom with minimal entropy get voted to the top.
Re: A Big Study About Honesty Turns Out to Be Based on Fake Data
#49I've read two of Dan Arielly's books and while occasionally entertaining and occasionally insightful, I could never stop feeling like he was a bullshit artist. His books read as if he read a paper about people liking counterintuitive headlines with sciency-sounding explanations, and he p-hacked his entire career to take advantage of it and become the next Malcolm Gladwell. I kinda expected to see this sort of rebuttal sooner, considering how much of a thorn he was in economists' sides, but not at all surprised to see that it eventually happened.
Re: A Big Study About Honesty Turns Out to Be Based on Fake Data
#50Science is a process, a way to find truth. It is not in itself truth. It is subject to all the same human flaws and venality as any other human endeavor. Just because something is published in a paper doesn’t make it true. Just because the paper was peer reviewed doesn’t make it true. Just because there is a “consensus” doesn’t make it true. I’m sick and tired of people saying “the science is settled” or “trust the s…
> people saying “the science is settled” or “trust the science” These two are not of the same level of severity. The first statement should usually raise eyebrows, as science is almost never "settled". The first and second laws of thermodynamics are pretty solid. For most other things we can entertain some skepticism. But the second statement raises the question: trust relative to what? If your options are to trust s…
Ironically, you picked the one "law" in physics which is technically a statistical statement rather than a law per se.