Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data detectives spotted fake numbers in a widely cited paper

economist.com

Post reply on HN
Print view View original

31–40 of 42 posts

Re: Data detectives spotted fake numbers in a widely cited paper

#31

Post by PaulDavisThe1st »

hrhdkdlfnrne wrote: post #8

A lot of science today is basically parallel construction. You start with a sexy story that you know will get you a lot of press, like "promising you will be honest actually makes you behave in an honest way" and then you just make that paper happen, however you can. Under the publish or perish system, scientists don't have time to actually research the topic, and imagine if it fails to confirm - you just wasted a lo…

> A lot of science today is basically parallel construction. You start with a sexy story that you know will get you a lot of press, like "promising you will be honest actually makes you behave in an honest way" and then you just make that paper happen, however you can.

Gigantic accusation, zero evidence.

If this was a paper, rather than an HN comment, I'd say there was every chance that it would be self-illustrating.

Re: Data detectives spotted fake numbers in a widely cited paper

#32

Post by btilly »

atoav wrote: post #27
hrhdkdlfnrne wrote: post #8

A lot of science today is basically parallel construction. You start with a sexy story that you know will get you a lot of press, like "promising you will be honest actually makes you behave in an honest way" and then you just make that paper happen, however you can. Under the publish or perish system, scientists don't have time to actually research the topic, and imagine if it fails to confirm - you just wasted a lo…

I think it is important not to extrapolate from a replication crisis in one field (e.g. psychology) to all the other sciences, because the picture painted by this (to my knowledge) doesn't accurately describe the underlying practises.

No, it is exactly the opposite.

The well-documented failings of psychology should cause us to take a closer look at other fields to see if they have similar problems. If they don't, then great. If they do, then fix them.

For example people studying metascience have found that a lot of medical research is of questionable accuracy. It is not as bad as psychology. But it is bad and I'm glad that people are taking this problem seriously.

Re: Data detectives spotted fake numbers in a widely cited paper

#33

Post by feikname »

uuidgen wrote: post #9
feikname wrote: post #2

It always suprises me how people don't do minimum effort for hiding this stuff. And no one notices. (or don't care about the obviously suspicious aspect) Do scientists actually read what they cite?

Did YOU even check what you cite? The AUTHORS of the original paper got a dataset from a company. They didn't assume the fraud from the start and published the paper based on it. Later, when they tried to analyze the issue more in-depth, they couldn't replicate the results. THE ORIGINAL AUTHORS PUBLISHED a paper about a failure to replicate. It was just then that someone looked at the original data and found that it…

> Did YOU even check what you cite?

I did completely read what was available to me without having an account.

> The AUTHORS of the original paper got a dataset from a company. They didn't assume the fraud from the start and published the paper based on it.

My comment is not about who the culprit is or isn't. Indeed, I don't mention anything about it.

Rather, it's about how, as the title says, a WIDELY cited paper has fabricated data following rather (IMO) obvious red flag patterns and none of the people -who cited the paper- raised issues about that.

Thus, I questioned whether scientists read or not the papers they cite in the parent post. The question is not a judgment, I'm just truly curious since I'm not part of the formal academia, just an undergraduate.

Re: Data detectives spotted fake numbers in a widely cited paper

#34

Post by jbjohns »

guerrilla wrote: post #26
dr_dshiv wrote: post #19

Earlier quoted context omitted.

Science is what scientists do, like politics is what politicians do? Either science can be critiqued as a social construct or it is an unimpeachable Platonic aspiration. I can see both perspectives. But, communicating that science itself is a somewhat messy social phenomena might be better as a long-term message for the public.

Politics is definitely not defined as what politicians do and science is, as the GP said, when somone follows the scientific method which is something that happens all the time, far from a Platonic aspiration.

Is there such a method?

https://www.discovermagazine.com/planet-earth/the-scientific...

Re: Data detectives spotted fake numbers in a widely cited paper

#35

Post by 1cvmask »

snakeboy wrote: post #10

I think the original blog post [0] or Andrew Gelman's discussion of it [1] are both better sources for technical details and some historical context. In particular, this is not the first such issue for Dan Ariely, as Gelman points out, he has a history of sketchy scientific ethics like doing media tours for studies that he knows failed to replicate. [0] http://datacolada.org/98 [1] https://statmodeling.stat.columbia.…

And how it funnily relates to the Gell-Mann amnesia effect. Although named after physicis Murray Gell-Mann:

https://www.epsilontheory.com/gell-mann-amnesia/

Re: Data detectives spotted fake numbers in a widely cited paper

#36

Post by dang »

Previous threads on this. Others?

A study on dishonesty was based on fraudulent data - https://news.ycombinator.com/item?id=28271805 - Aug 2021 (42 comments)

Noted study in psychology fails to replicate, crumbles with evidence of fraud - https://news.ycombinator.com/item?id=28264097 - Aug 2021 (102 comments)

A Big Study About Honesty Turns Out to Be Based on Fake Data - https://news.ycombinator.com/item?id=28257860 - Aug 2021 (90 comments)

Evidence of fraud in an influential field experiment about dishonesty - https://news.ycombinator.com/item?id=28210642 - Aug 2021 (51 comments)

Re: Data detectives spotted fake numbers in a widely cited paper

#37

Post by hrhdkdlfnrne »

atoav wrote: post #27
hrhdkdlfnrne wrote: post #8

A lot of science today is basically parallel construction. You start with a sexy story that you know will get you a lot of press, like "promising you will be honest actually makes you behave in an honest way" and then you just make that paper happen, however you can. Under the publish or perish system, scientists don't have time to actually research the topic, and imagine if it fails to confirm - you just wasted a lo…

I think it is important not to extrapolate from a replication crisis in one field (e.g. psychology) to all the other sciences, because the picture painted by this (to my knowledge) doesn't accurately describe the underlying practises.

In medical sciences is worse. In one attempt to replicate "lamdmark" cancer studies 89% failed to replicate.

https://www.nature.com/articles/483531a

Re: Data detectives spotted fake numbers in a widely cited paper

#38

Post by dr_dshiv »

jbjohns wrote: post #34
guerrilla wrote: post #26

Earlier quoted context omitted.

Politics is definitely not defined as what politicians do and science is, as the GP said, when somone follows the scientific method which is something that happens all the time, far from a Platonic aspiration.

Is there such a method? https://www.discovermagazine.com/planet-earth/the-scientific...

Right. Nice article. Tldr; no, not really, "the scientific method" is rhetoric.

Though i will say that they missed Robert Hooke's essay on the scientific method. I swear no one knows about this (even though Hooke was a founding and seminal member of the English Royal Society) because Hooke sounds insane. Who makes titles like this? I love it:

A scheme, or idea of the present state of natural philosophy, and how its defects may be remedied by a methodical proceeding in the making experiments and collecting observations wereby to compile a natural history, as to the solid basis for the superstructure of philosophy

Re: Data detectives spotted fake numbers in a widely cited paper

#39

Post by guerrilla »

jbjohns wrote: post #34
guerrilla wrote: post #26

Earlier quoted context omitted.

Politics is definitely not defined as what politicians do and science is, as the GP said, when somone follows the scientific method which is something that happens all the time, far from a Platonic aspiration.

Is there such a method? https://www.discovermagazine.com/planet-earth/the-scientific...

Yes, despite the demarcation problem, I think there is.

https://plato.stanford.edu/entries/scientific-method/

And I think the demarcation problem can be in some ways be solved by distinguishing on predictivity.

Re: Data detectives spotted fake numbers in a widely cited paper

#40

Post by atoav »

btilly wrote: post #32
atoav wrote: post #27

Earlier quoted context omitted.

I think it is important not to extrapolate from a replication crisis in one field (e.g. psychology) to all the other sciences, because the picture painted by this (to my knowledge) doesn't accurately describe the underlying practises.

No, it is exactly the opposite. The well-documented failings of psychology should cause us to take a closer look at other fields to see if they have similar problems. If they don't, then great. If they do, then fix them. For example people studying metascience have found that a lot of medical research is of questionable accuracy. It is not as bad as psychology. But it is bad and I'm glad that people are taking this p…

Good point. We should be careful in other fields as well.

I came at it from another direction here — there are people who are like: "Look at the replication crisis in psychology — this is proof science cannot be trusted in general". So what I meant to argue here is that this conclusion cannot be drawn automatically, not that we shouldn't scrutinize other fields (we should!)

Post reply on HN