Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Data detectives spotted fake numbers in a widely cited paper

economist.com

Post reply on HN
Print view View original

11–20 of 42 posts

Re: Data detectives spotted fake numbers in a widely cited paper

#11

Post by smitty1e »

uuidgen wrote: post #9
feikname wrote: post #2

It always suprises me how people don't do minimum effort for hiding this stuff. And no one notices. (or don't care about the obviously suspicious aspect) Do scientists actually read what they cite?

Did YOU even check what you cite? The AUTHORS of the original paper got a dataset from a company. They didn't assume the fraud from the start and published the paper based on it. Later, when they tried to analyze the issue more in-depth, they couldn't replicate the results. THE ORIGINAL AUTHORS PUBLISHED a paper about a failure to replicate. It was just then that someone looked at the original data and found that it…

Divison of Labour[1] is powerful. The need is to incentivise feedback loops to QA the data on the front end.

The fear of reputational implosion is apparently insufficient.

[1] https://en.m.wikipedia.org/wiki/Division_of_labour

Re: Data detectives spotted fake numbers in a widely cited paper

#12

Post by thrwyoilarticle »

hrhdkdlfnrne wrote: post #8

A lot of science today is basically parallel construction. You start with a sexy story that you know will get you a lot of press, like "promising you will be honest actually makes you behave in an honest way" and then you just make that paper happen, however you can. Under the publish or perish system, scientists don't have time to actually research the topic, and imagine if it fails to confirm - you just wasted a lo…

There's a lot of value being abused in the term 'science'. Science is a highly valued concept but it's the result of following the scientific method, not the output of anyone with a postgrad.

Re: Data detectives spotted fake numbers in a widely cited paper

#13

Post by blunte »

Seems to me...

Any author of a published paper who will not stand behind the paper should have their name removed.

When there is just one name left, that person either accepts responsibility for the content, or they too disavow it and get removed.

When there are no names left, the paper is retracted.

Re: Data detectives spotted fake numbers in a widely cited paper

#14

Post by new_guy »

blunte wrote: post #13

Seems to me... Any author of a published paper who will not stand behind the paper should have their name removed. When there is just one name left, that person either accepts responsibility for the content, or they too disavow it and get removed. When there are no names left, the paper is retracted.

More than that, they should have their degree revoked too.

The spam in these journals puts Buzzfeed to shame.

Re: Data detectives spotted fake numbers in a widely cited paper

#15

Post by ZeroGravitas »

snakeboy wrote: post #10

I think the original blog post [0] or Andrew Gelman's discussion of it [1] are both better sources for technical details and some historical context. In particular, this is not the first such issue for Dan Ariely, as Gelman points out, he has a history of sketchy scientific ethics like doing media tours for studies that he knows failed to replicate. [0] http://datacolada.org/98 [1] https://statmodeling.stat.columbia.…

The datacolada post makes the very reasonable request that all data should be released, and scientists should make that a standard thing to do by doing it themselves and requesting others do it.

It feels like this could be applied retroactively too.

In this case the 2012 authors still had the data that they released in 2020 which is how the analysis got done that showed evidence of fraud. Might be worth just asking a whole bunch of people to release data they previously hadn't and collectively putting some time and effort into that.

Re: Data detectives spotted fake numbers in a widely cited paper

#16

Post by vidarh »

feikname wrote: post #2

It always suprises me how people don't do minimum effort for hiding this stuff. And no one notices. (or don't care about the obviously suspicious aspect) Do scientists actually read what they cite?

The problem is often that reading it is insufficient. Unless you try to replicate, it may well look plausible, and often missing data or details creates barriers that makes you need to want to replicate really badly to put in the effort.

Re: Data detectives spotted fake numbers in a widely cited paper

#17

Post by motohagiography »

hrhdkdlfnrne wrote: post #8

A lot of science today is basically parallel construction. You start with a sexy story that you know will get you a lot of press, like "promising you will be honest actually makes you behave in an honest way" and then you just make that paper happen, however you can. Under the publish or perish system, scientists don't have time to actually research the topic, and imagine if it fails to confirm - you just wasted a lo…

Fraud like this is next-order dispicable because of the knock-on effects it has on knowledge and policy. I don't know that stating that is going to have an effect on people who engage in fraud as a matter of course or as passive parties to it, but here we are. The incentives here are what makes most research an unsuitable substitute or appeal to authority for pubic policy. It's a proxy for a petty elitism, and it means to "trust science," just becomes scientistic nihilism. After all, these days you don't need popular consent when you have scientists.

Reading reports like this, I can see why there has been so much criticism in recent years of the concept of the banality of evil, and why so many researchers affect concern for the environment, because more than anyone, they seem to understand what it means to be responsible for poisoning an ecosystem and they need to get out in front of those narratives. Sure, it's just a bit of fraud in a journal, just like it's just a bit of PCB or mercury in a lake, and only a minority of the population who will be impacted, but someone has to call it out as nihilism, or we're a party to it as well.

As much as I dislike blockchain ideas, it makes me think a blockchain DAG of metadata about the integrity of published papers for citations, reproducability, and evidence of certain types of fraud would rebalance the incentives a bit.

Re: Data detectives spotted fake numbers in a widely cited paper

#18

Post by bobcostas55 »

uuidgen wrote: post #9
feikname wrote: post #2

It always suprises me how people don't do minimum effort for hiding this stuff. And no one notices. (or don't care about the obviously suspicious aspect) Do scientists actually read what they cite?

Did YOU even check what you cite? The AUTHORS of the original paper got a dataset from a company. They didn't assume the fraud from the start and published the paper based on it. Later, when they tried to analyze the issue more in-depth, they couldn't replicate the results. THE ORIGINAL AUTHORS PUBLISHED a paper about a failure to replicate. It was just then that someone looked at the original data and found that it…

It's seems highly likely that Ariely did it, not the company.

Re: Data detectives spotted fake numbers in a widely cited paper

#19

Post by dr_dshiv »

thrwyoilarticle wrote: post #12
hrhdkdlfnrne wrote: post #8

A lot of science today is basically parallel construction. You start with a sexy story that you know will get you a lot of press, like "promising you will be honest actually makes you behave in an honest way" and then you just make that paper happen, however you can. Under the publish or perish system, scientists don't have time to actually research the topic, and imagine if it fails to confirm - you just wasted a lo…

There's a lot of value being abused in the term 'science'. Science is a highly valued concept but it's the result of following the scientific method, not the output of anyone with a postgrad.

Science is what scientists do, like politics is what politicians do? Either science can be critiqued as a social construct or it is an unimpeachable Platonic aspiration. I can see both perspectives. But, communicating that science itself is a somewhat messy social phenomena might be better as a long-term message for the public.
Post reply on HN