Live data from Hacker News

Evidence of Fraud in an Influential Study About Procrastination

datacolada.org

271–280 of 292 posts

Re: Evidence of Fraud in an Influential Study About Procrastination

#271

Earlier quoted context omitted.

News reporting and journalism has always mostly been a for-profit enterprise. And no matter what business model one might choose (ads, subs, charity, etc) there will be perverse incentives working in conflict with pure truth seeking and newsworthiness. Audience capture, pleasing advertisers, the politics/interests of the owners, etc. I'm sure non-profit sponsorship has it's own issues, but I don't think those issues…

I think ads were better because everyone was aware of the conflict of interest issues and took steps to manage it. They admitted it could be an issue and would have internal firewalls where ad sales weren't allowed to talk to the journalists, things like that. Advertisers accepted it because their goal wasn't to manipulate news reporting. The problem with the "philanthropic" funding is not only is there no firewall,…

If the theory is that advertising makes conflicts of interest more transparent, I don't think I agree. Advertisers can be stakeholders across diverse ranges of concerns - conflicts of interest are often indirect. They can influence what doesn't get written, as much as what does. The real bad actors can even just disguise ads as content. We still need to have some basic trust that institutions are respecting firewalls that - by and large - they have to self police

> The problem with the "philanthropic" funding is not only is there no firewall, but it's explicitly pay-to-play agenda pushing

> if you go to the AP website today and look at some random article paid for by these grants

Why do you say there is no firewall? Is that really true? The AP has their guidelines here and I don't see it as substantially different than what a publication might do for advertisers and other stakeholders who may be tempted to exert editorial control:

- https://www.ap.org/the-definitive-source/behind-the-news/how...

If you have specific instances where these arrangements have clearly compromised their journalism, that would be good to see.

In the meantime, I think we can do two things at the same time here: resist overdosing on maximal cynicism, but also be clear eyed about the ways money can tempt institutions to throw integrity out the window.

> The right wing are saying things like, journalism is fake because they're just pushing a left wing agenda for money.

> But it turns out that the rightists are correct and that's exactly what journalists are doing, so it's hardly their fault that mistrust was created and cultivated.

Generally speaking, no they are not correct, and most of them are just outright cynically lying, while engaging in the very behaviors of which they accuse others.

Re: Evidence of Fraud in an Influential Study About Procrastination

#272

One thing I didn't know before reading this: an unusually large effect size can itself be a red flag. You don't have to understand all the statistics to at least notice when a result looks almost too good to be true.

This is one of the things that did in Jan Hendrik Schön.

I can't remember the exact measurement but he was showing a set of curves that were simply too neat for a relatively small sample size - roughly 100 or so plastic transistors. In reality you cannot get that neat an effect without a huge sample population

Re: Evidence of Fraud in an Influential Study About Procrastination

#273

Earlier quoted context omitted.

I think ads were better because everyone was aware of the conflict of interest issues and took steps to manage it. They admitted it could be an issue and would have internal firewalls where ad sales weren't allowed to talk to the journalists, things like that. Advertisers accepted it because their goal wasn't to manipulate news reporting. The problem with the "philanthropic" funding is not only is there no firewall,…

There certainly are problems, but the biggest is that reality puts two other factors ahead of publishing the truth: 1) People don't like having their beliefs challenged. This will slant all news organizations towards the biases of their readers. We see this at all levels. The shorter and more personalized the feedback loop the faster it happens. This makes the news with things like ChatGPT, the feedback is very quick…

Point (1) is why news institutions that cover a wide range of interests and have a broad audience have tended to fare better. Depending too much on a particular niche audience leads to audience capture.

WSJ and Fox News have the same owners but WSJ still has one of the best and most credible newsrooms in the business (their opinion section is another matter).

Meanwhile Fox News is a lie factory that only tells it's large (but niche) audience what it wants to hear. And now they have rival networks eating away at the pie by being more sensational and even more shamelessly obsequies. They're stuck in a radicalization doom loop.

Re: Evidence of Fraud in an Influential Study About Procrastination

#274
post #130

Earlier quoted context omitted.

On the other hand, this seems like a super interesting path for an (unglamorous) PhD -- replicate the N most important findings in a field, subject to some restrictions (maybe focus on the ones that are "easy", or haven't been replicated yet, or the ones that could be patched together to make an acceptable PhD thesis). This sort of stuff really appeals to me, I think I might end up doing something like that in 5-10 y…

Good luck finding an advisor/a department that awards PhDs for that. “It’s not novel”.

I think you're right that this will be the most common objection, but wrong that it will be exceedingly difficult to find an advisor that would support this. I assume that if you _can_ overturn some foundational result, you won't face criticism over novelty (potentially you can argue that overturning important accepted results is inherently novel). Then the rest is just math:

Assume that n% of foundational studies are demonstrably incorrect if replication were attempted. Then if you replicate X foundational studies, there's a 1 - (1-n/100)^X chance that you'll show one foundational study to be incorrect. Then I think you just need to find an advisor who believes the incorrectness rate n, and the replication attempts X is high enough for you to revert a foundational study over the course of your PhD.

Re: Evidence of Fraud in an Influential Study About Procrastination

#275

Earlier quoted context omitted.

> Science isn't self correcting.... I am not sure what your proposal is? It seems to be the solution is only more science? This meme that science can't solve anything or is ineffective is also a meme that needs to die. Science is about the only thing that we have to make sense of the world. What else is there? > 1. What exactly is the scientific method? It's not like there's a constitutional amendment that lays it ou…

Replicability isn't enough to be scientific. Many studies have invalid methodologies, so replicating them by following the same method just propagates wrongness and makes the problems worse. And if you don't follow the same method the original authors will claim it's not a replication study. https://blog.plan99.net/replication-studies-cant-fix-science... My proposal is: let science be done by those who need its resul…

> Replicability isn't enough to be scientific. Many studies have invalid methodologies, so replicating them by following the same method just propagates wrongness and makes the problems worse. And if you don't follow the same method the original authors will claim it's not a replication study.

Replicability test also happens in the real world. If gravity is not actually 9.81 m/s^2 then we will eventually have to come to term to that. For all we know, we could be completely wrong about quantum mechanics and the reality of subatomic particles, but it doesn't matter because we can use the predicted behavior and it actually works.

The question is to ask: what is one thing that I would never discard from a scientific principle. If you remove replicability, it does not matter if you have accurately captured a measurement of the universe once, it is completely pointless if you are not able to replicate it again.

Re: Evidence of Fraud in an Influential Study About Procrastination

#276

Earlier quoted context omitted.

My take on the self correction is, that later research usually cannot be build on wrong results. So over the decades or centuries the correct results prevail.

>My take on the self correction is, that later research usually cannot be build on wrong results. AFIK, it does not seem to happen much with mainstream medicinal research. Many layers of fraud/incompetence/assumptions for centuries with no correction in sight. Of course there are certain sub-areas that may have advanced. In fact it's the modern day religion, having replaced the god centric religion of yesteryears.

I'll very happily (as a significant benefactor) take this century's medicine over the last. And the one or two centuries before that aren't even worth starting to think about. So something must be working wioth mainstream medicinal research.

Re: Evidence of Fraud in an Influential Study About Procrastination

#277

Earlier quoted context omitted.

Why do we even accept anything as true without replication by default?

Speak for yourself! My default position for a long time has been that all results in the social sciences are garbage. If they go against intuition or my biases, then they are most likely garbage. If they confirm my intuition or my biases, then they are definitely garbage.

Sure but I mean the community as a whole. Why has there been allowed to be a reproducibility problem at all? Why has the consensus not always been that without provably reproducible results it’s not to be taken seriously?

Re: Evidence of Fraud in an Influential Study About Procrastination

#278
post #244

Earlier quoted context omitted.

Economics is a particularly slippery subject, since double blind experiments are practically impossible. No doubt some of the things Marx wrote correctly describe reality, even if some of the social experiments built on his work were massive failures.

I mean more the Marxist school of thought, not just the economics. Things like critical theory and post modernism. But also, he made predictions of the near future in his time. They were incorrect. No updates were made. Those who did try to refine and update his ideas are far less influential. His original theory survives like gospel. I wish I could say only by non-academics. Edit - back to economics. It's funny. I w…

marxism (a materialist ideology) is incompatible with post-modernism, a post-truth ideology

Re: Evidence of Fraud in an Influential Study About Procrastination

#279

Earlier quoted context omitted.

I think ads were better because everyone was aware of the conflict of interest issues and took steps to manage it. They admitted it could be an issue and would have internal firewalls where ad sales weren't allowed to talk to the journalists, things like that. Advertisers accepted it because their goal wasn't to manipulate news reporting. The problem with the "philanthropic" funding is not only is there no firewall,…

If the theory is that advertising makes conflicts of interest more transparent, I don't think I agree. Advertisers can be stakeholders across diverse ranges of concerns - conflicts of interest are often indirect. They can influence what doesn't get written, as much as what does. The real bad actors can even just disguise ads as content. We still need to have some basic trust that institutions are respecting firewalls…

Yes, it's true. No more firewalls. Go read the article I originally linked, here it is again:

https://apnews.com/article/science-business-arts-and-enterta...

Some key paragraphs are:

> The Associated Press said Tuesday that it is assigning more than two dozen journalists across the world to cover climate issues, in the news organization’s largest single expansion paid for through philanthropic grants.

> The grant is for more than $8 million over three years, and about 20 of the climate journalists will be new hires. The AP has appointed Peter Prengaman as its climate and environment news director to lead the team.

> For many years, Journalists and philanthropists were more wary of each other. News organizations were concerned about maintaining independence and, until the past two decades, financially secure enough not to need help. Philanthropists didn’t see the need, or how journalists could help them achieve their goals.

> Nonprofit news organizations like ProPublica and Texas Tribune led the way in changing minds. The Salt Lake Tribune, which in 2019 became a nonprofit to attract more donors, and The Seattle Times are other pioneers.

> “As far as we’re concerned, it has been really seamless,” said Monica Drake, assistant managing editor at the Times. “It has allowed us to focus on things we want to cover and to do it independently.”

> Both sides had things to learn. For Carovillano, it was getting used to the idea that funders weren’t just being generous; they had their own goals to achieve. “This is a mutually beneficial arrangement,” he said.

> “I feel like our language has gone from people talking about local news dying to there being a renaissance of local news,” she said. “If you can provide good, high-quality journalism, people are willing to pay for that, people are willing to support it. I think there’s a lot of room for optimism. The trick is, can we move quickly enough?”

The article leads with climate grants but is really about the phenomenon in general, which covers many topics.

The quoted journalists are duplicitous, trying to reassure themselves. The text says that what they're doing used to be taboo because rich guys had "goals to achieve", but now it's considered "pioneering" and "mutually beneficial". Their justification is they didn't have any money and felt sad, but now they've got lots of money and are happy. They say the transition has been "seamless", i.e. readers aren't aware of it. And hilariously, at the end Carivillano claims there's room for optimism because people are willing to pay for high quality news - except "people" aren't paying, that's the reason they need to take money from a handful of donors.

It's obvious this compromises their journalism, especially in the stories they don't publish. These news orgs have basically been bought by the donors, large chunks of their staff only exist because of these grants. Thinking it doesn't compromise their news would be like saying Facebook simply buying all the news teams that cover the internet wouldn't have any effect.

Re: Evidence of Fraud in an Influential Study About Procrastination

#280

Earlier quoted context omitted.

Most of the problems are with academic fields that were barely funded before WW2.

Your comment claims that academics do not correct other academics. Your comment appears on a post in which, literally, academics are correcting other academics.

Data Colada is a handful of volunteers who do this kind of thing as extra-curricular activities, and rely primarily on tips. That is a tiny drop in an ocean of fraud, for which the institutions get no credit at all. Academic institutions don't systematically check for fraud, but they do systematically ignore fraud reports that come from outside the academy (of which there are many).
Post reply on HN