Live data from Hacker News

What has happened down here is the winds have changed

andrewgelman.com

111–120 of 187 posts

Re: What has happened down here is the winds have changed

#111
post #84
post #81

Earlier quoted context omitted.

If the community demands the ability to review code and data then the usage of closed data sets will simply decrease until they comply. We should not base our standards of research on the ease of complying. We should do what we can to produce the most accurate results.

The problem is not one of convenience, the problem is that you are depriving yourself of huge amounts of valuable (closed) data, and this may cause large gaps in what is researchable.

[deleted]

Re: What has happened down here is the winds have changed

#112
post #94

Earlier quoted context omitted.

> Good scientists should welcome this challenge with open arms They should welcome online trolling, abuse, and personal attacks? That is what Fiske criticizes in her article, not serious criticism and debate, which she explicitly welcomes and encourages.

They should welcome online trolling, abuse, and personal attacks? Nobody should welcome it, but -- as it comes with the territory -- they need to ignore it. She works on what is historically and cross-culturally arguably the most controversial subject, namely human sexuality. Attacks are to be expected, and the mature reaction to this is to lead by example: ignore the attacks and be as meticulously scientific as poss…

> it comes with the territory

I disagree; that's an excuse for abusive, unproductive behavior. In many environments, people manage to criticize ideas respectfully and productively, even on social forums like HN. Abuse is counter-productive and completely unnecessary.

Re: What has happened down here is the winds have changed

#113
post #85

> In short, Fiske doesn’t like when people use social media to publish negative comments on published research. This doesn't at all match Fiske's article. As people on HN know very well, there's a difference between, on one hand, serious, constructive criticism; and on the other, the trolling, abuse, and just endless barrage of nonsense that buries any signal in noise, all of which is common online. It's clear Fiske…

If she's complaining about trolling and abuse, she should probably say so, and not use terms like "methodological terrorism". That is a) a rather unprofessional accusation and b) a term that brings to mind scientific criticism , not abuse.

> If she's complaining about trolling and abuse, she should probably say so

She does, clearly and explicitly, and I even quoted some of it.

Re: What has happened down here is the winds have changed

#114
Who is paying for all this bad science? Do they even have a choice in what they are funding?

We have so many "industries" now whose function is to exploit the taxpayer. We wouldn't have all this bad science or bad risk taking if there weren't a huge patsy willing to pay for it all without regard to actual results.

Re: What has happened down here is the winds have changed

#115
post #61

It's good that these issues finally get a bit more attention, however I fear that we're still only seeing the tip of the iceberg. There are so many fields of science that don't care about replication at all, work with far too low numbers of subjects and don't bother about issues like publication bias. I don't think it's a stretch to say that the majority of stuff that's done at university is cargo cult science. And d…

That's interesting. What kind of computer science research is done with control/experimental groups of users? I haven't paid any attention to academic computer science in, well, decades, but I kind of assumed it was all "we got this algorithm to do this thing," like math, with not much of a human element.

Re: What has happened down here is the winds have changed

#116
Psychology as a science has far fewer reliable results than it claimed it did. It's not that the winds have changed. It's that the field was broken.

Realizing this is a good thing. It's going to set back careers and reduce funding, though.

Next, economics?

"Science is prediction, not explanation." - Fred Hoyle

Re: What has happened down here is the winds have changed

#117
post #94

Earlier quoted context omitted.

They should welcome online trolling, abuse, and personal attacks? Nobody should welcome it, but -- as it comes with the territory -- they need to ignore it. She works on what is historically and cross-culturally arguably the most controversial subject, namely human sexuality. Attacks are to be expected, and the mature reaction to this is to lead by example: ignore the attacks and be as meticulously scientific as poss…

> it comes with the territory I disagree; that's an excuse for abusive, unproductive behavior. In many environments, people manage to criticize ideas respectfully and productively, even on social forums like HN. Abuse is counter-productive and completely unnecessary.

   I disagree;
What do you disagree with? The factual claim?

It's not relevant that some manage to discuss sexuality respectfully and productively. Most can't, as of October 2016.

Claims of scientific misconduct (in a general sense) ought to be taken seriously by scientists. There is nothing else to say.

What alternative do you suggest? Somebody said mean things on Twitter, hence psychologists like Fiske should continue as before?

Re: What has happened down here is the winds have changed

#118
post #87
post #59

Good scientists should welcome this challenge with open arms, sunlight all of their data and code in a repository, and show how proper statistically rigorous research is conducted. That's an elevator right to the top of the field. If I wrote code that messed up, I'd patch it. If someone else points out a bug? Even better, now I don't have to find it myself. But I would definitely not feel attacked or nor would I blam…

"sunlight all of their data and code in a repository" Good idea where possible, but "all their data" will be problematic in many social science studies, and also in medicine, because of privacy laws.

Also, in many cases in the physical sciences, the data sets (both from experiments and simulations) may simply be too big for long term storage, let alone sharing. CERN is pushing the envelope at around 1 TB new shared data per day, but projects run by groups as small as five people can easily generate tens of terabytes in a day. E.g. with turbulence simulations or tomographic PIV or even just multiple high-speed cameras.

Re: What has happened down here is the winds have changed

#119
post #59

Good scientists should welcome this challenge with open arms, sunlight all of their data and code in a repository, and show how proper statistically rigorous research is conducted. That's an elevator right to the top of the field. If I wrote code that messed up, I'd patch it. If someone else points out a bug? Even better, now I don't have to find it myself. But I would definitely not feel attacked or nor would I blam…

> sunlight all of their data and code This is not always possible. For example, if you are working with proprietary subscription-only datasets where the researcher has no redistribution rights.

Then the provider of the subscription must be able to vouch for it or that data should not be used. The provider should also be .... sensitive to the need for that data to be used well, for any conclusion drawn from it to be reproducible.

Re: What has happened down here is the winds have changed

#120

Psychology as a science has far fewer reliable results than it claimed it did. It's not that the winds have changed. It's that the field was broken. Realizing this is a good thing. It's going to set back careers and reduce funding, though. Next, economics? "Science is prediction, not explanation." - Fred Hoyle

A lot of the blog post was explaining what's happened in psychology in the past 10 years (and you're right - it turns out a lot of results aren't reliable), but I think "the winds have changed" is specifically getting at well-known people like Fiske who're clinging on to the past. The winds have changed in the sense that calling her out even a few years ago would have been unusual, possibly risky for a career. Now the author can come close to accusing her of being unscientific and have a lot of people agree with him.

(I got all that from the article. I'm probably wrong)

Post reply on HN