Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What has happened down here is the winds have changed

andrewgelman.com

Post reply on HN
Print view View original

91–100 of 187 posts

Re: What has happened down here is the winds have changed

#91

Post by Someone »

wyager wrote: post #90
Someone wrote: post #87

Earlier quoted context omitted.

"sunlight all of their data and code in a repository" Good idea where possible, but "all their data" will be problematic in many social science studies, and also in medicine, because of privacy laws.

Anonymize the data then, if plausible. We ought not to trust studies that are inherently irreproducible.

It's practically impossible to anonymize data without throwing away the information contained in it, doubly so if you don't know at the time what questions critics will want to see answered on that data.

Re: What has happened down here is the winds have changed

#92

Post by wyager »

hackuser wrote: post #77
s_q_b wrote: post #37

First, I don't understand why rapid feedback is bad. Communication adapts to new mediums all the time, and academia goes with it. The arXiv pre-print system is a great example of a field adapting well to a new technology. Secondly, addressing the article's main point, I'm being told that social media comments about their studies are so traumatic and detrimental that they leave their careers. That's a problem with the…

> I don't understand why rapid feedback is bad She doesn't say rapid feedback is bad. She says online personal attacks and abuse are bad.

You are falling for a classic argumentative attack. Everyone agrees "abuse" (e.g. beating your wife) is bad, so Fisk just redefines "abuse" to mean "earnest, blunt, non-sugar-coated feedback" but you keep on using the connotations of the real definition.

Re: What has happened down here is the winds have changed

#93

Post by mikeklaas »

botexpert wrote: post #70
tgarma1234 wrote: post #54

I was a big fan of "Thinking fast, thinking slow", like pretty much everyone, until the replication crisis came along. Now I see a lot of the work in this field as a sort of scientifically informed stereotyping that might someday end up on par with phrenology, but it's still fun to read and consider as if it were true. It's really awful how pseudo-science can get so far though.

Did it turn out that research in the book is flawed or not reproducible?

Yes, some of the studies haven't been reproducible. I don't have references right now, unfortunately.

Re: What has happened down here is the winds have changed

#94

Post by mafribe »

hackuser wrote: post #75
s_q_b wrote: post #59

Good scientists should welcome this challenge with open arms, sunlight all of their data and code in a repository, and show how proper statistically rigorous research is conducted. That's an elevator right to the top of the field. If I wrote code that messed up, I'd patch it. If someone else points out a bug? Even better, now I don't have to find it myself. But I would definitely not feel attacked or nor would I blam…

> Good scientists should welcome this challenge with open arms They should welcome online trolling, abuse, and personal attacks? That is what Fiske criticizes in her article, not serious criticism and debate, which she explicitly welcomes and encourages.

   They should welcome online trolling, 
   abuse, and personal attacks?
Nobody should welcome it, but -- as it comes with the territory -- they need to ignore it.

She works on what is historically and cross-culturally arguably the most controversial subject, namely human sexuality. Attacks are to be expected, and the mature reaction to this is to lead by example: ignore the attacks and be as meticulously scientific as possible in the hope of inspiring others to be similarly mature. All the more so if you work in a field (psychology) that has a long history of unsupported claims and lack of methodological rigour.

The reason science works, the reason we can fly to Mars, have a supercomputer in everybody's pocket today, can treat many diseases, is because sufficiently many adhere to a rigorous scientific methodology and every scientist is extremely critical of colleagues who don't. It is incumbent upon all scientists to hold themselves and others up to the highest standards.

Computer science grew out of worries about the lack of rigor in the foundations of mathematics,triggered by the (re-)discovery of non-Euclidian geometry. Mathematics, the most rigorous of all scientific fields, is currently beginning to move towards formally verified proofs. To be even more reliable.

Re: What has happened down here is the winds have changed

#95

Post by WalterSear »

hackuser wrote: post #77
s_q_b wrote: post #37

First, I don't understand why rapid feedback is bad. Communication adapts to new mediums all the time, and academia goes with it. The arXiv pre-print system is a great example of a field adapting well to a new technology. Secondly, addressing the article's main point, I'm being told that social media comments about their studies are so traumatic and detrimental that they leave their careers. That's a problem with the…

> I don't understand why rapid feedback is bad She doesn't say rapid feedback is bad. She says online personal attacks and abuse are bad.

http://slatestarcodex.com/2014/11/03/all-in-all-another-bric...

Re: What has happened down here is the winds have changed

#96

Post by jsprogrammer »

wyager wrote: post #92
hackuser wrote: post #77

Earlier quoted context omitted.

> I don't understand why rapid feedback is bad She doesn't say rapid feedback is bad. She says online personal attacks and abuse are bad.

You are falling for a classic argumentative attack. Everyone agrees "abuse" (e.g. beating your wife) is bad, so Fisk just redefines "abuse" to mean "earnest, blunt, non-sugar-coated feedback" but you keep on using the connotations of the real definition.

Where can I find the redefinition?

Re: What has happened down here is the winds have changed

#97

Post by wgyn »

Someone wrote: post #91
wyager wrote: post #90

Earlier quoted context omitted.

Anonymize the data then, if plausible. We ought not to trust studies that are inherently irreproducible.

It's practically impossible to anonymize data without throwing away the information contained in it, doubly so if you don't know at the time what questions critics will want to see answered on that data.

That's not quite true. E.g. Google's email spam models are trained without access to the raw text. In general the field of differential privacy is developing rather quickly.

Re: What has happened down here is the winds have changed

#98

Post by conistonwater »

hackuser wrote: post #75
s_q_b wrote: post #59

Good scientists should welcome this challenge with open arms, sunlight all of their data and code in a repository, and show how proper statistically rigorous research is conducted. That's an elevator right to the top of the field. If I wrote code that messed up, I'd patch it. If someone else points out a bug? Even better, now I don't have to find it myself. But I would definitely not feel attacked or nor would I blam…

> Good scientists should welcome this challenge with open arms They should welcome online trolling, abuse, and personal attacks? That is what Fiske criticizes in her article, not serious criticism and debate, which she explicitly welcomes and encourages.

I think you are missing the point of both the blog post and Fiske's own article. What she labels as "personal attacks" was, in fact, legitimate scientific criticism, and she tries to dismiss it based on it not being published according to the right norms (according to her). She's trying to bring accusations of trolling and abuse, and you are taking her word for it as if that is the right description of what has been happening so far. She doesn't "explicitly welcome" serious criticism, because criticism published in scientific journals is rather seriously flawed and ineffective (too slow, too strong an incentive to be biased in favour of already published papers: see, for example, the discussions and failed replications that arose from Bem's ESP paper, that was really terrible). This is consistent with her comment about "methodological terrorism", which is as wrong-headed as can be, and I think clear evidence that it's not about "online abuse" in the usual sense.

If you read Gelman's posts (going back quite a while), for example, you'll see that the criticism that Fiske dismisses as trolling and abuse is nothing of the sort, and being published on a blog does not make it less valid.

In the end, it is part of the culture of science and part of being a good scientist that one should be willing to accept criticism, and maybe refutations, of one's own work, so trying to get above it by claiming "personal attacks" is seriously bad form.

P.S. There was another example of this kind of an "attack" when somebody wrote a bot for finding statistics errors: https://news.ycombinator.com/item?id=12643978

Re: What has happened down here is the winds have changed

#99

Post by vitno »

wgyn wrote: post #97
Someone wrote: post #91

Earlier quoted context omitted.

It's practically impossible to anonymize data without throwing away the information contained in it, doubly so if you don't know at the time what questions critics will want to see answered on that data.

That's not quite true. E.g. Google's email spam models are trained without access to the raw text. In general the field of differential privacy is developing rather quickly.

I'm somewhat confused about how they would be doing that. Do you have any references to blogs/papers on this technique?

Re: What has happened down here is the winds have changed

#100

Post by eveningcoffee »

nickff wrote: post #84
Cerium wrote: post #81

Earlier quoted context omitted.

If the community demands the ability to review code and data then the usage of closed data sets will simply decrease until they comply. We should not base our standards of research on the ease of complying. We should do what we can to produce the most accurate results.

The problem is not one of convenience, the problem is that you are depriving yourself of huge amounts of valuable (closed) data, and this may cause large gaps in what is researchable.

This was the point. You have to pay the price for the principles and hope that the tide turns one day.

If you take your suggested easy route then this day may never come.

Post reply on HN