Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What has happened down here is the winds have changed

andrewgelman.com

Post reply on HN
Print view View original

81–90 of 187 posts

Re: What has happened down here is the winds have changed

#81

Post by Cerium »

jgalt212 wrote: post #63
s_q_b wrote: post #59

Good scientists should welcome this challenge with open arms, sunlight all of their data and code in a repository, and show how proper statistically rigorous research is conducted. That's an elevator right to the top of the field. If I wrote code that messed up, I'd patch it. If someone else points out a bug? Even better, now I don't have to find it myself. But I would definitely not feel attacked or nor would I blam…

> sunlight all of their data and code This is not always possible. For example, if you are working with proprietary subscription-only datasets where the researcher has no redistribution rights.

If the community demands the ability to review code and data then the usage of closed data sets will simply decrease until they comply. We should not base our standards of research on the ease of complying. We should do what we can to produce the most accurate results.

Re: What has happened down here is the winds have changed

#82

Post by rwallace »

jgalt212 wrote: post #63
s_q_b wrote: post #59

Good scientists should welcome this challenge with open arms, sunlight all of their data and code in a repository, and show how proper statistically rigorous research is conducted. That's an elevator right to the top of the field. If I wrote code that messed up, I'd patch it. If someone else points out a bug? Even better, now I don't have to find it myself. But I would definitely not feel attacked or nor would I blam…

> sunlight all of their data and code This is not always possible. For example, if you are working with proprietary subscription-only datasets where the researcher has no redistribution rights.

Then it should be required whenever and to the extent it is possible. And where it is not possible, we should be considerably more skeptical of claims made, because trusting a claim based on a proprietary dataset requires a hefty dollop of faith.

Re: What has happened down here is the winds have changed

#83

Post by mcguire »

jgalt212 wrote: post #63
s_q_b wrote: post #59

Good scientists should welcome this challenge with open arms, sunlight all of their data and code in a repository, and show how proper statistically rigorous research is conducted. That's an elevator right to the top of the field. If I wrote code that messed up, I'd patch it. If someone else points out a bug? Even better, now I don't have to find it myself. But I would definitely not feel attacked or nor would I blam…

> sunlight all of their data and code This is not always possible. For example, if you are working with proprietary subscription-only datasets where the researcher has no redistribution rights.

And such data sets should be trusted...why?

Re: What has happened down here is the winds have changed

#84

Post by nickff »

Cerium wrote: post #81
jgalt212 wrote: post #63

Earlier quoted context omitted.

> sunlight all of their data and code This is not always possible. For example, if you are working with proprietary subscription-only datasets where the researcher has no redistribution rights.

If the community demands the ability to review code and data then the usage of closed data sets will simply decrease until they comply. We should not base our standards of research on the ease of complying. We should do what we can to produce the most accurate results.

The problem is not one of convenience, the problem is that you are depriving yourself of huge amounts of valuable (closed) data, and this may cause large gaps in what is researchable.

Re: What has happened down here is the winds have changed

#85

Post by mcguire »

hackuser wrote: post #72

> In short, Fiske doesn’t like when people use social media to publish negative comments on published research. This doesn't at all match Fiske's article. As people on HN know very well, there's a difference between, on one hand, serious, constructive criticism; and on the other, the trolling, abuse, and just endless barrage of nonsense that buries any signal in noise, all of which is common online. It's clear Fiske…

If she's complaining about trolling and abuse, she should probably say so, and not use terms like "methodological terrorism". That is a) a rather unprofessional accusation and b) a term that brings to mind scientific criticism , not abuse.

Re: What has happened down here is the winds have changed

#86

Post by sbmassey »

forgotpwtomain wrote: post #47
Kenji wrote: post #27

Oh, free speech and criticism is okay as long as it's moderated and appropriate... Well played. The work of psychology researchers influences our LAWS. They influence how I can live my life. It's a very personal thing. I will criticise them however I want and as much as I want until this branch of 'science' has cleaned up its conduct. If your work cannot withstand criticism, it's worthless. Get out of science, Susan…

> The work of psychology researchers influences our LAWS. They influence how I can live my life. It's a very personal thing. While I agree with the above sentiment: > If your work cannot withstand criticism, it's worthless. Get out of science, Susan T. Fiske, you give it a bad name. I think the tone is very much incorrect. While researchers should be responsible for bad science, you may it sound like it was necessari…

Does the intent matter? Even if it was just a career-long run of incompetence, "getting out of science" would seem a reasonable starting point.

Re: What has happened down here is the winds have changed

#87

Post by Someone »

s_q_b wrote: post #59

Good scientists should welcome this challenge with open arms, sunlight all of their data and code in a repository, and show how proper statistically rigorous research is conducted. That's an elevator right to the top of the field. If I wrote code that messed up, I'd patch it. If someone else points out a bug? Even better, now I don't have to find it myself. But I would definitely not feel attacked or nor would I blam…

"sunlight all of their data and code in a repository"

Good idea where possible, but "all their data" will be problematic in many social science studies, and also in medicine, because of privacy laws.

Re: What has happened down here is the winds have changed

#88

Post by unimpressive »

jgalt212 wrote: post #63
s_q_b wrote: post #59

Good scientists should welcome this challenge with open arms, sunlight all of their data and code in a repository, and show how proper statistically rigorous research is conducted. That's an elevator right to the top of the field. If I wrote code that messed up, I'd patch it. If someone else points out a bug? Even better, now I don't have to find it myself. But I would definitely not feel attacked or nor would I blam…

> sunlight all of their data and code This is not always possible. For example, if you are working with proprietary subscription-only datasets where the researcher has no redistribution rights.

The problem is that a lot of studies don't suffer from this problem, and their data/code isn't open either. What we're looking at is a default-closed mindset, which needs to be inverted ASAP if these people want us to trust them in any significant capacity.

Re: What has happened down here is the winds have changed

#89

Post by wyager »

hackuser wrote: post #75
s_q_b wrote: post #59

Good scientists should welcome this challenge with open arms, sunlight all of their data and code in a repository, and show how proper statistically rigorous research is conducted. That's an elevator right to the top of the field. If I wrote code that messed up, I'd patch it. If someone else points out a bug? Even better, now I don't have to find it myself. But I would definitely not feel attacked or nor would I blam…

> Good scientists should welcome this challenge with open arms They should welcome online trolling, abuse, and personal attacks? That is what Fiske criticizes in her article, not serious criticism and debate, which she explicitly welcomes and encourages.

Have you seen what passes for "online abuse" right now? The fact that Fisk earnestly uses the phrase "methodological terrorism" suggests that she is not to be trusted with negative connotations.

Everyone agrees "abuse" (connotation: wife beating) is bad, so people like Fisk just redefine "abuse" to mean "being blunt online", and people who aren't inoculated against semantic overloading techniques fall for it.

Re: What has happened down here is the winds have changed

#90

Post by wyager »

Someone wrote: post #87
s_q_b wrote: post #59

Good scientists should welcome this challenge with open arms, sunlight all of their data and code in a repository, and show how proper statistically rigorous research is conducted. That's an elevator right to the top of the field. If I wrote code that messed up, I'd patch it. If someone else points out a bug? Even better, now I don't have to find it myself. But I would definitely not feel attacked or nor would I blam…

"sunlight all of their data and code in a repository" Good idea where possible, but "all their data" will be problematic in many social science studies, and also in medicine, because of privacy laws.

Anonymize the data then, if plausible. We ought not to trust studies that are inherently irreproducible.
Post reply on HN