Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A bot crawled thousands of studies looking for simple math errors

vox.com

Post reply on HN
Print view View original

141–150 of 221 posts

Re: A bot crawled thousands of studies looking for simple math errors

#141

Post by jessaustin »

disgruntledphd2 wrote: post #106
c3534l wrote: post #34

> and whether online critiques of past work constitute “bullying” or shaming. The PubPeer comments are just a tiny part of that debate. But it can hit a nerve. > Susan Fiske, a former president of the Association for Psychological Science, alluded to Statcheck in an interview with Business Insider, calling it a “gotcha algorithm.” > The draft of the article said these online critics were engaging in “methodological t…

You are entirely correct (p But apparently we also have Susan Fiske. I think that the real difficulty with psychology is that people care too much. Its a little easier to be objective around the decay of atomic particles than about the intelligence of different genders and races. Additionally, many psychologists are not comfortable with statistics. A large proportion went into psychology to avoid maths, and are somew…

An interesting recent complaint about Fiske doesn't entirely spare Kahneman:

http://andrewgelman.com/2016/09/21/what-has-happened-down-he...

Re: A bot crawled thousands of studies looking for simple math errors

#142

Post by wyager »

dahart wrote: post #122
int_19h wrote: post #107

Earlier quoted context omitted.

So they shouldn't run it on the old papers, so as to not offend anyone's sensibilities? Do we just declare said papers completely useless then? Or do they keep getting cited by new papers, and used to guide policy? If the latter, then not vetting them using the newer and better methodology would be unethical.

> If the latter, then not vetting them using the newer and better methodology would be unethical. On the contrary, it would be unethical to hold people to new standards that didn't exist when the work was done. If you have a beer and then next week the laws change the drinking age to 65 years old, should you go to jail? This is simply not how things are done, in science or in society. Laws and standards change all th…

> new standards that didn't exist when the work was done

Being correct is not a new standard.

> This is simply not how things are done, in science or in society.

Oh, I say! Positively indecent! A moral outrage! We can't have our morals compromised by scientific objectivity!

> we do not and will never revisit all prior work and formally re-judge whenever standards or policies change.

Have you ever heard of Principia Mathematica?

> nothing can be done about the old papers

Except to try and figure out when they're wrong, as this bot is doing.

Re: A bot crawled thousands of studies looking for simple math errors

#143

Post by serge2k »

> some found the emails annoying from PubPeer [since PubPeer notifies authors of comments] if Statcheck found no potential errors

I would.

> There’s a big, uncomfortable question of how to criticize past work, and whether online critiques of past work constitute “bullying” or shaming.

It's facts about your work. Learn to handle it or quit pretending to be a scientist.

> The gist of her article was that she feared too much of the criticism of past work had been ceded to social media, and that the criticism there is taking on an uncivil tone

Valid enough point. Criticism and correction can be done in a civil manner, and in an accepted forum.

Re: A bot crawled thousands of studies looking for simple math errors

#144

Post by taneq »

lutusp wrote: post #138
c3534l wrote: post #34

> and whether online critiques of past work constitute “bullying” or shaming. The PubPeer comments are just a tiny part of that debate. But it can hit a nerve. > Susan Fiske, a former president of the Association for Psychological Science, alluded to Statcheck in an interview with Business Insider, calling it a “gotcha algorithm.” > The draft of the article said these online critics were engaging in “methodological t…

> If these are attitudes typical of psychology, then I cannot say I consider psychology to be a proper social science. In my experience, such attitudes are typical. Here's a 2014 paper by a Harvard psychology professor who tries to argue that null replication results shouldn't be taken seriously because the failure to replicate ipso facto indicts the study's methods: http://jasonmitchell.fas.harvard.edu/Papers/Mitche…

Wow, so many responses that come to mind and they're all of the form 'un' + some swearing + 'scientific'.

Re: A bot crawled thousands of studies looking for simple math errors

#146

Post by 6stringmerc »

hueving wrote: post #130
6stringmerc wrote: post #19

What a clever and, dare I say it, fantastically useful experiment! So much less harm than even "door knob twisting" type explorations - no, this was using published works and pretty much running them through a process to verify or not verify accuracy. Unsolicited? So what! As a practiced writer I make unsolicited judgments on language usage all the time. Are these people that completely write from their own minds and…

Putting grammar on the same level as this is wrong. Poor grammar and spelling doesn't frequently change the entire results of a paper. Math errors though can easily be the difference between significance and insignificance, changing the entire set of conclusions that can be drawn from the paper.

Uh, no, poor grammar or words or phrases used improperly can genuinely have cascade / domino effects. Brush up on "Eats, Shoots, and Leaves" if you wish to counter-balance your math-centric logic notions with how similar principles apply in written language. English is, without question, one of the more tricky (re: confusing) mutts of linguistics.

Re: A bot crawled thousands of studies looking for simple math errors

#147

Post by dahart »

wyager wrote: post #142
dahart wrote: post #122

Earlier quoted context omitted.

> If the latter, then not vetting them using the newer and better methodology would be unethical. On the contrary, it would be unethical to hold people to new standards that didn't exist when the work was done. If you have a beer and then next week the laws change the drinking age to 65 years old, should you go to jail? This is simply not how things are done, in science or in society. Laws and standards change all th…

> new standards that didn't exist when the work was done Being correct is not a new standard. > This is simply not how things are done, in science or in society. Oh, I say! Positively indecent! A moral outrage! We can't have our morals compromised by scientific objectivity! > we do not and will never revisit all prior work and formally re-judge whenever standards or policies change. Have you ever heard of Principia M…

Please try to judge comments in their context and not look for excuses to try and humiliate someone with sarcasm. I was discussing standards with @int_19h, who brought up the issue of new methodologies. Of course being correct isn't a new standard, I agree with you, but that's not what was being discussed.

Of course morals shouldn't be compromised by scientific objectivity. Again, you're arguing a straw man - that's not the issue I was talking about.

I have stated multiple times, including my first reply to you, that I think the bot is fine. My argument in context is that a paper cannot change because it has been published. Do you disagree with that? That doesn't have any bearing on whether bots or people find & publish errors later. It does have a bearing on how people will respond to PR campaigns to publish errors when nothing can be done about it on the part of the author. Statcheck will do good things for authors who get to use it before they publish rather than after.

Maybe you're not reading all of what I wrote? Maybe I hurt your feelings?

Re: A bot crawled thousands of studies looking for simple math errors

#148

Post by cylinder »

c3534l wrote: post #34

> and whether online critiques of past work constitute “bullying” or shaming. The PubPeer comments are just a tiny part of that debate. But it can hit a nerve. > Susan Fiske, a former president of the Association for Psychological Science, alluded to Statcheck in an interview with Business Insider, calling it a “gotcha algorithm.” > The draft of the article said these online critics were engaging in “methodological t…

There go Americans throwing around the terrorism word again. https://news.ycombinator.com/item?id=12620132

Re: A bot crawled thousands of studies looking for simple math errors

#149

Post by zerocrates »

cushychicken wrote: post #131
6stringmerc wrote: post #19

What a clever and, dare I say it, fantastically useful experiment! So much less harm than even "door knob twisting" type explorations - no, this was using published works and pretty much running them through a process to verify or not verify accuracy. Unsolicited? So what! As a practiced writer I make unsolicited judgments on language usage all the time. Are these people that completely write from their own minds and…

I think "bullying" is a reductive term for what's actually happening. I think the point trying to be conveyed is that there is a certain decline in the author's personal status when they get holes punched in their publications.

Sounds like a nice incentive to double-check before publishing.

Re: A bot crawled thousands of studies looking for simple math errors

#150

Post by ramchip »

lutusp wrote: post #138
c3534l wrote: post #34

> and whether online critiques of past work constitute “bullying” or shaming. The PubPeer comments are just a tiny part of that debate. But it can hit a nerve. > Susan Fiske, a former president of the Association for Psychological Science, alluded to Statcheck in an interview with Business Insider, calling it a “gotcha algorithm.” > The draft of the article said these online critics were engaging in “methodological t…

> If these are attitudes typical of psychology, then I cannot say I consider psychology to be a proper social science. In my experience, such attitudes are typical. Here's a 2014 paper by a Harvard psychology professor who tries to argue that null replication results shouldn't be taken seriously because the failure to replicate ipso facto indicts the study's methods: http://jasonmitchell.fas.harvard.edu/Papers/Mitche…

> Someone who publishes a replication is, in effect, saying something like,“You found an effect. I did not. One of us is the inferior scientist.”

Wow... I really don't see it like that.

Post reply on HN