Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A bot crawled thousands of studies looking for simple math errors

vox.com

Post reply on HN
Print view View original

11–20 of 221 posts

Re: A bot crawled thousands of studies looking for simple math errors

#11

Post by Practicality »

merpnderp wrote: post #7
alanfalcon wrote: post #3

I find it very disconcerting that people are trying to fend off criticism of previously published studies by calling it "bullying" or sometimes worse. What do feelings have to do with science?

Objectivity is listed as a problem with science curriculum. Welcome to the future. http://nsuworks.nova.edu/cgi/viewcontent.cgi?article=2467&co...

While I am not a big fan of contemporary feminism, I am pretty sure taking issue with masculinity or "gendered" text is not the same thing as taking issue with objectivity.

Edit: Found this quote in your source "Poststructuralism “rejects objectivity and the notions of an absolute truth and single reality,”"

Hmm, this seems an awful lot like a religion.

Re: A bot crawled thousands of studies looking for simple math errors

#12

Post by hoopism »

nkrisc wrote: post #6
mratzloff wrote: post #5

Earlier quoted context omitted.

Right. If you can't handle critique (or embarrassment from invalidated conclusions), don't publish. Write in your diary instead.

Exactly. This isn't even subjective critique, it's math. Your math is right or it's wrong. Lastly from what I can tell there isn't even any value judgement attached, simply a notification of incorrect math.

This was my thought exactly.

The most disturbing aspect to this whole deal is the response of the administrators who I assume represent a good deal of the community.

The researches who contributed this project ought to be thanked and rewarded for their participation... not called terrorists.

Re: A bot crawled thousands of studies looking for simple math errors

#13

Post by michaelt »

I can certainly understand people being nervous about academic debate moving to social media. It would be a hassle for climate scientists if every paper got a brigade of climate change deniers criticising it and you had to respond to those criticisms.

But this example - someone notifying you there's a mistake in your paper, when there really is a mistake? That seems like a strong argument /for/ academic debate via social media, not /against/ it.

Re: A bot crawled thousands of studies looking for simple math errors

#14

Post by dahart »

snerbles wrote: post #10
mratzloff wrote: post #4

Here's the GitHub page: https://github.com/MicheleNuijten/statcheck And if you're curious how it works, as I was: Statcheck uses regular expressions to find statistical results in APA format. When a statistical result deviates from APA format, statcheck will not find it. The APA formats that statcheck uses are: t(df) = value, p = value; F(df1,df2) = value, p = value; r(df) = value, p = value; [chi]2 (df, N = value) =…

> When a statistical result deviates from APA format, statcheck will not find it. Incentive for authors to obfuscate their math?

Pretty sure APA format is required for APA journals, but either way, fortunately, the desire on the part of authors to get papers published and on the part of reviewers to have standards and clear presentation outweighs anyone's desire to game statcheck.

But you do make a great point in that statcheck could and should red-line papers that present no discernible stats, and provide links to the APA style guide!

Re: A bot crawled thousands of studies looking for simple math errors

#15

Post by dahart »

alanfalcon wrote: post #3

I find it very disconcerting that people are trying to fend off criticism of previously published studies by calling it "bullying" or sometimes worse. What do feelings have to do with science?

When I read that sentence, I immediately thought it sounded like some people were being childish and defensive.

OTOH, a bunch of people got unsolicited error reports for already published papers, and I can understand being initially irritated. This was, by the admission of the statcheck authors, a way to get attention quickly.

My guess is it'll all settle down and people won't complain about bullying once everyone just uses statcheck like they do spell-check.

To answer your question though, I think feelings have everything to do with science. There are many reasons different people do science, and they all stem from emotions. Some people do science out of curiosity, to solve mysteries. Some do science as a means to an end, to gain knowledge required to further some other goal. Some do science to gain social standing & intellectual superiority. Some do science to help others understand how the world works. In all cases, the reasons people are doing science is because of a want, some kind of desire to achieve a goal. Nature and physics will continue to exist whether we explore it or not, we do so because we care, and caring is a feeling.

Re: A bot crawled thousands of studies looking for simple math errors

#16

Post by lucb1e »

WillAbides wrote: post #8
alanfalcon wrote: post #3

I find it very disconcerting that people are trying to fend off criticism of previously published studies by calling it "bullying" or sometimes worse. What do feelings have to do with science?

> What do feelings have to do with science? When the science in question is psychology, feelings have everything to do with it.

Studying feelings does not make feelings a part of studying.

Re: A bot crawled thousands of studies looking for simple math errors

#18

Post by schmichael »

snerbles wrote: post #10
mratzloff wrote: post #4

Here's the GitHub page: https://github.com/MicheleNuijten/statcheck And if you're curious how it works, as I was: Statcheck uses regular expressions to find statistical results in APA format. When a statistical result deviates from APA format, statcheck will not find it. The APA formats that statcheck uses are: t(df) = value, p = value; F(df1,df2) = value, p = value; r(df) = value, p = value; [chi]2 (df, N = value) =…

> When a statistical result deviates from APA format, statcheck will not find it. Incentive for authors to obfuscate their math?

It'd be easier for them to not obfuscate and use the checker themselves prior to publishing.

Re: A bot crawled thousands of studies looking for simple math errors

#19

Post by 6stringmerc »

What a clever and, dare I say it, fantastically useful experiment!

So much less harm than even "door knob twisting" type explorations - no, this was using published works and pretty much running them through a process to verify or not verify accuracy.

Unsolicited? So what! As a practiced writer I make unsolicited judgments on language usage all the time. Are these people that completely write from their own minds and don't use a spell check or grammar check program of any sort before sending their material for editorial review? I'd strongly doubt it, because it's a tool to make communication more accurate. Math and formulas having a similar procedural check sounds quite constructive to me.

It's not bullying to point out errors; it's bullying to use the existence of errors to belittle or insult a person. I don't see that happening here. Sure, it's a little sterile or "cold" in this fashion, but I think that's for the best if such a process / tool can gain acceptance. It just spits out results and I think that's all it should do. Neat to read about.

Re: A bot crawled thousands of studies looking for simple math errors

#20

Post by icantdrive55 »

dahart wrote: post #15
alanfalcon wrote: post #3

I find it very disconcerting that people are trying to fend off criticism of previously published studies by calling it "bullying" or sometimes worse. What do feelings have to do with science?

When I read that sentence, I immediately thought it sounded like some people were being childish and defensive. OTOH, a bunch of people got unsolicited error reports for already published papers, and I can understand being initially irritated. This was, by the admission of the statcheck authors, a way to get attention quickly. My guess is it'll all settle down and people won't complain about bullying once everyone ju…

And there's people/corporations/institutions who do science to further their agenda.

This is a huge part of science. I wish it wasn't.

I welcome any program that looks for errors. Now let's expose the bad studies, or bad science?

I welcome any program that makes this job easier.

I wish I wasen't so cynical towards science. I have just been duped too many times.

Post reply on HN