Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A bot crawled thousands of studies looking for simple math errors

vox.com

Post reply on HN
Print view View original

61–70 of 221 posts

Re: A bot crawled thousands of studies looking for simple math errors

#61

Post by wyager »

cafebeen wrote: post #42
wyager wrote: post #29

Earlier quoted context omitted.

If someone objectively screwed up a published study, there's no room for feelings or social consideration. People might be wasting their precious time and money pursuing impossible goals based on the incorrect study. Let's say someone finds a mistake in a math paper. Do they politely ask the author to fix it? What happens if the author ignores them? The scientific community doesn't have time to coddle and pursue ever…

It's not coddling as much as professional courtesy. Mistakes are often brought up at conferences and in peer review, but most scientists within a specific research area try to be on good terms with one another and don't see value in publicly shaming colleagues for their mistakes.

There are two options here:

Use a bot to find tons of mistakes automatically and risk coming across as rude.

Or

Let social trivialities dominate scientific discourse and let most of those mistakes go unchecked forever because there's no feasible way to "politely" address hundreds of authors who made mistakes and keep checking back to make sure they actually fixed the mistake.

The former is clearly the preferable choice. Some individual scientists will suffer for it, but the scientific community as a whole will benefit greatly.

Re: A bot crawled thousands of studies looking for simple math errors

#62

Post by nitrogen »

nicops wrote: post #56
SilasX wrote: post #52

Earlier quoted context omitted.

Yes! Similarly, I remember a few years back being shocked at the attitude toward the replication crisis, where authors of irreproducible work were miffed that no one talked to them first. I was like, "Um, the whole point of science is that it's knowledge in the public record . If you need to 'chill' with the original researcher to get critical details, that's not science anymore." Remember, the studies might form the…

> If you need to a 'chill' with the original researcher to get critical details, that's not science anymore. Certainly there is always the possibility that there was some details misunderstood, something that needs to be clarified, a print error, etc. Your "that's not science anymore" statement seems highly exaggerated. People are not supposed to communicate only via papers.

The usefulness of a paper that doesn't stand on its own is rather limited, though.

Re: A bot crawled thousands of studies looking for simple math errors

#63

Post by danharaj »

wyager wrote: post #61
cafebeen wrote: post #42

Earlier quoted context omitted.

It's not coddling as much as professional courtesy. Mistakes are often brought up at conferences and in peer review, but most scientists within a specific research area try to be on good terms with one another and don't see value in publicly shaming colleagues for their mistakes.

There are two options here: Use a bot to find tons of mistakes automatically and risk coming across as rude. Or Let social trivialities dominate scientific discourse and let most of those mistakes go unchecked forever because there's no feasible way to "politely" address hundreds of authors who made mistakes and keep checking back to make sure they actually fixed the mistake. The former is clearly the preferable choi…

Science being a social activity, I think you have to justify the presumption that we are talking about social trivialities here.

Re: A bot crawled thousands of studies looking for simple math errors

#64

Post by mturmon »

mratzloff wrote: post #4

Here's the GitHub page: https://github.com/MicheleNuijten/statcheck And if you're curious how it works, as I was: Statcheck uses regular expressions to find statistical results in APA format. When a statistical result deviates from APA format, statcheck will not find it. The APA formats that statcheck uses are: t(df) = value, p = value; F(df1,df2) = value, p = value; r(df) = value, p = value; [chi]2 (df, N = value) =…

Very helpful -- this points out that this is a very focused tool on simple calculations of test statistics. They appear in papers formatted very strictly, like this:

  t(37) = −4.93, p
or:

  χ2(1, N = 226) = 6.90, p
Because of the strict format and the limited scope of the tool, one would suspect the false positive rate is very low. And because it's just a simple calculation (not a matter of interpretation), authors should not (in theory) be offended by getting a notice.

The link to their paper is: http://link.springer.com/article/10.3758%2Fs13428-015-0664-2

Re: A bot crawled thousands of studies looking for simple math errors

#65

Post by SilasX »

nicops wrote: post #56
SilasX wrote: post #52

Earlier quoted context omitted.

Yes! Similarly, I remember a few years back being shocked at the attitude toward the replication crisis, where authors of irreproducible work were miffed that no one talked to them first. I was like, "Um, the whole point of science is that it's knowledge in the public record . If you need to 'chill' with the original researcher to get critical details, that's not science anymore." Remember, the studies might form the…

> If you need to a 'chill' with the original researcher to get critical details, that's not science anymore. Certainly there is always the possibility that there was some details misunderstood, something that needs to be clarified, a print error, etc. Your "that's not science anymore" statement seems highly exaggerated. People are not supposed to communicate only via papers.

Sure, it's inevitable that, in some cases, eventually someone will have to go back to check notes because some factor mattered when no one thought to write it down.

But there's a difference between that, vs "we expect that someone will have to make personal contact with the original researchers in order to replicate it".

If you're explaining away replication failures by such non-contact (as the quote is), that's confirmation of a problem (in keeping with the standards of science), not a vindication of the results.

There's an additional danger of making it so that "you can't replicate until you have social contact with the original researchers". That way lies favoritism: it's harder to criticize someone as you get closer to them socially, and they can withhold the capability of criticism by not engaging the critics.

Re: A bot crawled thousands of studies looking for simple math errors

#66

Post by dahart »

wyager wrote: post #61
cafebeen wrote: post #42

Earlier quoted context omitted.

It's not coddling as much as professional courtesy. Mistakes are often brought up at conferences and in peer review, but most scientists within a specific research area try to be on good terms with one another and don't see value in publicly shaming colleagues for their mistakes.

There are two options here: Use a bot to find tons of mistakes automatically and risk coming across as rude. Or Let social trivialities dominate scientific discourse and let most of those mistakes go unchecked forever because there's no feasible way to "politely" address hundreds of authors who made mistakes and keep checking back to make sure they actually fixed the mistake. The former is clearly the preferable choi…

There is a third option that's better than either one you proposed.

Get people writing new papers to use statcheck before they publish.

The only rudeness here was an avoidable choice - publishing statcheck's results on a huge set of already published papers. The statcheck authors chose to do that for exposure, they even said so, it was not for posterity or the scientific well being of the community.

I don't personally think what they did was wrong, I don't particularly care that some people felt it was rude. But the fact of the matter is that the rude part was completely avoidable.

Re: A bot crawled thousands of studies looking for simple math errors

#68

Post by jfoutz »

Practicality wrote: post #11
merpnderp wrote: post #7

Earlier quoted context omitted.

Objectivity is listed as a problem with science curriculum. Welcome to the future. http://nsuworks.nova.edu/cgi/viewcontent.cgi?article=2467&co...

While I am not a big fan of contemporary feminism, I am pretty sure taking issue with masculinity or "gendered" text is not the same thing as taking issue with objectivity. Edit: Found this quote in your source "Poststructuralism “rejects objectivity and the notions of an absolute truth and single reality,”" Hmm, this seems an awful lot like a religion.

I'm not in any sense an expert. A lot of it seems to boil down to what assumptions are implicit.

When you consider something like math, you can pin down what axioms have what consequences. This set of rules produces a group, adding some more axioms produces a field, things like that.

When you get to more ordinary stuff like living life, it's not clear what axioms you picked, vs what axioms i picked. Most people don't think about it at all.

Imagine a coworker that just started showing up topless. That would be freakishly weird. But really, in a professional setting, why would it matter? There are some handwavy arguments about the nature of professionalism, but all of that relies on what axioms you pick for your culture.

Anyway. We all have these ideas about how we're supposed to interact, but we're all playing by different rules. So you get into these Wittgenstein kinds of discussions.

Re: A bot crawled thousands of studies looking for simple math errors

#69

Post by danharaj »

Practicality wrote: post #54
Jun8 wrote: post #36

Earlier quoted context omitted.

Relevant part of the report: "As these examples show, the STEM syllabi explored in this study demonstrated a view of knowledge that was to be acquired by the student, which promotes a view of knowledge as unchanging. This is further reinforced by the use of adverbs to imply certainty such as “actually” and “in fact” which are used in syllabi to identify information as factual and beyond dispute (Biber, 2006a; 2006b).…

The equating of objective facts with "male-dominant views" is just bizarre. I mean, seriously, they are saying that in the feminist view is that there is no objective reality? I suppose they are saying that feminists... don't live in... reality. You can't make this stuff up. At least they're up front about it, I suppose.

A fact is a human construct. It is a symbol. You just conflated facts with reality. A cursory examination of reality will reveal that observations of reality are observer dependent. So what is an objective fact?

An objective fact must be one that is covariant in change of observer interpreting the fact. Physical laws like General Relativity are covariant in this way: the facts of relativity differ symbolically depending on the reference frame of the observer but different observers' facts are related in a coherent way.

In GR an observer is a reference frame. In order to define objective fact you need to presuppose what an observer is.

It is a masculine behavior to consider this question unimportant. A feminine science is much more interested in the study of the subjective because you can't be objective without understanding subjectivity.

Ultimately an observer in science is a human. That means objective fact can only be understood by understanding the subjectivity of the scientist. An objective fact is in fact an intersubjective fact, and intersubjectivity is an essential contribution to science by feminism. It's a good keyword to google.

Re: A bot crawled thousands of studies looking for simple math errors

#70

Post by mturmon »

douche wrote: post #23
alanfalcon wrote: post #3

I find it very disconcerting that people are trying to fend off criticism of previously published studies by calling it "bullying" or sometimes worse. What do feelings have to do with science?

I guess I can accuse my compiler of bullying me.

Bully it back with:

  $ cc test.c || sudo rm -f $(which cc)
"Listen, cc, we can do this the easy way, or the hard way..."
Post reply on HN