A bot crawled thousands of studies looking for simple math errors • Hacker News Board

Re: A bot crawled thousands of studies looking for simple math errors

#191

Post by TheCoelacanth » Thu, Oct 06, 2016, 2:00 PM UTC

eli_gottlieb wrote: post #156

nicops wrote: post #56

Earlier quoted context omitted.

> If you need to a 'chill' with the original researcher to get critical details, that's not science anymore. Certainly there is always the possibility that there was some details misunderstood, something that needs to be clarified, a print error, etc. Your "that's not science anymore" statement seems highly exaggerated. People are not supposed to communicate only via papers.

>People are not supposed to communicate only via papers. A paper and its supplementary materials are supposed to be enough to reproduce the experiment. In practice, this often fails, but that is a fault in the scientific process. Science isn't just about empirical knowledge, it's about public and redundant empirical knowledge, as opposed to losing important knowledge of the natural world when the original investigato…

Wouldn't those problems in the scientific process get corrected more easily if you contacted the original author to see if there are any details that were missed and then publish those details with your results instead of just publishing a paper that says "Nope, couldn't reproduce"?

Re: A bot crawled thousands of studies looking for simple math errors

#192

Post by [deleted] » Thu, Oct 06, 2016, 2:33 PM UTC

lifthrasiir wrote: post #166

monochromatic wrote: post #151

Earlier quoted context omitted.

dmd is a respected HNer, so it's inappropriate to question him.

It is inappropriate to use that fact to dismiss a valid request for the evidence. Being famous in some (possibly popular) website does not mean that one's experience has not been biased.

[deleted]

Re: A bot crawled thousands of studies looking for simple math errors

#193

Post by apathy » Thu, Oct 06, 2016, 2:39 PM UTC

OJFord wrote: post #189

apathy wrote: post #171

Earlier quoted context omitted.

In general, if you have to put science in the name, it isn't.

Computer Science?

That's long for math

Re: A bot crawled thousands of studies looking for simple math errors

#194

Post by Lambdanaut » Thu, Oct 06, 2016, 2:48 PM UTC

tgb wrote: post #97

dmd wrote: post #49

Earlier quoted context omitted.

Oh, I made a huge stink about it (and this was just one example of many comments of that type). This was at Penn in 2007.

Not to question you, but I can't find any evidence that Fiske was ever teaching at Penn. Her CV [1] is quite exhaustive, but only lists giving colloquia at Penn. [1] http://static1.1.sqspcdn.com/static/f/1605966/27061956/14652...

> Not to question you

It's quite okay. That's exactly the kind of attitude that's being scrutinized here ;)

Re: A bot crawled thousands of studies looking for simple math errors

#195

Post by OJFord » Thu, Oct 06, 2016, 3:03 PM UTC

apathy wrote: post #193

OJFord wrote: post #189

Earlier quoted context omitted.

Computer Science?

That's long for math

Which definitely is science; QED.

Re: A bot crawled thousands of studies looking for simple math errors

#196

Post by cushychicken » Thu, Oct 06, 2016, 3:33 PM UTC

M_Grey wrote: post #177

cushychicken wrote: post #131

Earlier quoted context omitted.

I think "bullying" is a reductive term for what's actually happening. I think the point trying to be conveyed is that there is a certain decline in the author's personal status when they get holes punched in their publications.

As it should be, when you publish a stupid error that can substantially alter your conclusions.

I'm not sure that's the healthiest attitude. The pursuit here is knowledge, not prestige. Getting personal doesn't do anything for the higher goal.

I read once that the attitude in NASA over Space Shuttle code was very focused not on who wrote the bug, but how to fix the bug. The thought was that who caused the failure was not nearly as important as preventing the failure, as "failure" in that context generally meant loss of human life.

Re: A bot crawled thousands of studies looking for simple math errors

#197

Post by chriswarbo » Thu, Oct 06, 2016, 3:38 PM UTC

yakult wrote: post #162

mentat wrote: post #101

Earlier quoted context omitted.

> People are not supposed to communicate only via papers. That we use written communication that persists through generations is the basis of science and society in general. If we cannot communicate sufficiently via papers, we're in a world of trouble.

I have a concrete counterexample. Let's say I write a paper presenting a model, plus some numerical results of large simulations. The code is based on gluing together various pieces of open source code. All these codes are typical scientist codes that are held together with duct tape. My paper is short, but I spent a lot of effort munging things together, and I'm fairly certain nobody can reproduce my results without…

A lot of thought has gone into such questions. For example, see the guidelines at https://www.epsrc.ac.uk/about/standards/researchdata/

Re: A bot crawled thousands of studies looking for simple math errors

#198

Post by [deleted] » Thu, Oct 06, 2016, 3:40 PM UTC

int_19h wrote: post #173

jlarocco wrote: post #152

Earlier quoted context omitted.

I don't necessarily think this is bullying or humiliating, but it's silly to think that being done by a "bot" and being "factual" has anything to do with it. If a malware "bot" secretly posted people's porn viewing history to their facebook page, would that not be humiliating for those people? Or would it not because it was factual and done by a "bot"? Clearly it would be. Saying it was done by a bot is no excuse for…

Being done by a bot is not an excuse in general. However, when pointing out objective factual mistakes, I think there is a difference between your colleague pointing it out, and an automated tool pointing it out. Even if result is the same, the former can be embarrassing, hence why we learn to phrase negative responses in a roundabout way. But politeness cannot be expected of a brainless machine, and so it can delive…

[deleted]

Re: A bot crawled thousands of studies looking for simple math errors

#199

Post by lifthrasiir » Thu, Oct 06, 2016, 3:43 PM UTC

cfqycwz wrote: post #170

lifthrasiir wrote: post #166

Earlier quoted context omitted.

It is inappropriate to use that fact to dismiss a valid request for the evidence. Being famous in some (possibly popular) website does not mean that one's experience has not been biased.

Go five levels up from your comment and you might find a joke you're missing—unless you're in on it too and it's sailing right over my head

Oh-kay, now I understood what happened (the fifth level is invisible from the parent comment, ugh). Thank you for pointing out where I screwed up :S

Re: A bot crawled thousands of studies looking for simple math errors

#200

Post by dclowd9901 » Thu, Oct 06, 2016, 4:02 PM UTC

SerLava wrote: post #78

6stringmerc wrote: post #19

What a clever and, dare I say it, fantastically useful experiment! So much less harm than even "door knob twisting" type explorations - no, this was using published works and pretty much running them through a process to verify or not verify accuracy. Unsolicited? So what! As a practiced writer I make unsolicited judgments on language usage all the time. Are these people that completely write from their own minds and…

Certainly not bullying. It's actually shocking to me that these studies could have been published without running a similar check on all of the arithmetic. The technology to automate this has been possible for decades.

Right. To my mind, this level of scrutiny is little different than a linter or static analyzer for a programming language.