Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

False claims in a widely-cited paper

statmodeling.stat.columbia.edu

Post reply on HN
Print view View original

81–90 of 181 posts

Re: False claims in a widely-cited paper

#81

Post by paulpauper »

hansvm wrote: post #19
sillysaurusx wrote: post #3

Earlier quoted context omitted.

> and actually good and original stuff gets rejected This seems to be the key part. Are you sure that's true? In other news, (a) apparently you can now submit URLs with anchors to HN, previously a perennial problem; (b) this submission anchors to a comment that just says "I will try this. Suggestions welcome" with no further context. Ironically, (b) was exactly why (a) was disallowed for the longest time. Anchors are…

> good stuff rejected, are you sure that's true In the academic circles I frequent, it's not true. Any one journal might reject the good stuff, but it doesn't take more than a few applications to find a journal who recognizes it, and the cost of producing the research is so high that with the current career incentives it'd be ridiculous not to continue submitting. That does mean that journal "quality" matters less th…

I mean at a top or middle ranked journal. There are tons of predatory journals that will publish anything

Re: False claims in a widely-cited paper

#83

Post by thayne »

> their policies allow only authors to request corrections

Say what now?

So the only way to get a correction for a paper is if the author is willing to publicly admit they messed up? Something that an unethical researcher is very unlikely to do.

Re: False claims in a widely-cited paper

#84

Post by thayne »

rudderdev wrote: post #79

Peer reviews need to be more transparent and accountable. Otherwise, we are sure to lose to the misinformation war that is rapidly reaching its peak, thanks but no thanks to AI.

How would you do that? One reason peer reviews are usually anonymous is to prevent retribution (ex. rejecting someone's paper, because they rejected yours). Not that it is perfect at doing that, but if you make it more transparent, you may trade one problem for another.

Re: False claims in a widely-cited paper

#85

Post by bonesss »

rectang wrote: post #75
zx8080 wrote: post #71

Somewhat unrelated but relevant thought: from software engineering experience in large orgs, correction of any issue rarely worth any effort. AI will drive commiting more and more papers with less and less review. The review takes effort, too much in the age of easy generation. With this, science will probably lose trust even more in the coming years.

Brandolini's Bullshit Asymmetry Principle is the adage for our age: > The amount of energy needed to refute bullshit is an order of magnitude bigger than that needed to produce it. https://en.wikipedia.org/wiki/Brandolini%27s_law

I’m cautiously optimistic that LLMs have a role in addressing that asymmetry to the side of good faith actors.

Gish galloping bad faith trolls aren’t new. LLMs shape their BS into fluffy BS that isn’t particularly more effective. But now, We Have The Technology, refuting a pile of poo semi-accurately should be cheap (or at least getting cheaper).

I don’t need an LLM on my phone that can do tax law in Georgia the country. But an “AI Assistant” that could highlight logical fallacies, shifting goalposts, non-responsive dialog, rhetorical obfuscations, etc, would be useful online, at the bar, and work (ie when HR tries to “HR” you, but also is lying and obfuscating about it).

We already have models and people that bullshit. Maybe refutation models are the cure… Chinese needle snakes to catch the lizards, Gorillas to catch the snakes…

Re: False claims in a widely-cited paper

#86

Post by bradley13 »

Some years ago, my institution (primarily a teaching college) decided it needed an additional accreditation. The organization they went with requires faculty to publish. Including our undergrad business faculty.

We all know that "publish or perish" is stupid. The premier example of Goodhart's Law: “When a measure becomes a target, it ceases to be a good measure.” Why can't our highly paid administration understand this?

Re: False claims in a widely-cited paper

#88

Post by scheme271 »

hansvm wrote: post #19
sillysaurusx wrote: post #3

Earlier quoted context omitted.

> and actually good and original stuff gets rejected This seems to be the key part. Are you sure that's true? In other news, (a) apparently you can now submit URLs with anchors to HN, previously a perennial problem; (b) this submission anchors to a comment that just says "I will try this. Suggestions welcome" with no further context. Ironically, (b) was exactly why (a) was disallowed for the longest time. Anchors are…

> good stuff rejected, are you sure that's true In the academic circles I frequent, it's not true. Any one journal might reject the good stuff, but it doesn't take more than a few applications to find a journal who recognizes it, and the cost of producing the research is so high that with the current career incentives it'd be ridiculous not to continue submitting. That does mean that journal "quality" matters less th…

Personally, I'm shocked it went from submission to publication 5 weeks! I didn't think that was possible.

Re: False claims in a widely-cited paper

#89

Post by froh »

oh how I'd love to see a gitlab GitHub like infrastructure and culture for scientific publication. let them have the repo private/authors and reviewers only until publication.

but all flaws are issues, later reported issues are right next to the paper, heck there could even badges for publication and review status...

a woman may dream...

Re: False claims in a widely-cited paper

#90

Post by thayne »

chromacity wrote: post #26
lotsofpulp wrote: post #6

Earlier quoted context omitted.

I have always assumed the further away from math and physics a field is, the higher the probability of any given “research” to be false. Even biology, I might give 50% odds at best, but that is due to the difficulty of observing and measuring in that field. Anything past biology might as well be as fiction due to how easy it is to come up with whatever results you want.

I think that theoretical math and physics are special, but probably not in the way you assume. It's just that there isn't a whole lot of grant money, prestige, or influence associated with them (unless you accomplish something truly exceptional). Computer science is very close to math and should be even easier to verify, but there's plenty of dubious results published every year, simply because it's more profitable t…

There are dubious results published in every subject, including math and physics (whether theoretical or experimental). The difference is that such results are less likely to be widely cited and accepted by the field. For math and theoretical physics, the reader can (assuming sufficient knowledge and skill) verify the result themselves, so if your proof is incorrect or not rigorous enough, you won't get cited. For experimental physics, it is more common for different teams to reproduce a result, or verify a result using a different method, so papers aren't usually widely cited unless they have been independently verified. Part of that is cultural, part of that is attempting to reproduce results is relatively straightforward compared to say experiments involving human subjects, and part if is because results are usually quantitative, so "we did the same thing as paper X, but with more precision" is still interesting enough to be published.
Post reply on HN