Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why the world of scientific research needs to be disrupted

gigaom.com

Post reply on HN
Print view View original

41–50 of 86 posts

Re: Why the world of scientific research needs to be disrupted

#41

Post by jpdoctor »

e_g wrote: post #21
jpdoctor wrote: post #5

Earlier quoted context omitted.

> In my personal experience the main problem with scientific research today is caused by publication counts representing the sole metric for academic success. It also leads to bloat in the literature: People learn the "least publishable increment" and go publish that.

Exactly. A study by Tuliving and Madigan ( http://alicekim.ca/AnnRev70.pdf ) outlines this nicely. In the words of David T. Lykken ( http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.118... ): In their 1970 Annual review chapter on Memory and Verbal learning, Tulving and madigan reported that they had independently rated 540 published articles in terms of their "contribution to knowledge". With "remarkable agr…

Thanks for those links. During my own PhD, I was pretty sure that the "least publishable increment" became negative.

The underlying problem is that there is no penalty for publishing non-increments. For some inexplicable reason, citation impact assessments do not divide by the total number of publications.

I have no doubt that such a normalization factor would go a long way to fixing the problem.

Re: Why the world of scientific research needs to be disrupted

#42

Post by quanticle »

_delirium wrote: post #33
mironathetin wrote: post #12

Two comments: 1. Scientists publish in slow peer reviewed journals, because scientific publication has to be peer reviewed. The reviewers are a very small selection of other scientists, who are expert in the field of the publication. As long as the reviewers don't signal white smoke, a publication is no science by definition. This way of work is important to keep quality excellent. We certainly don't need more quickl…

On #2, it varies by field, but I agree somewhat, in that I generally think there's already a significant problem of people being too quick-on-the-draw without first familiarizing themselves with the literature. There is a lot of stuff published that rehashes (without citation) existing published results or minor variations on them, sometimes from decades earlier, that the authors and reviewers just didn't happen to b…

How much of that is because research from the 1970s is effectively locked away behind ACM and IEEE paywalls? How much research gets rehashed because the researcher can't find the result he or she is looking for via the atrocious search tools provided by the likes of Elsevier?

I think that more openness in scientific publication will in fact mitigate the problem of people being quick-on-the-draw, as contradictory findings and previous publications of similar results will be easier to find, even by the non-scientific audience.

Re: Why the world of scientific research needs to be disrupted

#43

Post by khafra »

cube13 wrote: post #25
dantheman wrote: post #23

Earlier quoted context omitted.

Speed is important for a number of factors, mistakes cost less if they can be quickly corrected. So being risky and publishing about your failures becomes useful (no one publishes failures...). Speed also reduces duplicate work. Peer review is slow because publishing and retracting used to take a long time, now it is instant. And as for the quality of the reviewer, perhaps it would make sense for the journal to emplo…

>Speed is important for a number of factors, mistakes cost less if they can be quickly corrected. So being risky and publishing about your failures becomes useful (no one publishes failures...). Speed also reduces duplicate work. You're missing the point of scientific publication. The entire point of publication is so others can reproduce your work. Speed doesn't work if you need a billion data points taken over 20 y…

> You're missing the point of scientific publication. The entire point of publication is so others can reproduce your work. Speed doesn't work if you need a billion data points taken over 20 years to prove a long term issue.

In theory, that's the point. In practice, no journals ever publish replications, so nobody wastes their time reproducing others' work when they could be working on something publishable or their next grant proposal.

Re: Why the world of scientific research needs to be disrupted

#44

Post by [deleted] »

pnathan wrote: post #35
_delirium wrote: post #33

Earlier quoted context omitted.

On #2, it varies by field, but I agree somewhat, in that I generally think there's already a significant problem of people being too quick-on-the-draw without first familiarizing themselves with the literature. There is a lot of stuff published that rehashes (without citation) existing published results or minor variations on them, sometimes from decades earlier, that the authors and reviewers just didn't happen to b…

Agreed. Vast amounts of very solid literature have 'fallen' off the public face of the planet in CS, except in the archives of publishing houses and libraries. (Of course, I'm confident part of the problem is that expensive paywalls discourage exhaustive trawling by non-academics).

[deleted]

Re: Why the world of scientific research needs to be disrupted

#46

Post by pnathan »

angersock wrote: post #36
pnathan wrote: post #34

Earlier quoted context omitted.

> 1. Are capable of understanding everything in the paper Agreed. In my academic field (Debuggers), to understand and know enough to give a real review of the subject requires reading what I would estimate to be somewhere around a thousand papers. It also requires keeping up with the major industrial producers of the product. I think the field has something like two to three thousand papers right now, I haven't check…

Out of curiosity, is there some place to locate a clear listing of the seminal works in your field? I'm genuinely curious how much of the difficulty here is actual breadth/depth of the subject matter, and how much of the difficulty is due to some systemic inefficiency in the way research is published and consumed.

Not that I've seen, but that does not mean I simply haven't stumbled into the right FAQ.

Re: Why the world of scientific research needs to be disrupted

#47

Post by waqf »

_delirium wrote: post #33
mironathetin wrote: post #12

Two comments: 1. Scientists publish in slow peer reviewed journals, because scientific publication has to be peer reviewed. The reviewers are a very small selection of other scientists, who are expert in the field of the publication. As long as the reviewers don't signal white smoke, a publication is no science by definition. This way of work is important to keep quality excellent. We certainly don't need more quickl…

On #2, it varies by field, but I agree somewhat, in that I generally think there's already a significant problem of people being too quick-on-the-draw without first familiarizing themselves with the literature. There is a lot of stuff published that rehashes (without citation) existing published results or minor variations on them, sometimes from decades earlier, that the authors and reviewers just didn't happen to b…

I have heard stories implying that you can reliably produce an "original" doctoral thesis by digging out something from the vast amount of work published 70–100 years ago in German.

Re: Why the world of scientific research needs to be disrupted

#50

Post by munin »

angersock wrote: post #39
hyperbovine wrote: post #29

Earlier quoted context omitted.

Here's a paper I read yesterday. Kindly review it for me: http://arxiv.org/pdf/1107.2875.pdf The scheme you are advocating might have succeeded 200 years ago, when science was happening at a much for fundamental level than it is now. These days, things are so specialized that what the OP says is spot-on: only a handful of people are qualified to knowledgeably comment on your work. What's more, these people are all /i…

Thank you sir/madam for an opportunity to better explore the depths of my ignorance. I have previously wondered what sort of use abstract algebra and geometry would be put to outside of various parts of applied mathematics or physics, and I would now appear to have an excellent example. :) As I've mentioned in another reply, I'd like to see the system cleaned up and made more accessible. I'm a mechanical engineer by…

"don't sweat the details if you don't get it" is probably the opposite of what you want to communicate to a reviewer ;) if there is a problem in "the details" the entire work might need to be scrapped.
Post reply on HN