Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why the world of scientific research needs to be disrupted

gigaom.com

Post reply on HN
Print view View original

31–40 of 86 posts

Re: Why the world of scientific research needs to be disrupted

#31

Post by Mvandenbergh »

angersock wrote: post #26
mironathetin wrote: post #12

Two comments: 1. Scientists publish in slow peer reviewed journals, because scientific publication has to be peer reviewed. The reviewers are a very small selection of other scientists, who are expert in the field of the publication. As long as the reviewers don't signal white smoke, a publication is no science by definition. This way of work is important to keep quality excellent. We certainly don't need more quickl…

I'm sorry, but I must disagree with the spirit of your points. Scientists publish in slow peer reviewed journals, because scientific publication has to be peer reviewed. The reviewers are a very small selection of other scientists, who are expert in the field of the publication. So, when you throw work out into the open, anyone with an interest in your topic (your functional peers) will be able to do their own analys…

If you can't be bothered to read new things unless they're "worth your time", the hell with you.

Realistically, even in a narrow field just reading (forget about working through the mathematics, checking the results, etc.) all the papers that are published through the slow existing process would take up more than 100% of working time. Clearly there has to be some kind of filtering process.

As it happens, I think that this process should be both faster and more open, but it is foolish to dismiss realistic concerns about how to decide which research it is most worthwhile to read.

There's a big difference between that and being afraid to be exposed to something outside their comfort zone (I am not too sure what you mean by that). You may hope to be so lucky as to have competitors with people like that on your team, but would you be happy to have on your team someone who spent literally their entire workday every day reading reviews of other people's work without contributing anything new of their own?

Re: Why the world of scientific research needs to be disrupted

#32

Post by bmuon »

mironathetin wrote: post #12

Two comments: 1. Scientists publish in slow peer reviewed journals, because scientific publication has to be peer reviewed. The reviewers are a very small selection of other scientists, who are expert in the field of the publication. As long as the reviewers don't signal white smoke, a publication is no science by definition. This way of work is important to keep quality excellent. We certainly don't need more quickl…

I agree that time isn't a matter of concern for most scientists.

The article seems to be a badly researched rehash of other Open Science articles. Open Science is not about faster delivery, it's about access to information. The problem is not that publications are slow, it's that without paying for the magazine other scientists, the media and non scientists can't access those papers and learn, divulge, criticize or contribute.

Re: Why the world of scientific research needs to be disrupted

#33

Post by _delirium »

mironathetin wrote: post #12

Two comments: 1. Scientists publish in slow peer reviewed journals, because scientific publication has to be peer reviewed. The reviewers are a very small selection of other scientists, who are expert in the field of the publication. As long as the reviewers don't signal white smoke, a publication is no science by definition. This way of work is important to keep quality excellent. We certainly don't need more quickl…

On #2, it varies by field, but I agree somewhat, in that I generally think there's already a significant problem of people being too quick-on-the-draw without first familiarizing themselves with the literature. There is a lot of stuff published that rehashes (without citation) existing published results or minor variations on them, sometimes from decades earlier, that the authors and reviewers just didn't happen to be familiar with. If you accelerate everything and distribute the peer-review to people with less deep knowledge of the literature, I suspect this will only increase, adding more noise to the literature as we spend the 2010s rediscovering the 1970s, accompanied by triumphant press releases and blog posts about Breakthrough Discoveries.

I notice this particularly in computer science, where it seems nobody is capable of locating or reading journal articles published before around 1995, a handful of canonical "classic" papers excepted.

Re: Why the world of scientific research needs to be disrupted

#34

Post by pnathan »

cube13 wrote: post #25
dantheman wrote: post #23

Earlier quoted context omitted.

Speed is important for a number of factors, mistakes cost less if they can be quickly corrected. So being risky and publishing about your failures becomes useful (no one publishes failures...). Speed also reduces duplicate work. Peer review is slow because publishing and retracting used to take a long time, now it is instant. And as for the quality of the reviewer, perhaps it would make sense for the journal to emplo…

>Speed is important for a number of factors, mistakes cost less if they can be quickly corrected. So being risky and publishing about your failures becomes useful (no one publishes failures...). Speed also reduces duplicate work. You're missing the point of scientific publication. The entire point of publication is so others can reproduce your work. Speed doesn't work if you need a billion data points taken over 20 y…

> 1. Are capable of understanding everything in the paper

Agreed. In my academic field (Debuggers), to understand and know enough to give a real review of the subject requires reading what I would estimate to be somewhere around a thousand papers. It also requires keeping up with the major industrial producers of the product. I think the field has something like two to three thousand papers right now, I haven't checked in a while.

For my work, I am just hitting the seminal papers and trying to avoid spending time reading work that led nowhere. I've read maybe two-three hundred articles (haven't tracked), and my thesis cites over one hundred.

So no, ordinary people won't do that. Ordinary computer programmers won't do that. Most people are not capable (or prepared) to understand everything in a given paper and comment on what has gone before and what has been tried before.

Academic knowledge maintenance is the total philosophical opposite of tl;dr and Twitter.

Re: Why the world of scientific research needs to be disrupted

#35

Post by pnathan »

_delirium wrote: post #33
mironathetin wrote: post #12

Two comments: 1. Scientists publish in slow peer reviewed journals, because scientific publication has to be peer reviewed. The reviewers are a very small selection of other scientists, who are expert in the field of the publication. As long as the reviewers don't signal white smoke, a publication is no science by definition. This way of work is important to keep quality excellent. We certainly don't need more quickl…

On #2, it varies by field, but I agree somewhat, in that I generally think there's already a significant problem of people being too quick-on-the-draw without first familiarizing themselves with the literature. There is a lot of stuff published that rehashes (without citation) existing published results or minor variations on them, sometimes from decades earlier, that the authors and reviewers just didn't happen to b…

Agreed. Vast amounts of very solid literature have 'fallen' off the public face of the planet in CS, except in the archives of publishing houses and libraries.

(Of course, I'm confident part of the problem is that expensive paywalls discourage exhaustive trawling by non-academics).

Re: Why the world of scientific research needs to be disrupted

#36

Post by angersock »

pnathan wrote: post #34
cube13 wrote: post #25

Earlier quoted context omitted.

>Speed is important for a number of factors, mistakes cost less if they can be quickly corrected. So being risky and publishing about your failures becomes useful (no one publishes failures...). Speed also reduces duplicate work. You're missing the point of scientific publication. The entire point of publication is so others can reproduce your work. Speed doesn't work if you need a billion data points taken over 20 y…

> 1. Are capable of understanding everything in the paper Agreed. In my academic field (Debuggers), to understand and know enough to give a real review of the subject requires reading what I would estimate to be somewhere around a thousand papers. It also requires keeping up with the major industrial producers of the product. I think the field has something like two to three thousand papers right now, I haven't check…

Out of curiosity, is there some place to locate a clear listing of the seminal works in your field?

I'm genuinely curious how much of the difficulty here is actual breadth/depth of the subject matter, and how much of the difficulty is due to some systemic inefficiency in the way research is published and consumed.

Re: Why the world of scientific research needs to be disrupted

#37

Post by mechanical_fish »

dantheman wrote: post #23
mironathetin wrote: post #12

Two comments: 1. Scientists publish in slow peer reviewed journals, because scientific publication has to be peer reviewed. The reviewers are a very small selection of other scientists, who are expert in the field of the publication. As long as the reviewers don't signal white smoke, a publication is no science by definition. This way of work is important to keep quality excellent. We certainly don't need more quickl…

Speed is important for a number of factors, mistakes cost less if they can be quickly corrected. So being risky and publishing about your failures becomes useful (no one publishes failures...). Speed also reduces duplicate work. Peer review is slow because publishing and retracting used to take a long time, now it is instant. And as for the quality of the reviewer, perhaps it would make sense for the journal to emplo…

Peer review is slow because publishing and retracting used to take a long time, now it is instant.

Have you ever done a peer review? It took me hours to do each one. To do it well, you have to stare at each procedure and each conclusion and try to imagine what could be going wrong.

You may also have to read a lot of literature, both in general (so that you can recognize the difference between something genuinely new and something that was already published in 1975) and in particular (so that you can make sure the manuscript is not misrepresenting its own references).

And, given what's at stake in a review – you're basically holding a year or more of some grad student's life in your hands, at a minimum – it's only respectful to take it seriously.

perhaps it would make sense for the journal to employ professional scientists that only review...

This is either obvious, or silly. Obvious, to the extent that journals always have employed scientifically trained editors – they're called "editors" – to do everything from tweaking bad phrasing to offering criticism to ultimately making the final decision about what gets published. Silly, because journal editors are rarely experts in your specific field. How can they be? There are a lot of scientific fields. It's impossible to be "up to date on all the current research" in every single one of them, to the requisite level of detail. And most fields aren't awash in money to the extent that they can support a full-time editor with complete expertise. In the general case, you have to rely on peer review because only your peers have the incentive to be experts in your field.

Re: Why the world of scientific research needs to be disrupted

#38

Post by rflrob »

JoeAltmaier wrote: post #19
mironathetin wrote: post #12

Two comments: 1. Scientists publish in slow peer reviewed journals, because scientific publication has to be peer reviewed. The reviewers are a very small selection of other scientists, who are expert in the field of the publication. As long as the reviewers don't signal white smoke, a publication is no science by definition. This way of work is important to keep quality excellent. We certainly don't need more quickl…

The article made the point that this is an "accident of paper". Peer review comes from the expense of curating paper documents. Of course it is arguable that peer review weeds out noise too. But you have to get your work in front of a peer, and they have to understand your contribution. Its also arguable that peer review leads to stagnation and missed opportunities/discoveries. In fact, surely it does.

>But you have to get your work in front of a peer, and they have to understand your contribution.

A very small (but growing) minority of journals, like PLoS One or , make no effort to understand or estimate the significance of the contribution, only whether or not the work is scientifically sound.

Re: Why the world of scientific research needs to be disrupted

#39

Post by angersock »

hyperbovine wrote: post #29
angersock wrote: post #26

Earlier quoted context omitted.

I'm sorry, but I must disagree with the spirit of your points. Scientists publish in slow peer reviewed journals, because scientific publication has to be peer reviewed. The reviewers are a very small selection of other scientists, who are expert in the field of the publication. So, when you throw work out into the open, anyone with an interest in your topic (your functional peers) will be able to do their own analys…

Here's a paper I read yesterday. Kindly review it for me: http://arxiv.org/pdf/1107.2875.pdf The scheme you are advocating might have succeeded 200 years ago, when science was happening at a much for fundamental level than it is now. These days, things are so specialized that what the OP says is spot-on: only a handful of people are qualified to knowledgeably comment on your work. What's more, these people are all /i…

Thank you sir/madam for an opportunity to better explore the depths of my ignorance. I have previously wondered what sort of use abstract algebra and geometry would be put to outside of various parts of applied mathematics or physics, and I would now appear to have an excellent example. :)

As I've mentioned in another reply, I'd like to see the system cleaned up and made more accessible. I'm a mechanical engineer by schooling and a graphics programmer by trade, so I don't frighten easily at math or algorithms. That said, this paper gave me quite a start. How deep down the rabbit-hole would you say it is in the field of computer vision? Is it fairly advanced/niche in application, or is it about some sort of foundational knowledge that is supposed to be known backwards and forwards by practitioners of the art?

I feel as though a better way of presenting the papers might aid in understanding (clear list of dependencies for knowledge, clear list of applications, clear note of "hey, this is a minor optimization, so don't sweat the details if you don't get it", etc.), but my personal metrics for judging this may not be correct. Any feedback would be appreciated.

Re: Why the world of scientific research needs to be disrupted

#40

Post by ellyagg »

mironathetin wrote: post #12

Two comments: 1. Scientists publish in slow peer reviewed journals, because scientific publication has to be peer reviewed. The reviewers are a very small selection of other scientists, who are expert in the field of the publication. As long as the reviewers don't signal white smoke, a publication is no science by definition. This way of work is important to keep quality excellent. We certainly don't need more quickl…

I'm sorry, your claim seems to be: The process is working perfectly right now. This is absurd. The scientific method is a lot more open to interpretation than you suggest, and indeed it is interpreted with wide variety in different disciplines.

The methods of science will be disrupted and improved. Bright lay people will absolutely be able to poke holes in research once the methods and data are completely open, as has been indisputably proven with the advent and prominence of open source software. Research has shown systemic flaws in numerous disciplines. These sorts of flaws fester because of the closed nature of the system. Open research will have its day. It's only a matter of time.

Post reply on HN