Live data from Hacker News

Why the world of scientific research needs to be disrupted

gigaom.com

11–20 of 86 posts

Re: Why the world of scientific research needs to be disrupted

#11
post #6

Unless scientists and researchers start to put the interests of collaboration and “open science” ahead of their desire to be promoted or win tenure the system will not change This is the main problem because most won't.

Yea, the real question is how to show their efforts in an open system so they can get tenure without fear of being "scooped" by being open

Bingo. There is good reason to fear being scooped. It can cost you funding, and thus, your job.

Re: Why the world of scientific research needs to be disrupted

#12
Two comments:

1. Scientists publish in slow peer reviewed journals, because scientific publication has to be peer reviewed. The reviewers are a very small selection of other scientists, who are expert in the field of the publication. As long as the reviewers don't signal white smoke, a publication is no science by definition. This way of work is important to keep quality excellent. We certainly don't need more quickly published junk that stirs the world but is found to be inaccurate later.

Can we accelerate the process of peer reviewing? I doubt that. As I have written, the reviewers are other scientists who have to find the time to read and comment new articles. The more time they spend with this, the less time they spend doing research. So the small amount of experts usually is responsible for being slow.

You may be quicker by internet publishing, if (IF!) you work on a highly dynamic field where other scientists are hungry for your results. As the author writes, this is the exception. Most scientific results will be useful only a long time after the publication, if they will be useful at all. In this way, science is much faster than application already.

2. The author writes, we could make progress faster. Please: why? It seems to be a sign of the time that we want to accelerate everything. It is most likely that this will only produce more noise. We need to slow down our lives and our thinking to stay accurate and produce real value. I am glad that science results, at least in my field, are still reliably scrutinized by many. This way I know that reading and spending the time to understand the stuff is worth my time.

Re: Why the world of scientific research needs to be disrupted

#13
Tim Gowers, fields medallist and the person who started the first polymath project just wrote a couple of blog posts (the latter including suggestions from the former) on a new model of math publishing. They are highly detailed and well thought out and also talk about things one might not understand unless one works in academia.

https://gowers.wordpress.com/2011/10/31/how-might-we-get-to-...

https://gowers.wordpress.com/2011/11/03/a-more-modest-propos...

Re: Why the world of scientific research needs to be disrupted

#14
I think the major funding agencies are aware of the problem and the potential, and are working on solutions in their own way (for example, the NSF Office of Cyberinfrastructure's DataNet program, http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=503141). But major cultural change in the way scientific research is conducted just isn't going to happen overnight.

Re: Why the world of scientific research needs to be disrupted

#15

Two comments: 1. Scientists publish in slow peer reviewed journals, because scientific publication has to be peer reviewed. The reviewers are a very small selection of other scientists, who are expert in the field of the publication. As long as the reviewers don't signal white smoke, a publication is no science by definition. This way of work is important to keep quality excellent. We certainly don't need more quickl…

To add to that, 3. Scientists (particularly academics) should focus on a smaller number of solid, flushed-out papers rather than seeking out the minimum publishable unit. The latter seems to be a byproduct of the tenure process (IMO) and just adds noise.

Re: Why the world of scientific research needs to be disrupted

#16
post #10
post #6

Unless scientists and researchers start to put the interests of collaboration and “open science” ahead of their desire to be promoted or win tenure the system will not change This is the main problem because most won't.

I absolutely would if I could do it and still get funded. I'd love to share everything openly. Just pay my bills as I do it. Currently, science is funded based on individual results. You can't blame folk for wanting to be promoted or to win tenure.

> Currently, science is funded based on individual results.

Yeah, there's a direct tension between the move over the past few decades towards very competitive grant funding (10% funding rates, short grant terms, much fewer long-term, large block grants for centers), and a goal of having everyone share everything altruistically. If you purposely set up science funding so that it encourages cutthroat competition, people are going to have to behave like cutthroat competitors, and those who try not to will (on average) lose to those who do.

Re: Why the world of scientific research needs to be disrupted

#17
Another vector for disruption is with statistics. Scientists use statistics to analyze the results of their experiments all the time. Unfortunately, most scientists are not statisticians. They spend a lot of time on statistics that they want to spend doing things they're good at. They make lots of mistakes, for example using the wrong statistical test or deciding what test to use after the experiment, etc.

With today's computer speeds there is no reason to do the statistics the way they're doing it. You don't need to make strong and unjustified assumptions about your data because then you can use a simple T-test. Today with simulation you can do pretty much any statistical test you want. You can do Bayesian inference.

However, most of these scientists are not capable of programming that themselves in R. What is needed is a simple GUI interface where they can state their assumptions and enter their data. Then the program calculates their posteriors and does any hypothesis test they want. The statistics scientists use today are optimized for pen and paper. The assumptions no longer hold. Who cares if the computation takes 500 milliseconds on a computer instead of 3 milliseconds?

Re: Why the world of scientific research needs to be disrupted

#18
There are already some efforts to speed up peer-review and make more scientific publications available for free to anyone, e.g., http://www.plosone.org/

> Unless scientists and researchers start to put the interests of collaboration and “open science” ahead of their desire to be promoted or win tenure the system will not change

Something similar could be said about so many industries, including our own.

Re: Why the world of scientific research needs to be disrupted

#19

Two comments: 1. Scientists publish in slow peer reviewed journals, because scientific publication has to be peer reviewed. The reviewers are a very small selection of other scientists, who are expert in the field of the publication. As long as the reviewers don't signal white smoke, a publication is no science by definition. This way of work is important to keep quality excellent. We certainly don't need more quickl…

The article made the point that this is an "accident of paper". Peer review comes from the expense of curating paper documents.

Of course it is arguable that peer review weeds out noise too. But you have to get your work in front of a peer, and they have to understand your contribution. Its also arguable that peer review leads to stagnation and missed opportunities/discoveries. In fact, surely it does.

Re: Why the world of scientific research needs to be disrupted

#20
post #3

In my personal experience the main problem with scientific research today is caused by publication counts representing the sole metric for academic success. Research is 'optimized' for publications and segmented according to conference schedules. That in itself I consider detrimental to the research effort. However the major damage in terms of advancing resarch in my opinion is caused by the induced tendency of schol…

There's been an attempt to measure paper quality using the "impact factor", but as with seemingly every measure, it's led to shenanigans. Some publishers encourage authors to cite papers from their journals, and some editors write survey papers that happen to cite a bunch of articles from their journals. A recent AMS Notices covered this in math, and I sort of expect it's worse in other disciplines.
Post reply on HN