Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redesigning the Scientific Paper

theatlantic.com

Post reply on HN
Print view View original

71–80 of 110 posts

Re: Redesigning the Scientific Paper

#71

Post by robotresearcher »

contravariant wrote: post #20
jostmey wrote: post #3

We need GitHub for science. But that's not enough. It needs to be combined with a mechanism for peer-review and publishing that funding agencies will find acceptable--that's the key.

I assume you don't just means using Github for version control of scientific papers, because that sounds pretty pointless to me. As far as tracking incremental improvements over time goes, I think it'll be hard to do better than our current method of including references to papers. It's impossible to track ideas the same way tracking code works (which itself is limited for similar reasons). It would be nice if you co…

ArXiv has versioned papers and a way of citing them. It’s working pretty well.

Re: Redesigning the Scientific Paper

#72

Post by hnhn12 »

mettamage wrote: post #63

Regarding the article itself: Brett Victor is amazing and so is Strogatz. They are both my heroes actually. But I do think there is a difference between scientific professionals communicating amongst each other and scientific communication to the public. And if mathematicians understood Strogatz his paper at the time when it was published, and there were enough mathematicians to disseminate the knowledge, then should…

>Part of the reason why mathematicians and computer scientists (as researchers) conceive of new algorithms in the first place is because a lot of them are very strong in visualizing algorithms and 'being their own computer'.

Can you explain this in more detail?

Re: Redesigning the Scientific Paper

#73

Post by p1esk »

nonbel wrote: post #70
p1esk wrote: post #68

Earlier quoted context omitted.

You seem to be exceptionally well funded, and/or have few deadline constraints I wish! :) you get spammed with "good ideas" Again, I wish! In the subfield I'm focused on at the moment (efficient mapping of NN algorithms to specialized hardware, low precision computation, model compression) I don't see good ideas very often (fewer than one good paper a week). Previously I worked on music generation - also didn't reall…

I don't mean this to be adversarial, but what exactly is it you do that would not be sped up by checking someone else's results directly before fiddling around and then trying out your own implementation?

But that's my point: their results are not that important to me.

As an example, recently I saw a paper on NN weight quantization, which had a very interesting idea, but the results were not impressive. I don't remember if they had any code published or not, but it didn't matter - I wanted to see what kind of results I'd get if I implemented it. Turned out it works really well, much better than what they reported in the paper.

Re: Redesigning the Scientific Paper

#74

Post by michaelbarton »

p1esk wrote: post #66
jfaucett wrote: post #9

> These programs tend to be both so sloppily written and so central to the results that it’s contributed to a replication crisis, or put another way, a failure of the paper to perform its most basic task: to report what you’ve actually discovered, clearly enough that someone else can discover it for themselves. This is the crux of the of the problem IMHO - at least for the fields I study (AI/ML). Replicating the resu…

I also work in AI/ML field (deep learning), and usually I don't care if the paper has corresponding code or not. I read papers to find good ideas. If I find it, I can implement it myself. I rarely need more than a couple of days to test an idea (e.g. Hinton's capsules model took 4-5 hours to implement). The benefits of own implementation should be obvious. If something important is missing or does not make sense, I u…

What your preferred software to implement these? A framework like chainer, or purely in numpy/MATLAB?

Re: Redesigning the Scientific Paper

#75

Post by currymj »

p1esk wrote: post #66
jfaucett wrote: post #9

> These programs tend to be both so sloppily written and so central to the results that it’s contributed to a replication crisis, or put another way, a failure of the paper to perform its most basic task: to report what you’ve actually discovered, clearly enough that someone else can discover it for themselves. This is the crux of the of the problem IMHO - at least for the fields I study (AI/ML). Replicating the resu…

I also work in AI/ML field (deep learning), and usually I don't care if the paper has corresponding code or not. I read papers to find good ideas. If I find it, I can implement it myself. I rarely need more than a couple of days to test an idea (e.g. Hinton's capsules model took 4-5 hours to implement). The benefits of own implementation should be obvious. If something important is missing or does not make sense, I u…

this is the right way to go about things if you have certain goals, for sure.

sometimes you need to replicate exactly the same training method, on exactly the same data — for instance if you want to use it as a baseline on a known dataset. then it becomes really important to have the code, because while an adequate replication might be easy, it takes a lot of trial and error to get perfectly the same model.

Re: Redesigning the Scientific Paper

#76

Post by nonbel »

p1esk wrote: post #73
nonbel wrote: post #70

Earlier quoted context omitted.

I don't mean this to be adversarial, but what exactly is it you do that would not be sped up by checking someone else's results directly before fiddling around and then trying out your own implementation?

But that's my point: their results are not that important to me. As an example, recently I saw a paper on NN weight quantization, which had a very interesting idea, but the results were not impressive. I don't remember if they had any code published or not, but it didn't matter - I wanted to see what kind of results I'd get if I implemented it . Turned out it works really well, much better than what they reported in…

Here is an idea: inverse dropout.

How would you implement that?

Re: Redesigning the Scientific Paper

#77

Post by p1esk »

nonbel wrote: post #76
p1esk wrote: post #73

Earlier quoted context omitted.

But that's my point: their results are not that important to me. As an example, recently I saw a paper on NN weight quantization, which had a very interesting idea, but the results were not impressive. I don't remember if they had any code published or not, but it didn't matter - I wanted to see what kind of results I'd get if I implemented it . Turned out it works really well, much better than what they reported in…

Here is an idea: inverse dropout. How would you implement that?

Link to paper?
Post reply on HN