Live data from Hacker News

Let me see your papers, let me see your source

cflewis.posterous.com

31–32 of 32 posts

Re: Let me see your papers, let me see your source

#31
post #30
post #15

I see some of the comments here talk about the code being useless it terms of using it the same way the author of the paper has. Personally for a lot of the papers I have read in NLP based stuff I'd feel more comfortable just having some source to look through to give a better idea of how it was done. Some papers are quiet abstract about their method and aren't really helpful past giving you basic ideas without some…

I'm not sure I agree. In most NLP papers I've read the real meat of the paper is not in the code, but in a couple of equations that, with a lot of mostly-mechanical wrapping around, implementing, debugging, and plugging into well-understood parts should make sense and be a contribution in and of themselves. And for these other things, it might be good to see code a couple of times, but mostly at first, and to get up…

I guess it depends on the level your at, it comes back to an article a while ago that to truly understand all the new research coming out you have to be actively contributing research of your own and interacting with others in your field.

Your right about the classifier, not needing a mention now. My problem starting out of my honours thesis is that I would read something like that then have to go research the thing they have just mentioned in passing because their core audience knows all about it already. So I guess it has an aspect of knowing the best starting point for what you want to research as well.

Re: Let me see your papers, let me see your source

#32
post #11

I agree with the feeling, but a lot of the time, scientific software is just not ready for real usage. My software at least is completely useless most of the time (you have to open it in the interpreter and type the right incantations, and the data has to be in a non-trivial massaged way, filesystem paths are hardcoded everywhere, etc). It takes some effort to turn a works-for-me research tool into something releasea…

It's been a few years since this was relevant for me, but my primary concern would be that in order to publish actual code I would either have to spend months cleaning it up, writing documentation etc., or spend the next year fielding support calls. Or both.

I agree with another poster here that having somebody else repeating the experiment with their own implementation is a better test for validity - if a second paper just copies the source code from the first and makes a few tweaks, mistakes could easily carry over.

But having the data available would be great.

Post reply on HN