Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Should I publish my research code?

news.ycombinator.com

Post reply on HN
Print view View original

141–150 of 363 posts

Re: Ask HN: Should I publish my research code?

#141

Post by Derbasti »

Source Code or it Didn't Happen.

Science that is not reproducible is not science.

If you can, publish something high-level. Matlab or Python or Julia is fine. C or Java, not so much, because the build environment will not be available any longer after a few years. Actually, if you can, publish several translations.

And don't forget to publish your data sets as well. And your data augmentation or whatever. Everything you need to reproduce your results.

And for the love of Knuth, DO NOT OPTIMIZE YOUR CODE. Dumb code is good code in science. You would not believe what kinds of havoc some algorithms wreaked on my systems in the name of optimization. Optimizations that made a ten-year-old algorithm run in two nanoseconds instead of four (vastly exaggerated). Optimizations that obfuscated otherwise perfectly reasonable algorithms.

The goal is reproducibility.

Re: Ask HN: Should I publish my research code?

#142

Post by klyrs »

archontes wrote: post #67
del82 wrote: post #25

> it's substantially more work to clean and organize the code for publishing, it will increase the surface for nitpicking and criticism (e.g. coding style, etc). Matt Might has a solution for this that I love: Don't clean & organize! Release it under the CRAPL[0], making explicit what everyone understands, viz.: "Generally, academic software is stapled together on a tight deadline; an expert user has to coerce it int…

I'd like to piggyback and say that increasing the surface for nitpicking and criticism is exactly why OP should release his code. It improves the world's ability to map data to the phenomenon being observed. It becomes auditable . Certainly don't clean it up unless you're going to repeat the experiment with the cleaned up code.

Agreed on both points! As somebody who bridges research and production code, I can typically clean code faster than I can read & understand it. It really helps to have a known-good (or known-"good") example so that I can verify that my cleanup isn't making undesired changes.

And, yeah. I've found some significant mistakes in research code -- authors have always been grateful that I saved them from the public embarrassment.

Re: Ask HN: Should I publish my research code?

#143

Post by maurits »

Given the enormous amount of papers that come out, I personally tend to read papers that come with code (and data) first.

For me, it shows the authors are confident yet also open to critique. Which is a wonderful thing.

Secondly, I usually need the code to really understand the paper.

Re: Ask HN: Should I publish my research code?

#144

Post by waoush »

In my experience, typically researchers only publish the relevant and/or core algorithms of their research. If you would like, you can always publish the code to Github (if it isn't already), and reference it in your paper.

If it is too much work to refactor the code for publishing, you can also just publish pseudocode.

I don't think anyone will nitpick or criticize coding style or things like that unless it is particularly egregious (ie naming variables something vulgar etc). The point of research papers is to communicate new and valuable findings. If people in this conference or journal are nitpicking things like that, you may want to find a different place to submit your work.

I don't know what your field is, but in Computer Science I can't say I have ever known people to consider their code a competitive advantage. The only time they might shy from releasing code is when they think they can commercialize it or something.

Re: Ask HN: Should I publish my research code?

#145

Post by dgacmu »

You've gotten a ton of feedback already, but: Please do! Don't try to make it perfect. Just publish it. As the saying goes: "The perfect is the enemy of the good." Release the code you used to get YOUR results. You can always improve it later if it turns out people end up really interested in it.

(FWIW, I'm a professor at an R1 university. I give this advice to all of my Ph.D. students and strongly, strongly encourage them to put their code out there on our github.)

Re: Ask HN: Should I publish my research code?

#148

Post by ckemere »

As someone making a career in academia, I recognize both pros and cons here, but I think that the pros far outweigh the cons. Essentially, I think the question is one of identity - do you want your reputation to be "This investigator is the kind of person who's code is always available"? I know that as I evaluate job applications, funding proposals, or papers, I weigh this reputation highly, and consider the opposite "This investigator is the kind of person that hesitates to share their code" to be a big red flag.

BUT, I have definitely encountered the situation where I read a paper, then looked at the associated code, and found that the exciting result was entirely because of a bug. The reputation, "This investigator is someone who does shoddy, error-prone work" is probably the worst possible one.

Re: Ask HN: Should I publish my research code?

#149

Post by shadowgovt »

It seems to me that if it would take two months to replicate the actual mechanism, you're doing the world a favor by publishing what that two months of work resulted in.

If you want to do the world or further favor, get a grad student to read it first and indicate where they cannot follow the code. In my brief stint in academia, I saw very little overlap between brilliant theoreticians coming up with novel approaches and code to support them and people who knew how to write readable code.

Re: Ask HN: Should I publish my research code?

#150

Post by shadowgovt »

cblconfederate wrote: post #95

Depends on the field. Let's assume you are not in math /CS/Physics What can go "wrong" - Someone may find a minor rounding error and now you have to issue a correction to the paper which, laudable as it is, is a bad thing - You 'll end up having to maintain an open-source-something and possibly forks - Your open source code may end up as a github repo in which you are just one of the contributors, not the owner and o…

Isn't it worse for everyone if the paper but not the code is published and somebody doesn't find the error?

The world suffers for acting off of untrue information, and eventually the author suffers for having been wrong all this time.

Post reply on HN