Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Should I publish my research code?

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 363 posts

Re: Ask HN: Should I publish my research code?

#11

Post by Icko »

PragmaticPulp wrote: post #2

Is your goal to help advance the science and our general knowledge? Publish the code. You don’t even need to clean it up. Just publish. Don’t worry about coding style nitpicks. Having the code and data available actually protects you from claims of fabrication or unseen errors in hidden parts of your research. On the other hand, if your goal is only to advance your own career and you want to inhibit others from opera…

Not sure if posting the paper only is even the best move. I personally never work with papers with no code published. Just not worth the effort to reproduce them, when I can use the second SOTA for nearly no performance penalty and much less effort.

All the groundbreaking papers in deep learning in the last decade had code published. So if you're aiming for thousands of citations, you need code.

Re: Ask HN: Should I publish my research code?

#14

Post by chriskanan »

Having a polished public implementation can lead to a massive increase in the number of citations a paper recieves, if it is really a useful system. Some of my papers I think would have received far fewer citations if I had not released the code. Of course, if it is a really niche area with only a handful of researchers, this may not be true.

Re: Ask HN: Should I publish my research code?

#15

Post by motiejus »

I highly recommend adding it. It doesn't have to be exposed, but is super useful for anyone who will want to reproduce or build upon your work later.

You can embed this to the PDF, e.g. see section A.1 [1] for how.

[1]: https://raw.githubusercontent.com/motiejus/wm/main/mj-msc-fu...

Re: Ask HN: Should I publish my research code?

#16

Post by lumost »

The economic incentive of science is for your work to be replicated and cited. Not publishing the code and data means your work is harder to reuse for subsequent studies and will hurt citations.

If it's uncommon to release code then I'd doubt anyone in the peer review will review it.

Re: Ask HN: Should I publish my research code?

#17

Post by nomilk »

> it will increase the surface for nitpicking and criticism

Anyone who programs publicly (via streaming, blogging, open source) opens themselves up for criticism, and 90% of the time the criticism is extremely helpful (and the more brutally honest, the better).

I recall an Economist magazine author made their code public, and the top comments on here were about how awful the formatting was. The criticism wasn't unwarranted, and although harsh, would have helped the author improve. What wasn't stated in the comments is that by publishing their code, the author already placed themselves ahead of 95% of people in their position who wouldn't have had the courage to do so. In the long run, the author will get a lot better and much more confident (since they are at least more aware of any weaknesses).

I'd weigh up the benefits of constructive (and possibly a little unconstructive) criticism and the resulting steepening of your trajectory against whatever downsides you expect from giving away some of your competitive advantage.

Re: Ask HN: Should I publish my research code?

#18

Post by analog31 »

Depends on the climate of the field you're in, and where you're at in your career. There are fields where entire research groups routinely harvest preliminary ideas from graduate student publications, and then finish them and rush to publication before the student realizes what's happened.

I'd say, grad student owes nobody anything until they finish, because they're bearing the greatest risk of losing priority, and the openness of science is being used against them. Nothing lost by waiting until they have their degree in the bag before sharing. Then clean it up and use it as part of your portfolio. Or append it to your thesis. Advancing science after you've secured your career is a fair compromise.

I love open source and open science, but also look back on my own graduate studies, and I chose a topic that was protected by virtue of a large capital investment plus domain knowledge that was not represented by code. Also, my thesis predates widespread use of the Internet. ;-)

Re: Ask HN: Should I publish my research code?

#20

Post by techwind »

anon_123g987 wrote: post #13

One argument against publishing code is that maybe there's an error (or more) in your code, which validates your possibly mistaken theory, and forcing an independent reimplementation by others would uncover this problem.

That's a good point I'd not considered. I suppose it's ultimately a risk scientists/researchers take to not lead humanity down a broken path however I'm of the firm belief the truth will "out" and transparency almost always trumps opaqueness
Post reply on HN