Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Should I publish my research code?

news.ycombinator.com

Post reply on HN
Print view View original

81–90 of 363 posts

Re: Ask HN: Should I publish my research code?

#81

Post by TrueDuality »

clintonb wrote: post #71

If you publish it, please make sure it works today, and can work tomorrow. Pin the versions of any dependencies, or bundle them if feasible. Also, include basic instructions for running your code. I helped my wife with a replication study that should have been straightforward, and I was unable to get the code running after about a week. I don’t necessarily believe the research was suspect, but broken code does draw m…

I disagree that this is a requirement. It is a strong nice to have, but even if the code isn't maintained or runnable on other systems it is incredibly valuable as a reference when reading through ambiguous sections in a paper to go look at what the author was trying to do.

Will it take some work to reproduce the results with broken code? Sure, but its better to have the code than not. Cleaning it up might be the straw that prevents that code from getting released at all.

Re: Ask HN: Should I publish my research code?

#83

Post by acaloiar »

In the past I've chosen to publish key algorithms. Publishing your entire code base can become a substantial demand for your support. As an open source project supported by one person, that can be very demanding.

So identify what's most critical or novel about your work and publish that.

Re: Ask HN: Should I publish my research code?

#84

Post by armchairhacker »

del82 wrote: post #25

> it's substantially more work to clean and organize the code for publishing, it will increase the surface for nitpicking and criticism (e.g. coding style, etc). Matt Might has a solution for this that I love: Don't clean & organize! Release it under the CRAPL[0], making explicit what everyone understands, viz.: "Generally, academic software is stapled together on a tight deadline; an expert user has to coerce it int…

Anecdotally most of the research papers I see and have worked on publish their code but don't really clean it up. Even papers by big companies like Microsoft Research. Still significantly better than not publishing the code at all.

Re: Ask HN: Should I publish my research code?

#85

Post by phdelightful »

I assume computer science results without published artifacts to be fake. When it's so easy to publish and run code, if the researcher can't even do that then I assume the code does not work and thus the results must be fabricated. If your work has trade secrets or something, research publication with peer review is the wrong way to distribute your results.

Computer engineering on novel systems is a bit harder, but a /complete/ spec of the system (enough for someone to precisely rebuild it) should be published in that case. Remote access on request to the prototype would be better.

Re: Ask HN: Should I publish my research code?

#86

Post by drdec »

Publish it on GitHub (or GitLab or your code hosting service of choice).

Then answer any criticism about it by asking for a PR.

To preempt code style complaints find a code formatter for your language and run everything through that first.

Refer to the repository in your paper, but don't put a link. Create a little bit of friction to get to the repo to discourage the casual readers who don't really need the code from popping over too easily.

Re: Ask HN: Should I publish my research code?

#87

Post by taubek »

If you are for Open science(https://en.wikipedia.org/wiki/Open_science_data), go ahead and publish it. Would you ever publish the code on some GIT platform? If you would, this would be the equivalent. A lot of researches don't want to give their data to the public, but if locking their data they are just making harder for others to confirm or improve their findings. I guess sometimes there are legal issues behind that, and sometimes it is pure ego.

Re: Ask HN: Should I publish my research code?

#88

Post by lanstin »

chrisseaton wrote: post #19

> it will increase the surface for nitpicking and criticism You're supposed to welcome criticism and 'nitpicking' as a scientist.

Not from untrained randos on the internet. Signal noise ratio and prior of “not a nutjob” have to be high enough to offset the cost of lost focus.

Re: Ask HN: Should I publish my research code?

#89

Post by nicec2 »

spaniard89277 wrote: post #43
nicec2 wrote: post #22

Earlier quoted context omitted.

I can attest to this. I myself am victim of this. My undergraduate thesis was plagiarized by two other papers. Code was 80% the same, they just added some trivial things. No citing of my work at all. Look at my other comment for more explanation - if you are working under less known advisor, or at less known university, there is a high chance that this will happen if your work is good.

Write to the journal they published in and call them out.

Matter of fact, I did, even with help of my advisor. The journal did not take any action (it is Q1 open access journal), since they come up with all kinds of mental gymnastics why it is not copied (which boiled down it is NOT 100% the same).

That was for the first occurrence. For the 2nd one, we just did not bother because it hurts my advisor's reputation as well. It is not in the interest of journal to admit the mistake once they made it -- they will fight you about it and try to keep their reputation/image up.

Re: Ask HN: Should I publish my research code?

#90

Post by taneq »

Please do! Maybe the paper is technically enough to reproduce the results but if other researchers can start from a working example, they can both verify your results and extend them with more original research far faster.

While any published code receives some nitpicking and bikeshedding, most academic code is terrible so unless you literally use random joke/meme variable names as your only 'documentation' (I wish I were joking) you're not going to look bad to anyone who matters.

Post reply on HN