If your field is not embracing open source yet, you should go for it ASAP. I believe in the end the field will recognize the benefits and move towards that and the sooner you are the larger impact you will make.
Ask HN: Should I publish my research code?
51–60 of 363 posts
Re: Ask HN: Should I publish my research code?
#52You're right, it is substantially more work to clean and organize the code for publishing. Being open about your work does make the attack surface much larger and more likely to be nitpicked, criticized, have an error found, etc.
But it is more honest. Whatever you think about the effort required to do this, there's value in honesty.
Here is an example of my own scientific work:
- paper [0]
- preprint [1]
- GitHub [2]
It certainly wasn't easy to get all of this done. But doing this can also be a guide for others. They get to see exactly what you've done so that they don't waste months on the exact implementation. They can see where maybe you've made some mistakes to avoid them. They can see so much of the implicit knowledge that is left out of your paper and learn from it. Your code isn't going to be perfect, but what paper is, either?
Everyone will be a critic, anyway, so make it easy to pick up criticism of the stuff you feel the least confident in and do better next time. You won't get better if no one sees your code.
[0]: https://cancerres.aacrjournals.org/content/81/23/5833
[1]: https://www.biorxiv.org/content/10.1101/2021.01.05.425333v2
[2]: https://github.com/LupienLab/3d-reorganization-prostate-canc...
Re: Ask HN: Should I publish my research code?
#53Earlier quoted context omitted.
I can attest to this. I myself am victim of this. My undergraduate thesis was plagiarized by two other papers. Code was 80% the same, they just added some trivial things. No citing of my work at all. Look at my other comment for more explanation - if you are working under less known advisor, or at less known university, there is a high chance that this will happen if your work is good.
Write to the journal they published in and call them out.
This kind of thing is best done after the thesis is in the bag. A student is racing against the clock. Grad study has many kinds of hard failures. At the most extreme, your advisor could up and die. The focus has to be on finishing. That's how you get out.
Re: Ask HN: Should I publish my research code?
#54>> Besides, many scientists look at code as a competitive advantage so in this case publishing the code will be removing the competitive advantage. That "competitive advantage" is just holding everyone back, slowing progress. This is particularly annoying to hear coming from "research" which I thought was supposed to be advancing the state of the art for the benefit of society. That's ostensibly the reason for publis…
Agreed, perhaps there is some competition in citations for follow up work, releasing source code makes it easier to get 'scooped' on your future plans section? (not saying that optimizing for citations is a good thing)
Re: Ask HN: Should I publish my research code?
#55Yes, you should publish. Mainly because it will give you a sense of accomplishing something, also, nobody cares really :). Especially about old code, if they do that's even better.
Re: Ask HN: Should I publish my research code?
#56In my view and personal experience, the pros outweigh the cons:
* You increase the impact of your work and as a consequence also might get more citations.
* It's the right thing to do for open and reproducible research.
* You can get feedback and improve the method.
* You are still the expert on your own code. That someone picks it up, implements an idea that you also had and publishes before you is unlikely.
* I never got comments like "you could organize the code better" and don't think researchers would tend to do this.
* Via the code you can get connected to groups you haven't worked with yet.
* It's great for your CV. Companies love applicants with open-source code.
Re: Ask HN: Should I publish my research code?
#57There's a lot of advice here, but very little data to support any of it. Since you're a scientist, why not take an experimental approach to answering this question? Publish your code, for one (selected) paper. Monitor (a) the download log, and (b) the emails you get related to your code.
I hypothesize that you will see some combination of three effects: (1) you will get lots of downloads (which means people are using your code, good work!), perhaps with lots of follow-up emails and perhaps not depending on what the code does; (2) you will get lots of emails from random nutjobs looking to pick holes in your work, and you will waste your time answering them; (3) you will get almost completely ignored.
Whatever the outcome, I think a lot of people would be interested in to hearing about what you learn.
Re: Ask HN: Should I publish my research code?
#58I'd only clean it of stuff like passwords and such, and add a header that the code is provided as-is.
You could add a disclaimer that the code was worked on until it provided a satisfactory result, and no further, and is not intended for (any) use. You might even add that, except for outright, actual errors that affect the result of the research, comments are discouraged.
I often publish very bad code, terrible terrible spaghetti, it's not how I write code at my job, because at my job, I'm paid to produce not only working and correct code, but also code that is maintainable and understandable and follows certain practices.
However, my hobby is not writing corporate code, but writing code that get done what I want to get done, nothing more, and sometimes less. It might even have actual bugs in it that I can plainly see and don't care about because they don't affect my uses
If people can't tell the difference, I don't care, not my problem. If a future employer can't tell the difference, I won't work with them.
Re: Ask HN: Should I publish my research code?
#59> it's substantially more work to clean and organize the code for publishing, it will increase the surface for nitpicking and criticism (e.g. coding style, etc).
I should take a couple of hours. The code works? You know how to reproduce what you did, right? It shouldn't be perfect. Shouldn't even pass code review. Should just work.
> many scientists look at code as a competitive advantage so in this case publishing the code will be removing the competitive advantage.
Well depends on the field I guess, but you also want recognition and impact. What is the point of publishing a result no one uses?
Re: Ask HN: Should I publish my research code?
#60The code should be published, and knowing this, researchers will hopefully try avoid certain commonly harmful practices. One of these is re-using the same script to run slightly different models by editing some of the hard-coded parameters. I've myself found more than one mistake in someone else's reported results due to this sort of thing. But identifying it was quite a bit of trouble because the record of what was ran was erased when they moved on to the next model.
What I would not expect from people is code that would necessarily run in your environment. For example, in many cases, the paths are going to be hard-coded, for a variety of reasons. It might be ideal to write code that will just work, in a reproducible environment, but that often takes more work than people are willing to commit to, given all the other things they have to do.
Finally, cleaning up your code for presentation is a final opportunity for you to discover any mistakes before you publish and then later have an embarrassing public retraction.