Ask HN: Should I publish my research code? • Hacker News Board

Re: Ask HN: Should I publish my research code?

#171

Post by dr_zoidberg » Fri, Jan 14, 2022, 4:43 PM UTC

anon_123g987 wrote: post #155

dabreegster wrote: post #139

Earlier quoted context omitted.

In my experience, the published paper is super vague on the approach, and implementing it without further references is really hard. I'm not necessarily arguing that papers should get longer and more detailed to counter this; expressing the details that matter in code seems like a more natural way to communicate anyway. Why trust results if you can't see the methodology in detail and apply the approach to your own da…

> In my experience, the published paper is super vague on the approach, and implementing it without further references is really hard. This is my experience, too, and in my opinion this is exactly what has to change for really reproducible research, not ready to run software supplied by the author. There are many good arguments in support of publishing code, but reproducibility is not one of them, that's all I'm sayi…

And just like OP said, it generally takes a couple of months to go from paper to working code. I've implemented a few papers as code as a side-gig for a while, and I wouldn't mind having a baseline from the authors to check and see if I'm following the paper correctly!

Re: Ask HN: Should I publish my research code?

#172

Post by modeless » Fri, Jan 14, 2022, 4:43 PM UTC

> The paper itself is enough to reproduce all the results.

Every researcher thinks this, and it's always wrong. If you care about scientific progress, publish the code and data.

Besides, available code should cause more people to look at your work and ultimately cite it.

Re: Ask HN: Should I publish my research code?

#173

Post by [deleted] » Fri, Jan 14, 2022, 4:44 PM UTC

[deleted]

Re: Ask HN: Should I publish my research code?

#174

Post by jhrmnn » Fri, Jan 14, 2022, 4:46 PM UTC

I'm a scientist too, in computational chemistry. To me, releasing the research code that accompanies a paper is an imperative. Increasingly, journals or individual (peer) reviewers demand it. It's essential for reproducibility. I consider the work that goes into making the code releasable simply part of the job.

Re: Ask HN: Should I publish my research code?

#175

Post by bee_rider » Fri, Jan 14, 2022, 4:48 PM UTC

Publishing code could be nice, if for example your code has a commercial application and a company wants to use it, a reference implementation might be nice.

Reproducibility -- I dunno. A re-implementation seems better for reproducibility. The paper is the specific set of claim, not the code. If there are built-in assumptions in your code (or even subtle bugs that somehow make it 'work' better), then someone who "reproduces" by just running your code will also have these assumptions.

Coding time -- are you sure? Professional coders are pretty good. If you have, for example, taken the true academic path and written your code in FORTRAN, there's every chance that a professional could bang out a proof of concept in Python or C++ in like a week (really depends on the type of code -- EIGEN and NUMPY save you from a whole layer of tedium that BLAS and LAPACK 'helpfully' provide). Really good pseudocode might be more useful than your actual code

Another note -- personally I treat my code as essentially the IP of my advisor. (He eventually open sources most things anyway). But do check on the IP situation if you want to open source it yourself. If you are working as a research assistant, some or all of your code may belong to you University. They probably don't care, but it is better to have the conversation before angering them.

Re: Ask HN: Should I publish my research code?

#176

Post by bumby » Fri, Jan 14, 2022, 4:49 PM UTC

UncleOxidant wrote: post #159

anon_123g987 wrote: post #108

Earlier quoted context omitted.

> It's the right thing to do for open and reproducible research. Everybody here talks about how publishing code helps (or even makes possible) reproducibility, but this is not true, on the contrary, it hinders it. Reproducing the results does not mean running the same code as the author and plotting the same figures as in the paper. This is trivial and good for nothing. Reproduction is other researchers independently…

Often the text in a paper that describes some algorithm will not be completely clear or complete. Providing the code fills in those blanks. I've taken a paper with code and gone through the code line by line comparing it with what was described in the paper. The code often ends up clarifying ambiguities. In some cases there's an outright disagreement between the paper and the code - that's where the feedback comes in…

What are your thoughts on including pseudocode within the paper directly? It seems to clear up some of the ambiguities while adding brevity since it doesn't provide the somewhat frivolous details of implementation. I think it also limits some of the potential stylistic critiques.

Re: Ask HN: Should I publish my research code?

#177

Post by emmericp » Fri, Jan 14, 2022, 4:51 PM UTC

Yes, you should publish it. Don't bother cleaning it up if you don't feel like it. No one will judge you for the code quality.

Published terrible code is far better than unpublished code.

Re: Ask HN: Should I publish my research code?

#178

Post by jonas21 » Fri, Jan 14, 2022, 4:51 PM UTC

rvanlaar wrote: post #40

What a great question. You've come to the right community. My main concern would be to make sure there are no passwords or secret keys in the data, not how it looks. You'll open yourself up for comments. They may be positive or negative. You'll only know how it pans out afterwards. Is the code something that you'll want to improve on for further research? If so publish it on github. It opens the way for others to con…

> My main concern would be to make sure there are no passwords or secret keys in the data, not how it looks.

Also personal data of any human subjects.

Re: Ask HN: Should I publish my research code?

#179

Post by kyruzic » Fri, Jan 14, 2022, 4:52 PM UTC

Yes you should absolutely publish it. I wrote a paper about modelling radio wave propagation through the ionosphere all my code for it is on my github. The reason you should is simple, you are providing proof that your numbers aren't just made up.

Re: Ask HN: Should I publish my research code?

#180

Post by inanutshellus » Fri, Jan 14, 2022, 4:52 PM UTC

bee_rider wrote: post #138

del82 wrote: post #25

> it's substantially more work to clean and organize the code for publishing, it will increase the surface for nitpicking and criticism (e.g. coding style, etc). Matt Might has a solution for this that I love: Don't clean & organize! Release it under the CRAPL[0], making explicit what everyone understands, viz.: "Generally, academic software is stapled together on a tight deadline; an expert user has to coerce it int…

I like it so far, other than 4) You recognize that any request for support for the Program will be discarded with extreme prejudice. I think that should be a "may" rather than a "will." If I find out someone is using my obscure academic code, and they ask for help, I'd be pretty pumped to help them (on easy requests at least).

The point of the license is to set your expectations as low as possible. Then, when you actually /do/ get support, you'll be ecstatic rather than non-plussed.