Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Should I publish my research code?

news.ycombinator.com

Post reply on HN
Print view View original

21–30 of 363 posts

Re: Ask HN: Should I publish my research code?

#21

Post by CJefferson »

nomilk wrote: post #17

> it will increase the surface for nitpicking and criticism Anyone who programs publicly (via streaming, blogging, open source) opens themselves up for criticism, and 90% of the time the criticism is extremely helpful (and the more brutally honest, the better). I recall an Economist magazine author made their code public, and the top comments on here were about how awful the formatting was. The criticism wasn't unwar…

Do you really mean 90% of the criticism is extremely helpful? Or did you mean 90% was useless.

I've published 100,000s of lines of code from my research over 20 years, and I think I've had exactly one useful comment from someone who wasn't a close collaborator I would have been sharing code with anyway.

I still believe research code should be shared, but don't do it because you will get useful feedback.

Re: Ask HN: Should I publish my research code?

#22

Post by nicec2 »

analog31 wrote: post #18

Depends on the climate of the field you're in, and where you're at in your career. There are fields where entire research groups routinely harvest preliminary ideas from graduate student publications, and then finish them and rush to publication before the student realizes what's happened. I'd say, grad student owes nobody anything until they finish, because they're bearing the greatest risk of losing priority, and t…

I can attest to this. I myself am victim of this. My undergraduate thesis was plagiarized by two other papers. Code was 80% the same, they just added some trivial things. No citing of my work at all.

Look at my other comment for more explanation - if you are working under less known advisor, or at less known university, there is a high chance that this will happen if your work is good.

Re: Ask HN: Should I publish my research code?

#23

Post by jaclaz »

As always I may be wrong, but the (admittedly very few) times I find an article/paper based or revolving around code that is interesting/useful for some purposes I read the "code is available on request" (or similar) as the (in-) famous Fermat's Last theorem note: Hanc marginis exiguitas non caperet.

Nowadays margins are large enough and cost nothing or next to nothing, and you don't probably have any other use of your code, so what would be the advantage for you in not publishing it?

What kind of competitive advantage does it give to you? (what many scientists think might be not as relevant as what you think about this "competitive advantage" secifically in your specific case/field)

About "cleaning it", why?

I mean, if as-is it works (but it is "ugly") it still works, what if in the process of "cleaning it" you manage to introduce a bug of some kind?

Unless you plan to also re-test it after the cleaning, I guess it would be better to not clean it at all.

Re: Ask HN: Should I publish my research code?

#24

Post by cowpig »

analog31 wrote: post #18

Depends on the climate of the field you're in, and where you're at in your career. There are fields where entire research groups routinely harvest preliminary ideas from graduate student publications, and then finish them and rush to publication before the student realizes what's happened. I'd say, grad student owes nobody anything until they finish, because they're bearing the greatest risk of losing priority, and t…

> There are fields where entire research groups routinely harvest preliminary ideas from graduate student publications, and then finish them and rush to publication before the student realizes what's happened.

Can you provide a source, or example of this? What does the Amazon of academia look like?

Re: Ask HN: Should I publish my research code?

#25

Post by del82 »

> it's substantially more work to clean and organize the code for publishing, it will increase the surface for nitpicking and criticism (e.g. coding style, etc).

Matt Might has a solution for this that I love: Don't clean & organize! Release it under the CRAPL[0], making explicit what everyone understands, viz.:

"Generally, academic software is stapled together on a tight deadline; an expert user has to coerce it into running; and it's not pretty code. Academic code is about 'proof of concept.'"

[0] https://matt.might.net/articles/crapl/

Re: Ask HN: Should I publish my research code?

#26

Post by CJefferson »

I've posted a huge amount of academic code (I've linked to a small number at the end). I think you should, but it won't help advance your career immediately. However, I still think it's better for science.

What is useful is if you can produce code people can build on and do their own cool stuff with -- then they will cite you. However, getting something to a state where it is tested for all reasonable inputs, has some basic docs, etc. is a hard untaking.

https://github.com/minion/minion (C++ constraint solver)

https://github.com/stacs-cp/demystify (Python puzzle solver)

https://github.com/peal/vole (Rust group theory solver)

Re: Ask HN: Should I publish my research code?

#27

Post by dagw »

tagh wrote: post #8

Publishing the code does have some selfish benefits too: better chance of people building on your research (and citing it).

It is at least as likely that they'll take your code, integrate it into their own research, and never mention you at any point in the process. So you have to be OK with that.

Re: Ask HN: Should I publish my research code?

#28

Post by analog31 »

cowpig wrote: post #24
analog31 wrote: post #18

Depends on the climate of the field you're in, and where you're at in your career. There are fields where entire research groups routinely harvest preliminary ideas from graduate student publications, and then finish them and rush to publication before the student realizes what's happened. I'd say, grad student owes nobody anything until they finish, because they're bearing the greatest risk of losing priority, and t…

> There are fields where entire research groups routinely harvest preliminary ideas from graduate student publications, and then finish them and rush to publication before the student realizes what's happened. Can you provide a source, or example of this? What does the Amazon of academia look like?

Biology, and synthetic chemistry. Unfortunately all anecdotal. I live near a major research university, have lots of friends who are involved at all levels, and relatives who are even closer to it. It tends to be in areas that require minimal capital investment to pivot into a new study. Also, the student pursuing the original idea is hampered by their own emerging skills. "My student's thesis just got scooped" is something that every professor has experienced or knows about.

My field, physics, much harder. Building my experiment required a bunch of expensive equipment (maybe half a million in today's dollars), gear that I built myself, the technique of operating it, and so forth.

My career, much harder. I work in business. You learn about my ideas when a patent comes out. ;-)

Re: Ask HN: Should I publish my research code?

#29

Post by bagacrap »

nomilk wrote: post #17

> it will increase the surface for nitpicking and criticism Anyone who programs publicly (via streaming, blogging, open source) opens themselves up for criticism, and 90% of the time the criticism is extremely helpful (and the more brutally honest, the better). I recall an Economist magazine author made their code public, and the top comments on here were about how awful the formatting was. The criticism wasn't unwar…

This. Feedback (less loaded term than "criticism") is something you should want. You can obviously ignore tabs vs spaces types of comments but if your code takes 2 months to get right then it probably still has bugs in it after 2 months and it would be a win if others started finding them for you. Also, if the style is really that bad then it could be obscuring bugs that would otherwise be easy to spot (missing braces, etc), and you might find bugs while fixing it up.

ps always use an auto formatter/linter. I can't believe we ever used to live without them. So much time used to be wasted re-wrapping lines manually and we'd still get it wrong.

Post reply on HN