Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Should I publish my research code?

news.ycombinator.com

Post reply on HN
Print view View original

261–270 of 363 posts

Re: Ask HN: Should I publish my research code?

#261

Post by bee_rider »

leksak wrote: post #195
inanutshellus wrote: post #180

Earlier quoted context omitted.

The point of the license is to set your expectations as low as possible. Then, when you actually /do/ get support, you'll be ecstatic rather than non-plussed.

When phrased like this, > 4) You recognize that any request for support for the Program will be discarded with extreme prejudice. There is no way I'd even make a request for support.

Exactly. If nothing else, a request for support has a chance of being an indication that there's somebody else in the field that cares about some aspect of the problem. I might not act on it, but it is good to have some other human-generated signal that says "look over there."

Re: Ask HN: Should I publish my research code?

#262

Post by bmitc »

cycomanic wrote: post #247
bmitc wrote: post #210

Earlier quoted context omitted.

> Academic code is about "proof of concept." Why does he think that but presumably not the same about the paper itself and the “equations”, plots, etc. contained within? It’s really not that hard to write pretty good code for prototypes. In fact, I can only assume that he and other professors never allowed or encouraged “proof of concept” code to be submitted as course homework or projects.

I think you don't understand the realities of work in scientific/academic organisations. Unless you work in computer science you likely never received any formal education on programming except for some numerical methods in c/matlab/fortran course during your studies (which often also focus more on the numerical methods and not the programming). So pretty much every person, just learned by doing. Moreover you are not…

I understand it fine. Like I said, it’s really not that hard to write pretty good code for prototypes. I'm not saying the code needs to be architected for scale or performance or whatever else needless expectation. I don't have a formal education in programming or computer science and write clean code just fine, as do some other non-computer science people I've worked with in advanced R&D contexts. And then some (many?) don't. It's not really about educational background, it's more about just being organized. Even when someone is "just" doing science, a lot of times, the code explicitly defines what's going on and has major impacts on the results. (Not to mention that plenty of people with computer science backgrounds write horrible code.)

If code is a large part of your scientific work, then it's just as important as someone who does optics keeping their optics table organized and designed well. If one is embarrassed by that, then too bad. Embarrassment is how we learn.

Lastly, you're describing problems with the academic world as if they are excuses. They're reasons but most people know the academic world is not perfect, especially with what is prioritized and essentially gamified.

Re: Ask HN: Should I publish my research code?

#263

Post by TremendousJudge »

andrewzah wrote: post #258
TremendousJudge wrote: post #244

Earlier quoted context omitted.

I never thought about this, but it's right. However, to get more nitpicky, most of the uses of "Comput[er|ing] Science" should be replaced with "Computer Engineering" anyway. If you are building something for a company, you are doing engineering, not science research

The average developer isn't often doing "engineering". Until we have actual standards and a certification process, "engineer[ing]" doesn't mean anything. The average software developer doesn't even know much math. Right now, "software engineer" basically means "has a computer, -perhaps- knows a little bit about what goes on under the hood".

I'm not talking about the "average developer", I'm talking about college graduates having a "Computer Science" degreee but in practice being "Computer engineers"

Re: Ask HN: Should I publish my research code?

#264

Post by mrek0 »

It would be nice if everybody would publish code for their papers. But in a field where most people don't do it, releasing your code will probably not be beneficial for you due to the loss of the competitive advantage. I know for people with cs background this sounds weird but it is reality n academia.

In your position, I would only release code which is not too hard to reproduce anyway or which only provides negligible competitive advantage for you. I mainly have "normal" paper in mind (experiments or data analysis) - if the main contribution is, for example, an algorithm which you want people to use, the you should publish an implementation obviously.

Re: Ask HN: Should I publish my research code?

#265

Post by born2discover »

This one depends on both the field you are in as well as your own academic philosophy OP.

If the paper is enough to reproduce the results AND cleaning up the code can/is tedious, then adding the "code and data are available upon request" note seems both fair and justified.

That way, whoever wants the code can still ask for it and it does not lay an unnecessary burden on the author.

Re: Ask HN: Should I publish my research code?

#266

Post by sitkack »

andrewzah wrote: post #258
TremendousJudge wrote: post #244

Earlier quoted context omitted.

I never thought about this, but it's right. However, to get more nitpicky, most of the uses of "Comput[er|ing] Science" should be replaced with "Computer Engineering" anyway. If you are building something for a company, you are doing engineering, not science research

The average developer isn't often doing "engineering". Until we have actual standards and a certification process, "engineer[ing]" doesn't mean anything. The average software developer doesn't even know much math. Right now, "software engineer" basically means "has a computer, -perhaps- knows a little bit about what goes on under the hood".

IfSmith

Re: Ask HN: Should I publish my research code?

#268

Post by asddubs »

erk__ wrote: post #252
actually_a_dog wrote: post #238

Earlier quoted context omitted.

Maybe a little OT, but, I'd rather it be called "comput ing science." Computers are just the tool. I believe it was Dijkstra who famously objected to it being called "computer science," because they don't call astronomy "telescope science," or something to that effect.

Peter Naur agreed with that which is why it is called "Datalogi" in Denmark and Sweden, his English term Datalogy never really caught on though. He wrote it in a letter to the editor in 1966 and it has pretty much been used since then here as Naur founded the first Datalogy university faculty a few years later. This also lead to a strange naming now we have data science as well where it is called "Data videnskab" whi…

informatics may be the closest english analogue. incidentally also what computer science is called in german (informatik)

Re: Ask HN: Should I publish my research code?

#270

Post by psmirnov »

albertzeyer wrote: post #197

> The paper itself is enough to reproduce all the results. No, this is almost never the case. It should be. But it cannot really be. There are always more details in the code than in the paper. Note that even the code itself might not be enough to reproduce the results. Many other things can matter, like the environment, software or library versions, the hardware, etc. Ideally you should also publish log files with a…

I agree with almost all of this, however I believe that publishing random seeds is dangerous in its own way.

Ideally, if your code has a random component (MCMC, bootstrapping, etc), your results should hold up across many random seeds and runs. I don’t care about reproducing the exact same figure you had, I want to reproduce your conclusions.

In a sense, when a laboratory experiment gets reproduced, you start off with a different “random state” (equipment, environment, experimenter - all these introduce random variance). We still expect the conclusions to reproduce. We should expect the same from “computational studies”.

Post reply on HN