Simple rules for documenting scientific software

A really great idea: put DOIs in your function documentation.

Writing things like:

Implements equation 3.2 of Foo et. al. (2005), doi:10.2.3/baz

has saved me no end of pain in the past.

I write scientific code for a living and my code is very difficult for anyone else to maintain. It is not because my code is badly documented or written, it is because what it does is very complex. Every module is documented why it exists and what it does and the code straightforward to read, yet the interaction of all the modules is very complex as it reflects the underlying complexity of the problem the code is solving. Some code is hard to maintain because it is solving a complex problem that few people understand.

If there is one thing I have about HN it is how smug they are about software engineering. The leading reason why scientific software stinks (and a fair fraction does) is not because scientists and software engineeres in that field suck, but because there is very strong incentives AGAINST writing better software.

Remember: This is not the 17th javascript framework, but software for problems that we don't understand going in. And often enough we have not understood the problem all that well even after a decade when we write the third code. These codes are research codes. Ongoing experiments. The main goal is NOT to produce long-term maintainable software, but to produce scientific understanding and build intuition about the systems that are modeled. The code is just another tool among experiements, analytic calculations and back-of-the-envelope discussions on a white board.

Rewriting the code every 5 years is an insane proposition in software engineering, but completely ok in some fields of science.

Would I like better language support to check SI units for me? Sure. Would I like highly performant libraries for vector fields, that work with gcc 4.6 on a top 500 machine? Sure. Would I like to be allowed to spend time on fixing yeah-I-guess-it-works code? You bet.

But would I like HN to just up about "scientists just need to learn to code"? Oh hell yes! Because -- believe it or not -- we often DO know better. But fixing code is not what the taxpayers, what YOU, pay us for. We are paided to understand nature. And until you are willing to pay higher taxes and spend more money and science and to invest more into fixing long term infrastructure you really do not get to be so damn condescending.

This is both wrong and dangerous.

I do hear this sentiment often: "Hey dudes, it's just a prototype! Why waste time making it pretty? Who cares? We're doing SCIENCE here!"

But prototypes (and beginnings in general) are precisely the time to be extra careful. Wrong turns and self-delusion are costlier, not cheaper, when you're the one paving the road for others.

In research, there are many ways to lead yourself astray, due to the inherent chaos of novelty, even without software bugs completely flipping the outcome. The idea that writing shitty code somehow "saves time" and is only worthwhile for the "17th javascript framework" has to go.

Articulating your thoughts into a sane logical structure (aka code), with sane names and motivation examples and conceptual units, saves you time even in the short run. Never mind 5 years. It also helps you avoid publishing unreliable, brittle, "SOTA" nonsense… of which there's sadly so much.

Also don't forget to document all dependencies and their versions!