11) Invariant: Someone should be able to go from a referenced paper or book to your code without scratching their head too much. This can be approached by either fortifying the documentation in the code or by specificity in a methods paper.
Simple rules for documenting scientific software
41–50 of 55 posts
Re: Simple rules for documenting scientific software
#42Earlier quoted context omitted.
Right. You do not always get outcompeted. But you run the risk. And yes, having good architecture in parts you need to change often is someting that pays of. As does having tests in section of the code that is brittle, hard to reason about or historically buggy. As does writing documentation on things that you had to spend an annoying amount of time on to understand. And I think that most researchers understand that…
I think I misunderstood your position earlier. My experience with academic codes seems to differ significantly from yours. It is not uncommon at all for scientific software to have no documentation at all in my experience, and very little if anything in terms of tests. I agree with you that it does not make sense to have detailed documentation and tests for all parts of scientific software. One must prioritize. What…
You are not wrong at all. And the amount of test coverage and documentation on needs depends a lot on the field, the numerical methods, the indended users, the type of the code (simulation vs analysis vs plotting) and so on.
The thing where I seem to have a totally different view from most of HN is that I think that we make valid tradeoffs, and that undocumented function and tests that only cover 40% of the code base is a perfectly fine state of affairs, instead of being caused by scientists that are too stupif for basic software engineering.
What a lot of software people also do not get at all is that variables such as x, v and a might be perfectly descriptive variable names.
Re: Simple rules for documenting scientific software
#43Earlier quoted context omitted.
> ...I don't think it's always true... I will have to re-evaluate my thinking on this. My knee-jerk response is to say this is wrong. I have seen too many cases where people report initial results on some elaborate model as if they explain more aspects of the data, but then the results fail to generalize, and the theory, model or implementation turns out to be flawed or fragile. It gets a publication, but the state o…
Yes, "being careful" refers to a variety of different techniques, not just better software engineering. What has ended up being publishable in my case wasn't related to better software engineering, rather, being more careful about model derivations and more rigorous about the available data. But it could have been software improvements. To be clear, I don't think everyone needs to be as careful as I am. A few researc…
This is a very important point. Your budget of "being careful" might be best spent on better software engineering. But it is also possible that you should rather spend it on other things such as better mathematical model, better input data sets or something else. And a (sub)field prospers most if different groups and researchers spent there (very limited and expensive) time on different things. Trying to shame everybody into following the magic 27 rules of software engineering is a step back, not forward.
Re: Simple rules for documenting scientific software
#44If there is one thing I have about HN it is how smug they are about software engineering. The leading reason why scientific software stinks (and a fair fraction does) is not because scientists and software engineeres in that field suck, but because there is very strong incentives AGAINST writing better software. Remember: This is not the 17th javascript framework, but software for problems that we don't understand go…
If you have "yeah I guess it works" code, you're likely producing "yeah I guess it's right" science, and that's not really the point it seems.
But yes, you're right. Perfecting things is at cross-purposes with the frenetic pace of research. Crappy untested code gets written under duress. Resist it. That needs to change, just as the whole funding model needs to reward verification more than new discovery... But back to the code, if the code is wrong, the science is likely to be as well.
Re: Simple rules for documenting scientific software
#45I manage a team of power plant design engineers writing complex scientific HPC software, mostly in Python (which drives Fortran 77 codes behind the scenes, among other things). It's been a long haul but through the years we've learned a lot of good lessons and are pretty productive at what I consider at least moderately good code. Everyone starts off by reading Clean Code and taking basic proficiency training. We kno…
One things that has worked well for us, is to give a very explicit invitation to every new code user to question every bit of the code. If something is not obvious, ask why it is there and why it is the way it is. Once the new user understand the code he then adds a comment to the code or explanation in the documentation. After four new users very little code is left where the design considerations is undocumented.
Re: Simple rules for documenting scientific software
#46A really great idea: put DOIs in your function documentation. Writing things like: Implements equation 3.2 of Foo et. al. (2005), doi:10.2.3/baz has saved me no end of pain in the past.
I do this as well. I started doing it in college mainly as a citation but I find it helpful to do a similar thing for this reason as well as for any of the less mainstream algorithms or theorems that would provide further information.
Re: Simple rules for documenting scientific software
#47If there is one thing I have about HN it is how smug they are about software engineering. The leading reason why scientific software stinks (and a fair fraction does) is not because scientists and software engineeres in that field suck, but because there is very strong incentives AGAINST writing better software. Remember: This is not the 17th javascript framework, but software for problems that we don't understand go…
By this argument, chemists should never wash glassware. Keep re-using it until it's too dirty, then through it out and make new beakers. Your tools affect the quality of your work. Work with shitty tools, get shitty results. I'll start believing your argument when scientists actually start publishing the code to go with their papers and make it reproducible.
Re-using code doesn't wear it out or contaminate it. This is one of the worst analogies I've heard here.
Re: Simple rules for documenting scientific software
#48Some good basic suggestions that should be followed when doing softwaare engineering. But they forgot one aspect:
Don't be too smart for your own sake when packaging your software. As an HPC administrator I stopped counting how many hours I spent installing scientific software with broken Makefiles, interactive installers, simple github dumps, unresolvable dependencies. Sadly this is quite prevelant in life sciences (I am looking at you bioconductor)
Re: Simple rules for documenting scientific software
#49I'm a 'scientific programmer' who learned how to code basically trial by fire.
My background was mechanical engineering which didn't focus on coding at all other than needing to use/learn MATLAB for several classes. Went to grad school for HPC/CFD and there I was given access to our group's simulation code and was let loose to implement whatever routines I needed to simulate my problems. The shared components were the input/output system and the primary routine drivers (time-stepping and fluid dynamics algorithms and the like), and what I mostly worked with were different constitutive models which hooked into the system. Parallelization was implemented via MPI and was mostly complete, so my only job with respect to parallelization was to make sure that my algorithms would work in parallel.
I ended up taking several programming courses, but these were 100% focused on topics like parallelization with MPI, shared memory parallelization, and optimizing code, and a short stint in GPU programming. I learned nothing about code management or best practices. Oh, and this was all using Fortran and C, though now I'm working with C++ and Python, but that's because the newer libraries seem to be C++ and Python is just easy to glue everything together with.
My general programming knowledge isn't super great, but sometime this year I managed to download an open source iOS app and without any prior knowledge of Xcode or iOS programming/Swift was able to figure out how to implement something that was missing.
This is already a lot of words, but lately I've been thinking about getting out of academia and getting into actual software development cause I figure I kinda do that anyway. Obviously the easiest connections I could make would be to maybe work for companies like ANSYS that develop CFD software, but I feel like my programming knowledge is seriously lacking for that. You can give me a scientific paper that describes an algorithm to do a thing and I'd have no issue implementing it into some existing codebase but I read words written in this discussion like "code smell" and "CI" and I have no idea what these things are.
Anyway, can anyone recommend me some books to read and/or provide some advice and/or anecdotes on jumping ship from HPC/CFD/scientific programming into a general programming developer career?
Re: Simple rules for documenting scientific software
#50Earlier quoted context omitted.
I write scientific code for a living and my code is very difficult for anyone else to maintain. It is not because my code is badly documented or written, it is because what it does is very complex. Every module is documented why it exists and what it does and the code straightforward to read, yet the interaction of all the modules is very complex as it reflects the underlying complexity of the problem the code is sol…
I don't buy it. I've worked with very complicated systems (ads at google) and it was pretty straightforward- it's the distributed systems communicating at RPC interfaces that is hard to understand. I've also worked with quantum chemistry codes and large supercomputer codes. The ones that were difficult to maintain were just badly written. There was an article recently about Jeff Dean and Sanjay Ghemawat- one of Sanja…
How would you make, say, a fourier transform or IIR filter design accessible to people who don't know any signal processing or complex numbers?
You can spend years studying these topics and their related math. Entire text books have been written on these subjects. Any one algorithm probably has multiple scientific publications, explaining how and why they work.
This much information can not fit into a few code comments and a README or two. Especially not in a format accessible to people without domain knowledge.