Live data from Hacker News

Simple rules for documenting scientific software

journals.plos.org

51–55 of 55 posts

Re: Simple rules for documenting scientific software

#51

I'm a 'scientific programmer' who learned how to code basically trial by fire. My background was mechanical engineering which didn't focus on coding at all other than needing to use/learn MATLAB for several classes. Went to grad school for HPC/CFD and there I was given access to our group's simulation code and was let loose to implement whatever routines I needed to simulate my problems. The shared components were th…

Have you thought about contributing to some open source software? It depends on the project, but often after you submit your code, you will get a code review. And you probably will also learn something about testing.

Re: Simple rules for documenting scientific software

#52
post #38

Earlier quoted context omitted.

I don't buy it. I've worked with very complicated systems (ads at google) and it was pretty straightforward- it's the distributed systems communicating at RPC interfaces that is hard to understand. I've also worked with quantum chemistry codes and large supercomputer codes. The ones that were difficult to maintain were just badly written. There was an article recently about Jeff Dean and Sanjay Ghemawat- one of Sanja…

How would you make, say, a fourier transform or IIR filter design accessible to people who don't know any signal processing or complex numbers? You can spend years studying these topics and their related math. Entire text books have been written on these subjects. Any one algorithm probably has multiple scientific publications, explaining how and why they work. This much information can not fit into a few code commen…

You don't need to understand FFTs to understand how to update good software. If there's a segfault generated by a program and I can run it though a debugger I should be able to fix the segfault without explicit domain knowledge of the code. My modifications should be testable via unit tests. There should also be a formalized process to incorporate my changes into the authoritative codebase. That process should contain a code reviewing process where one or more experts on the codebase critique my code.

These are all software engineering things that have nothing to do with domain knowledge.

Re: Simple rules for documenting scientific software

#53
post #38

Earlier quoted context omitted.

I don't buy it. I've worked with very complicated systems (ads at google) and it was pretty straightforward- it's the distributed systems communicating at RPC interfaces that is hard to understand. I've also worked with quantum chemistry codes and large supercomputer codes. The ones that were difficult to maintain were just badly written. There was an article recently about Jeff Dean and Sanjay Ghemawat- one of Sanja…

How would you make, say, a fourier transform or IIR filter design accessible to people who don't know any signal processing or complex numbers? You can spend years studying these topics and their related math. Entire text books have been written on these subjects. Any one algorithm probably has multiple scientific publications, explaining how and why they work. This much information can not fit into a few code commen…

Nothing about spectral transforms requires complicated software, beyond the need to optimize the algorithms. Sure, the inner components need specialists. No complaint there. But it's not like your stuff is "complicated". I guess the counter argument is something like Goto BLAS, where the author has an extremely complex internal mental model of Intel instruction scheduling and can emit incomprehensible, but absurdly fast instruction sequences. But of course, the BLAS API is dead simple and almost nobody using it needs to know the underlying solver complexity in detail.

Re: Simple rules for documenting scientific software

#54
post #38

Earlier quoted context omitted.

I write scientific code for a living and my code is very difficult for anyone else to maintain. It is not because my code is badly documented or written, it is because what it does is very complex. Every module is documented why it exists and what it does and the code straightforward to read, yet the interaction of all the modules is very complex as it reflects the underlying complexity of the problem the code is sol…

I don't buy it. I've worked with very complicated systems (ads at google) and it was pretty straightforward- it's the distributed systems communicating at RPC interfaces that is hard to understand. I've also worked with quantum chemistry codes and large supercomputer codes. The ones that were difficult to maintain were just badly written. There was an article recently about Jeff Dean and Sanjay Ghemawat- one of Sanja…

Well all I can say is I have worked with some pretty smart developers (much better than me) who have run slam bang straight into the domain knowledge problem and have got totally stuck. They have a habit of removing critical code that they think is not required or spending a huge amount of time solving trivia.

It is not that the code is really complex, but the underlying problem is complex. Some things are just hard problems and no matter how skilful a developer you are, if a problem requires 10 years of full time study in a particular area you will struggle if you don't have this background.

Re: Simple rules for documenting scientific software

#55
post #38

Earlier quoted context omitted.

I don't buy it. I've worked with very complicated systems (ads at google) and it was pretty straightforward- it's the distributed systems communicating at RPC interfaces that is hard to understand. I've also worked with quantum chemistry codes and large supercomputer codes. The ones that were difficult to maintain were just badly written. There was an article recently about Jeff Dean and Sanjay Ghemawat- one of Sanja…

Well all I can say is I have worked with some pretty smart developers (much better than me) who have run slam bang straight into the domain knowledge problem and have got totally stuck. They have a habit of removing critical code that they think is not required or spending a huge amount of time solving trivia. It is not that the code is really complex, but the underlying problem is complex. Some things are just hard…

Why would a developer remove critical code. You have tests for them... right? Your design doc, code comments, and function call paths in the code all point to why the code is needed. And if you can remove it without breaking anything, then it wasn't needed in the first place.

I think you have bigger problems with your devs, that don't have to do with your "problem domain complexity".

Post reply on HN