Live data from Hacker News

Why scientific programming does not compute

nature.com

101–110 of 170 posts

Re: Why scientific programming does not compute

#101
post #56

Earlier quoted context omitted.

Is this not simply a British English thing? I assumed it was, like "maths", since Nature is a British publication. HN users from the UK, can you confirm? gte is speaking about constructions in the article like: "As a general rule, researchers do not test or document their programs rigorously, and they rarely release their codes , making it almost impossible to reproduce and verify published results generated by scien…

Definitely not. Queen's English here, and "Maths" is a simple concatenation of Mathematics, meaning "Codes" would make no lexical sense. Therefore, we say "Code", just like you.

>concatenation

Contraction.

Re: Why scientific programming does not compute

#102
post #7
post #6

I want to write a "software style guide" for journalists and their editors. Software and Code are both mass nouns in technical language. "Code" can be in programs (aka, things that run), libraries (things that other programmers can use to make programs), or in samples to show people how to do things in their programs or libraries. Some people call short programs scripts. When you feel you should pluralize "software",…

Actually it's a science thing. In a scientific context "code" is understood to mean "program". For example: http://scholar.google.com/scholar?q=%22population+synthesis+... I'm not sure why this is though.

Scientists say "code" because "machine code" is a bit of a mouthful. They say "machine" because their computers used to be people, and an interrupt meant running excitedly into Sir George Everest's tent. If entrepreneurs had invented the machines, we'd call them "electronic clerks".

When the manual is titled "Theoria combinationis observationum erroribus minimis obnoxiae", you know you're dealing with legacy code. In that case it's a pretty cool legacy, though.

Re: Why scientific programming does not compute

#103
post #80
post #41

Rather than building these data analysis/visualization programs from scratch each time, my thought is that scientists should instead be writing them as modules for a data workflow application like RapidMiner. If you haven't heard of RapidMiner, you basically edit a flowchart where each step takes inputs and outputs, eg take some data and make a histogram, or perform a clustering analysis. Video of someone demoing it:…

Do you know of any other good data analysis applications similar to RapidMiner?

I don't know any first hand, but other's I've heard of: Taverna: http://taverna.org.uk/ Trident: http://www.microsoft.com/mscorp/tc/trident.mspx

Re: Why scientific programming does not compute

#105
post #92

Earlier quoted context omitted.

There is an important factor discouraging publishing source code - fear that there indeed are bugs and they will be exposed. This is blatantly "security through obscurity", but I fear it's a common attitude. If there are bugs and code is secret, even if someone else later points out that the results contradict their own findings, it's (presumably) not difficult to sweep the thing under the rug and cool it down. On th…

Won't you get a citation, whenever somebody uses your code?

Typically a review paper describing the software is what is actually cited, but yes!

For instance, in my department there is a guy who maintains an astrophysical software package called Cloudy. The faq[1] describes how to cite it. (Unlike a lot of the software mentioned here, that project actually is open source, uses version control, and was migrated from the original Fortran to C++.)

[1]http://www.nublado.org/wiki/FaqPage

Re: Why scientific programming does not compute

#106

Earlier quoted context omitted.

I agree it's a problem, but I think you have to fix the incentives to make meaningful change. When people are thrown into a cut-throat competitive environment, with tenure clocks, multiple junior professors per tenure slot, requirement to bring in grants to fund your research or you get shut down, etc., it doesn't encourage people to be altruistic and sharing.

I think the problem is fundamentally one of economics. Research is good, but you have to decide how much money to allocate to it. In order to decide, you need a metric for performance. Really, only scientists are qualified to judge whether the results of other scientists are worth anything, so currently the only metric we really have is publishing in peer-reviewed journals. Ultimately, therefore, that's where the inc…

How about trying to fix the current system by making somebody else using your software count as a "super citation"? (It could even arguably count as much as co-authorship.)

Re: Why scientific programming does not compute

#107
post #84
post #72

Earlier quoted context omitted.

I'm a M.S. student in mechanical engineering facing a similar situation, except I haven't put any code on Github (my advisor wants to keep it proprietary, but I probably would not bother putting it up even if he were ok with it). I've written around 15000 lines of MATLAB for my research and only a handful of people will ever need to see it. Some is well-structured and nicely commented, but other parts are incomprehen…

I wonder whether our priorities for research are misguided. Isn't research about extending the knowledge of humanity? Writing and passing on readable code would probably advance us further in total, than everyone starting basically from scratch. (I'm not faulting you, you just react to the incentives.)

Well, yes, it is about extending the knowledge of humanity- but it doesn't happen in a vacuum, and is subject to a lot of the same constraints as any other human activity. And, as you say, there are incentives at work- if the john_b's advisor had written "release usable MATLAB toolkit for $doing_whatever_john_b's_thesis_does" into his grant as a deliverable, you can bet that both john_b and the advisor would have made sure that it was in a releasable state, and also that the advisor would have had funds available to pay john_b to clean it up and get it ready to go- they would have been specifically allocated for that purpose in the grant's budget.

Re: Why scientific programming does not compute

#108
post #95

Earlier quoted context omitted.

"my advisor wants to keep it proprietary" This concerns me. Is this kind of thinking pervasive in public academic institutions? Avoiding the copyright ownership issues that tend to accompany such discussions, would it not be better to be more open about the code in an attempt to gain peer review? I understand your personal motivations about not publishing, but the statement about your advisor is what I'm worried abou…

Yes it is. Often it's not for nefarious reasons - it happens a lot where I work because we use data from collaborators that is unpublished, and it's considered unethical to jump over them by releasing code or results based on it. Of course, the problem is that it can sometimes take years to get large datasets published and this means that the code gathers dust and gets forgotten in the meantime. By contrast, the pape…

The ethical considerations with regard to "jumping" collaborators indeed make sense.

As to the last point, perhaps it's time the scientific community took software into consideration along with the data and it's resulting papers. At the least, acknowledge the problem. At best, decide where (alongside the data? with the paper in progress?) the software should be stored.

Re: Why scientific programming does not compute

#109
post #30

Earlier quoted context omitted.

My experience is while they may be respected, they aren't paid . And I'm not a charity.

Nobody is paid for doing research. The salary of a professor is about 1/3-1/5 of what the same person may get in industry. People who do research don't do it for the money.

Yeah, but you know who gets even more screwed, salary-wise, than the professor? I'll tell you who: their lab techs and staff scientists ("research associates"), and that most definitely includes the "programmer who knows some science".

Furthermore, at least at my institution, being the "programmer who knows some science" means that your position is entirely funded with "soft money", which means that your level of job security can be pretty low.

Re: Why scientific programming does not compute

#110
I think there are multiple reasons for this problem, and only one of them is a lack of training in software management. Another problem is that science is an inherently exploratory procedure. You design an experiment, gather some data, and then go about analyzing it. You have an idea of what you'll find, but depending on what you get, you might need to then reformat/restructure the data, transform it, cut it up, etc.

The problem is that this represents one of the worst problem cases in software design: evolving requirements. By itself this is bad enough. Recently I have been analysing data from a recent study. You start off with a data structure that you think represents things, but then you notice for example you need to synchronize several recordings; now you have to track time. You realize some recordings need to be split down the middle to aid in synchronization; now you need to add a 'part' field. You derive some value from several data points that takes a long time to compute, so you need to create a file to hold it. This needs to be kept in synch with the original data. Eventually you realize that text files aren't going to cut it; you start moving things to a database. Now you need to reconfigure your visualization program to read from the database. Then you realize that you want to add another similar derivative value, but this time it's a 3x3 matrix for each data point; time to extend the database again. etc.. etc.. Eventually you decide it would be best to really rewrite the codebase because it's becoming impossible to work with. Unfortunately the paper is due soon and you just need to generate a few more graphs..

And I didn't even mention the growing directory of scripts that aren't properly organized into modules, that end up with copy-pasted code because it's not very clear how to cleanly put this into a function, or which module it should belong to.

Now, this is bad enough when you have a CS degree and have designed several software frameworks in your life. Combine this with someone who knows nothing about software architecture and you have a really big problem on your hands. My point is this: it happens to the best of us, no matter how hard you try to organizing things, when you don't have the requirements available ahead of time.

The best approach I've found is to force myself to simply write functions as small as possible, that do one simple thing at a time. I try to break up functions as much as possible for reuse, and avoid copy-pasting code at all costs. Admittedly it's not always easy, sometimes a function that generates a particular graph just needs a certain number of lines of logic, and it's very difficult to modularize. Then you find that you want a similar graph but with a slightly different transformation on the Y axis... etc.. etc..

Post reply on HN