Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why scientific programming does not compute

nature.com

Post reply on HN
Print view View original

71–80 of 170 posts

Re: Why scientific programming does not compute

#71

Post by dasil003 »

For all the talk of "best practices" and "training" the depressing truth is that guaranteeing correct software is incredibly difficult and expensive. Professional software engineering practices aren't nearly sufficient to guarantee correctness with heavy math. The closest thing we have is NASA where the entire development process is designed and constantly refined in response to individual issues to create the checks and balances with the lofty goal of approaching bug impossibility at an organizational level. Unfortunately this type of evolutionary process is only viable for multi-year projects with 9-figure budgets. It's not going to work for the vast majority of research scientists with limited organizational support.

On the positive side, such difficulty is also in the nature of science itself. Scientists already understand that rigorous peer review is the only way to come to reliable scientific conclusions over time. The only thing they need help with understanding is that the software used to come to these conclusions is as suspect as—if not more so than—the scientific data collection and reasoning itself, and therefore all software must be peer-reviewed as well. This needs to be ingrained culturally into the scientific establishment. In doing so, the scientists can begin to attack the problem from the correct perspective, rather than industry software experts coming in and feeding them a bunch of cargo cult "unit tests" and "best practices" that are no substitute for the deep reasoning in the specific domain in question.

Re: Why scientific programming does not compute

#72

Post by john_b »

gallamine wrote: post #44

I'm a PhD student in Electrical Engineering. I'm currently working on a Monte Carlo-type simulation for looking at the underwater light field for underwater optical communication (no sharks!). I'm doing the development in MATLAB and I recently put all my code up on Github ( https://github.com/gallamine/Photonator ) to help avoid some of these problems (lack of transparency). Even if nobody ever looks/uses the code, I…

I'm a M.S. student in mechanical engineering facing a similar situation, except I haven't put any code on Github (my advisor wants to keep it proprietary, but I probably would not bother putting it up even if he were ok with it).

I've written around 15000 lines of MATLAB for my research and only a handful of people will ever need to see it. Some is well-structured and nicely commented, but other parts are incomprehensible and were written under severe time constraints. My advisor is not much of a programmer and will not be able to figure it out, and I feel bad for leaving a pile of crappy code to the person who inevitably follows in my footsteps, but I ultimately have a choice between writing fully commented, well-tested, and well-structured code and graduating a semester late (at the cost of several thousand dollars to myself), or writing code that's "just good enough" to get results on time. This is a solo project (there is no money for a CS student to intern) and I'm not getting paid to write code unlike a professional programmer, so every second I spend improving my code beyond the bare minimum costs me time and money.

Even if I were able to tidy up and publish all of my code, most mechanical engineers would not be able to understand it because most can't write code. Those who can mostly use FORTRAN, although C is becoming more common. Nonetheless, even those who could understand my code would have little incentive to read through 15000+ lines of code.

Unfortunately, as far as research code is concerned, a lot of trust is still required on the part of the reader of the publication. I agree that the transfer of knowledge should be handled differently, but until there is a strong incentive for researchers to write good code it will continue to be bad. Especially when many research projects only require the code to demonstrate something, after which it can be put in the closet.

Re: Why scientific programming does not compute

#73

Post by xxd »

tedjdziuba wrote: post #53

I know of a company, made up of scientists from academia, that develops software by writing the code (or "codes" as they call it) in Microsoft Word documents and e-mailing them to eachother. Some how, they are still in business. True story.

This wouldn't be a healthcare startup in SF, would it?

Re: Why scientific programming does not compute

#74

Post by reinhardt »

jzila wrote: post #33

My girlfriend is a PhD student in a pharmacology lab. I'm a software engineer working for an industry leader. Once, she and the lab tech were having issues with their analysis program for a set of data. It was producing errors randomly for certain inputs, and the data "looked wrong" when it didn't throw an error. I came with her to the lab on a Saturday and looked through the spaghetti code for about 20 minutes. Once…

Not sure how your anecdote relates to the conclusion. Forgetting, or even knowing why, to transpose a matrix is not an example of a problem that can be solved by "a basic understanding of software development". Hell, I'm sure there are many decent hackers that don't know what a matrix is, let alone spot such errors within a long sequence of computations.

Re: Why scientific programming does not compute

#75

Post by reinhardt »

xxd wrote: post #73
tedjdziuba wrote: post #53

I know of a company, made up of scientists from academia, that develops software by writing the code (or "codes" as they call it) in Microsoft Word documents and e-mailing them to eachother. Some how, they are still in business. True story.

This wouldn't be a healthcare startup in SF, would it?

I shudder to think that such a "company" could be developing health-related software.

Re: Why scientific programming does not compute

#76

Post by pygy_ »

rflrob wrote: post #50
pygy_ wrote: post #11

Earlier quoted context omitted.

You won't see any clean code written by scientists until (major) journals make it mandatory to submit code for peer review and publication. When it happens, I hope that they'll manage to agree on a sensible license (even though I won't set my hopes too high).

I have all of my code on github under a CRAPL license [1]. It assumes a certain amount of good-faith from others, but I feel that if you're worrying about getting scooped, your problem isn't ambitious enough. Luckily, my adviser agrees, and is very in favor of open releases of data [2]. [1] http://matt.might.net/articles/crapl/ [2] http://www.michaeleisen.org/blog/?p=440

The terms of the license are good, but its name is literally crappy:-/

Re: Why scientific programming does not compute

#77

Post by john_b »

sliverstorm wrote: post #3

Why not just hire comp scientists or programmers permanently? Adjust the company model, permanently segregate the work?

Even when money allows for this, it's not always possible to segregate the work. I.e. a scientist may fully understand the mathematics of his work, but not understand that translating the math directly into code will cause serious performance issues. A programmer should understand the performance issues, but without a deep and intuitive understanding of the mathematics, including what assumptions are implicit therein, will often have difficulty making the translation.

In many cases, projects with sufficient technical depth require detailed domain knowledge that can be acquired quicker and cheaper by hiring scientists or engineers with that knowledge and then teaching them to code (or code better), rather than segregating the work and dealing with the problems that result from the communication gap.

Re: Why scientific programming does not compute

#78

Post by demian »

dasil003 wrote: post #71

For all the talk of "best practices" and "training" the depressing truth is that guaranteeing correct software is incredibly difficult and expensive. Professional software engineering practices aren't nearly sufficient to guarantee correctness with heavy math. The closest thing we have is NASA where the entire development process is designed and constantly refined in response to individual issues to create the checks…

I agree that correct software can not be achive by industry's practices.

BUT isn't better to use "cargo cult best practices", as you call them, than code-and-fix without any kind of formal test or documentation?

The hole point of these software programming practices is to improve overall quality with limited resources, not to craft perfect code.

Re: Why scientific programming does not compute

#79

Post by jonathansizz »

crocowhile wrote: post #63
earl wrote: post #30

Earlier quoted context omitted.

My experience is while they may be respected, they aren't paid . And I'm not a charity.

Nobody is paid for doing research. The salary of a professor is about 1/3-1/5 of what the same person may get in industry. People who do research don't do it for the money.

I agree with your point, but I'd add the modifier that if you're a top-notch developer who forms an interest in a specific sub-discipline in science (my field is genomics, but there are many others where this would be equally applicable), there is certainly huge potential to carve out a niche and make a big name for yourself in your chosen field (as long as you take the science side as seriously as the coding side). Then you certainly could make big money, as Shaw implies and as you'd surmise from reading the linked article.

Re: Why scientific programming does not compute

#80

Post by droz »

mclin wrote: post #41

Rather than building these data analysis/visualization programs from scratch each time, my thought is that scientists should instead be writing them as modules for a data workflow application like RapidMiner. If you haven't heard of RapidMiner, you basically edit a flowchart where each step takes inputs and outputs, eg take some data and make a histogram, or perform a clustering analysis. Video of someone demoing it:…

Do you know of any other good data analysis applications similar to RapidMiner?
Post reply on HN