Why scientific programming does not compute • Hacker News Board

Re: Why scientific programming does not compute

#51

Post by GoogleMeElmo » Wed, Jul 06, 2011, 9:39 PM UTC

Yes this is a huge problem. I am a software engineer working at a research institute for bioinformatic. The biggest problem I encounter in my struggle for clean maintainable code, is that management down prioritize this task quite heavily.

The researchers produce code of questionable quality that needs to go into the main branch asap. Those few of the researchers that know how to code (we do a lot of image analysis), don't know anything about keeping it maintainable. There is almost a hostile stance against doing things right, when it comes to best practices.

The "Works on my computer" seal of approval have taken a whole new meaning for me. Things go from prototype to production by a single correct run on a single data set. Sometimes its so bad I don't know if I should laugh or cry.

Since we don't have a single test or, ever take the time to do a proper build system, my job description becomes mostly droning through repetitive tasks and bug hunting. It sucks the life right out of any self respecting developer.

There, I needed that. Feel free to flame my little rant down into the abyss. :)

Re: Why scientific programming does not compute

#52

Post by gallamine » Wed, Jul 06, 2011, 9:40 PM UTC

scott_s wrote: post #12

bh42222 wrote: post #5

As a general rule, researchers do not test or document their programs rigorously, and they rarely release their codes, making it almost impossible to reproduce and verify published results generated by scientific software, say computer scientists. Just stop doing that! Seriously, testing is not wasted effort and for any project that's large enough it's not slowing you down. For a very small and simple project testing…

The point is that since software development is not their main goal or background, their practices tend to be ad-hoc. We know the value of testing and documentation, but they do not. People don't know to stop doing something until they know it's a bad practice. And they're not going to know it's a bad practice until they discover that fact on their own (which can be a slow process) or someone teaches them (faster, bu…

The mindset of a scientist is that the code is a one-time thing to achieve a separate goal - data for a paper. The code isn't supposed to last, it's simply a stepping stone. For a lot of folks, whose research areas tend to move around, there isn't always the expectation that you'll get to a 2nd or 3rd paper on the same data.

Now, all of this is different if you research actually is building the model. But, I'm speaking for experience on the rest. I've built plenty of software tools that I need "right now" to get a set of data.

Re: Why scientific programming does not compute

#53

Post by tedjdziuba » Wed, Jul 06, 2011, 9:42 PM UTC

I know of a company, made up of scientists from academia, that develops software by writing the code (or "codes" as they call it) in Microsoft Word documents and e-mailing them to eachother.

Some how, they are still in business.

True story.

Re: Why scientific programming does not compute

#54

Post by losvedir » Wed, Jul 06, 2011, 9:43 PM UTC

gte910h wrote: post #6

I want to write a "software style guide" for journalists and their editors. Software and Code are both mass nouns in technical language. "Code" can be in programs (aka, things that run), libraries (things that other programmers can use to make programs), or in samples to show people how to do things in their programs or libraries. Some people call short programs scripts. When you feel you should pluralize "software",…

Is this not simply a British English thing? I assumed it was, like "maths", since Nature is a British publication. HN users from the UK, can you confirm? gte is speaking about constructions in the article like:

"As a general rule, researchers do not test or document their programs rigorously, and they rarely release their codes, making it almost impossible to reproduce and verify published results generated by scientific software, say computer scientists."

"As recognition of these issues has grown, software experts and scientists have started exploring ways to improve the codes used in science."

Re: Why scientific programming does not compute

#55

Post by pwang » Wed, Jul 06, 2011, 9:45 PM UTC

gallamine wrote: post #44

I'm a PhD student in Electrical Engineering. I'm currently working on a Monte Carlo-type simulation for looking at the underwater light field for underwater optical communication (no sharks!). I'm doing the development in MATLAB and I recently put all my code up on Github ( https://github.com/gallamine/Photonator ) to help avoid some of these problems (lack of transparency). Even if nobody ever looks/uses the code, I…

There is a market, and it's called libraries. Eventually you will use a language where software carpentry and code reuse is a core feature, and tested, modular libraries for not only core algorithms, but also deployment and dev-ops stuff (like managing a compute cluster on the cloud) will have standard approaches.

This is starting to shape up on the Python side of things, but it has stagnated a little bit. People who can and do write the foundational code are oftentimes too focused on making the code work, and not at all focused on improving the quality of the ecosystem that their code is part of. Open Source is a great mechanism for many things, but polishing up the last 20% is not one of them.

Re: Why scientific programming does not compute

#56

Post by madaxe » Wed, Jul 06, 2011, 9:46 PM UTC

losvedir wrote: post #54

gte910h wrote: post #6

I want to write a "software style guide" for journalists and their editors. Software and Code are both mass nouns in technical language. "Code" can be in programs (aka, things that run), libraries (things that other programmers can use to make programs), or in samples to show people how to do things in their programs or libraries. Some people call short programs scripts. When you feel you should pluralize "software",…

Is this not simply a British English thing? I assumed it was, like "maths", since Nature is a British publication. HN users from the UK, can you confirm? gte is speaking about constructions in the article like: "As a general rule, researchers do not test or document their programs rigorously, and they rarely release their codes , making it almost impossible to reproduce and verify published results generated by scien…

Definitely not. Queen's English here, and "Maths" is a simple concatenation of Mathematics, meaning "Codes" would make no lexical sense. Therefore, we say "Code", just like you.

Re: Why scientific programming does not compute

#57

Post by pwang » Wed, Jul 06, 2011, 9:46 PM UTC

ANH wrote: post #45

From personal experience, I attest that it can be more difficult than pulling teeth to get a scientist to commit code to a version control system.

Greg Wilson once commented that the subversive way to get scientists to use source control was not to pitch it as a code history tool, but rather as a nifty way to sync up code between their work machines, home machines, etc. He said he had a lot more traction with that than trying to lecture them about having code history.

Re: Why scientific programming does not compute

#58

Post by another » Wed, Jul 06, 2011, 9:47 PM UTC

neutronicus wrote: post #14

bh42222 wrote: post #5

As a general rule, researchers do not test or document their programs rigorously, and they rarely release their codes, making it almost impossible to reproduce and verify published results generated by scientific software, say computer scientists. Just stop doing that! Seriously, testing is not wasted effort and for any project that's large enough it's not slowing you down. For a very small and simple project testing…

The thing about scientific code is that it's often a potential dead end. The maintenance phase of the software life cycle is not as assured as it is in industry. Writing good engineering software is not the scientist's goal so much as demonstrating that someone else with a greater tolerance for tedium (also someone better-paid) could write good engineering software.

Exactly. I'd go further: in industry, the software is typically the end product, and the quality of the software is inherently relevant. In science, the output (the prediction of the simulation, the result of the analysis, etc.) is typically the end product, and the quality of the software is relevant only insofar as it affects the quality of the output.

In practice, of course, the quality of the software often does affect the quality of the output---but time spent on software quality creates less immediate value than it does in industry.

Re: Why scientific programming does not compute

#59

Post by [deleted] » Wed, Jul 06, 2011, 9:47 PM UTC

losvedir wrote: post #54

gte910h wrote: post #6

I want to write a "software style guide" for journalists and their editors. Software and Code are both mass nouns in technical language. "Code" can be in programs (aka, things that run), libraries (things that other programmers can use to make programs), or in samples to show people how to do things in their programs or libraries. Some people call short programs scripts. When you feel you should pluralize "software",…

Is this not simply a British English thing? I assumed it was, like "maths", since Nature is a British publication. HN users from the UK, can you confirm? gte is speaking about constructions in the article like: "As a general rule, researchers do not test or document their programs rigorously, and they rarely release their codes , making it almost impossible to reproduce and verify published results generated by scien…

[deleted]

Re: Why scientific programming does not compute

#60

Post by ak217 » Wed, Jul 06, 2011, 9:47 PM UTC

earl wrote: post #29

saulrh wrote: post #2

Something I heard from one of my professors once: "A programmer alone has a good chance of getting a good job. A scientist alone has a good chance of getting a good job. A scientist that can program, or a programmer that can do science, is the most valuable person in the building."

That has not been my experience. I spent years working on DNA aligners then in a wetlab building software for confocal laser microscopy. In both locations, the best paid and most highly valued people were the scientists. If you, say, were a good developer with masters in stats and a strong understanding (somewhere between an undergrad and an MS) of the relevant science... you were paid 1/2 as much as you would be pai…

Which aligner(s) did you work on?

Sequence analysis companies and labs which don't value software engineering get what they pay for: serious or crippling inefficiencies and inability to do analysis on their data or maintain continuity. Unfortunately, many of them don't even realize what they need or how bad their inefficiencies are.