Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why scientific programming does not compute

nature.com

Post reply on HN
Print view View original

31–40 of 170 posts

Re: Why scientific programming does not compute

#31

Post by ldar15 »

archgoon wrote: post #16
MasterScrat wrote: post #15

Earlier quoted context omitted.

Similarly, from Zed Shaw ( http://learnpythonthehardway.org/book/advice.html ) : "People who can code in the world of technology companies are a dime a dozen and get no respect. People who can code in biology, medicine, government, sociology, physics, history, and mathematics are respected and can do amazing things to advance those disciplines."

This seems at odds with the statement from the article "There needs to be a real shift in mindset away from worrying about how to get published in Nature and towards thinking about how to reward work that will be useful to the wider community." EDIT: What the heck is wrong with this? We have two opinions on the perceived value of programmers in scientific enterprises, one from someone who works in the field (David Ga…

Dont feel too bad. HN has become full of people who believe that the correct response to something they disagree with is to mod down, instead of to reply with a rebuttal. It sucks, but others will mod you up.

Re: Why scientific programming does not compute

#32

Post by gammarator »

earl wrote: post #29
saulrh wrote: post #2

Something I heard from one of my professors once: "A programmer alone has a good chance of getting a good job. A scientist alone has a good chance of getting a good job. A scientist that can program, or a programmer that can do science, is the most valuable person in the building."

That has not been my experience. I spent years working on DNA aligners then in a wetlab building software for confocal laser microscopy. In both locations, the best paid and most highly valued people were the scientists. If you, say, were a good developer with masters in stats and a strong understanding (somewhere between an undergrad and an MS) of the relevant science... you were paid 1/2 as much as you would be pai…

Agreed that low salaries are a disincentive for good engineers to work in science. It's worth noting that even the scientists are paid far less than people with commensurate training and experience in industry.

As for why scientists are more highly valued: they bring in the grants that keep the wheels turning (in academic circles; industry & national labs obviously differ).

Re: Why scientific programming does not compute

#33

Post by jzila »

My girlfriend is a PhD student in a pharmacology lab. I'm a software engineer working for an industry leader.

Once, she and the lab tech were having issues with their analysis program for a set of data. It was producing errors randomly for certain inputs, and the data "looked wrong" when it didn't throw an error. I came with her to the lab on a Saturday and looked through the spaghetti code for about 20 minutes. Once I understood what they were trying to do, I noticed that they had forgotten to transpose a matrix at one spot. A simple call to a transposition function fixed everything.

If this had been an issue that wasn't throwing errors, I don't know whether they would have even found the bug. I've been trying to teach my gf a basic understanding of software development from the ground up, and she's getting a lot better. But this does appear to be a systemic problem within the scientific community. As the article notes, more and more complicated programs are needed to perform more detailed analysis than ever before. This problem isn't going to go away, so it's important that scientists realize the shortcoming and take steps to curb it.

Re: Why scientific programming does not compute

#34

Post by a_dy »

saulrh wrote: post #2

Something I heard from one of my professors once: "A programmer alone has a good chance of getting a good job. A scientist alone has a good chance of getting a good job. A scientist that can program, or a programmer that can do science, is the most valuable person in the building."

I can attest to this.

Re: Why scientific programming does not compute

#35

Post by jleyank »

This is a difficult situation. Is it easier to train the domain experts to be competent programmers or train the competent programmers to be domain experts? In a research environment, I worry there's little time or interest in developing specs that can change in an instant or can't be written until the physics is understood.

We find it quite difficult trying to get programming out of people who don't know why Carbon has 4 bonds while Nitrogen has 3, for example.

Re: Why scientific programming does not compute

#36

Post by gammarator »

sliverstorm wrote: post #3

Why not just hire comp scientists or programmers permanently? Adjust the company model, permanently segregate the work?

This happens already with mechanical and electrical engineers in my field (space physics). There are challenges, though: the engineers are paid less than they would earn in industry, and long-term employment is contingent on scientists winning new grants which call for their skills. So, the low value for expertise (in academia) is largely driven by the grant funding system.

I could see a role for staff computer scientists in areas of research where computation plays a particularly large role, but for typical grad-student data munging, the cost/benefit ratio is likely far too high.

Re: Why scientific programming does not compute

#37

Post by mclin »

prospero wrote: post #18
sliverstorm wrote: post #3

Why not just hire comp scientists or programmers permanently? Adjust the company model, permanently segregate the work?

I did some work on data visualization for the astrophysics department when I was in college. I started to work with the simulation code, but found that the math was sprinkled everywhere, which made it really difficult for me to make structural changes without risking the integrity of the program. One of the most elusive skills for self-taught programmers is how to structure code properly. A good architecture would al…

This is why I think, rather than every data analysis/visualization program being written from scratch with it's own custom UI and I/O, formatting etc, these programs should be written as modules to a data workflow program like RapidMiner that handles all that for you.

Re: Why scientific programming does not compute

#38

Post by gammarator »

jleyank wrote: post #35

This is a difficult situation. Is it easier to train the domain experts to be competent programmers or train the competent programmers to be domain experts? In a research environment, I worry there's little time or interest in developing specs that can change in an instant or can't be written until the physics is understood. We find it quite difficult trying to get programming out of people who don't know why Carbon…

My feeling is that a one-semester required course for students in "software carpentry" [1] (as developed by Greg Wilson and discussed in the article) would cure many of the most serious ills in scientific software development. Students can't know they should be using version control, debuggers, and testing if they don't even know such things exist.

[1] http://software-carpentry.org/

Re: Why scientific programming does not compute

#39

Post by saulrh »

earl wrote: post #29
saulrh wrote: post #2

Something I heard from one of my professors once: "A programmer alone has a good chance of getting a good job. A scientist alone has a good chance of getting a good job. A scientist that can program, or a programmer that can do science, is the most valuable person in the building."

That has not been my experience. I spent years working on DNA aligners then in a wetlab building software for confocal laser microscopy. In both locations, the best paid and most highly valued people were the scientists. If you, say, were a good developer with masters in stats and a strong understanding (somewhere between an undergrad and an MS) of the relevant science... you were paid 1/2 as much as you would be pai…

That's possible. I have a friend who works in the university aerospace lab that can read and write C and Matlab. His major contribution is being able to both understand the engineering and write code to manipulate it, and it's basically guaranteed himself a position there. He may not be valued quite as much as the PI or one of the other professors, but he's pretty solid.

I'd also cite myself, but I don't count since I'm in a robotics lab.

Post reply on HN