Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Science needs more research software engineers

nature.com

Post reply on HN
Print view View original

151–160 of 366 posts

Re: Science needs more research software engineers

#151

Post by barry-cotter »

arvinsim wrote: post #148
jltsiren wrote: post #72

Earlier quoted context omitted.

While the academia has a lot of issues, software engineers should also adjust their expectations. First, if you choose to work for a non-profit, you should expect a huge pay cut. That's the way the society works. Or actually it's the opposite. If you work in a for-profit role, the society allows your employer to keep most of the value you create. If you create a lot of value, that gives you the leverage to negotiate…

My apologies for not being familiar with acronyms, but what is a PI?

Principal Investigator, the feudal lord who has near total power over the post docs and total power over the grad students in their lab. They can ruin your career at will and will almost certainly suffer no consequences whatsoever from doing so.

Re: Science needs more research software engineers

#152

Post by rob_c »

throwmamatrain wrote: post #60

This looks great, and I would have loved to see this when I was in the lab. When the software industry says to you: "We will nearly triple your salary, you don't have to work weekends, and you also don't have to feed the mice on a Sunday night." You will 100% take this deal. I was a 10yr+ academic tool maker in biochemistry, built cutting edge microscopes, hardware, and image analysis software. My lab was successful…

> at least in bioscience...

BINGO! that field is notoriously horrible and interacts extremely badly with a 'when not discovered here, not important' syndrome. Biology is brutal toward bio-physicists, mathematicians as well as people who code who they are forced to work with rather than seek out to help them.

I still hold up as an example nonsense discussions around p-values in bio vs actual work going on in statistics in maths departments. It shows how far detached they've become.

Not to criticize too strongly, but given the above, combined with it's reproducabilty crisis, and existential problem of being in the back-pocket of big-pharma, I seriously doubt the professional integrity of a lot of people in the field.

Move toward mathematics, physics and chemistry. There is (some) serious money and a good atmosphere around areas such as finite-element modelling, or wolphram like tools as an example. There is a lack of direct funding for decent posts but you get recognised and paid the equivalent as a peer, I know from working with some of these people. It's not to say it's 100% always without friction, but no job is I'd argue.

Re: Science needs more research software engineers

#153

Post by cycomanic »

bmitc wrote: post #43
cozzyd wrote: post #38

Earlier quoted context omitted.

On the flip side of that, many experiments run for decades or longer, and "state of the art" software is often a poor impedance match for that.

I'm not convinced of that. Because what I've seen (several times) on a somewhat small scale perhaps, is someone's code, sometimes written decades ago, never maintained upgraded or properly developed used for years on a system and eventually it gives out, forcing basically a rewrite. And state of the art can mean different things for differently scaled projects.

You have too look at the big picture though. How much time would have been needed to train the PhD student (might have been even the PI) to write more maintainable code 10 years ago, and maintain that code to "best practices" over the time, compared to write that it works once and need to rewrite once after 10 years?

Moreover, people tend to forget that scientific software evolves much more slowly than commercial software and the number of people who work on this are much fewer, so why always adopt the newest method, if the code is still working. Show me commercial projects that have had the stability of something like LAPACK or BLAS, those essentially had a stable API for > 50 years now, with very few bugs.

Re: Science needs more research software engineers

#154

Post by gvurrdon »

exdsq wrote: post #133
konschubert wrote: post #131

Earlier quoted context omitted.

Are you talking about the letters P, H and D? As in, if you don't have a PhD they don't see you as a peer?

I've honestly yet to meet a research software engineer without a PhD because of the academic bias you will get in, well, academia.

I work with two of them at the moment. One is planning to apply for a PhD studentship soon, but the other does not intend to do so.

Re: Science needs more research software engineers

#157

Post by busterarm »

altacc wrote: post #103
throwmamatrain wrote: post #60

This looks great, and I would have loved to see this when I was in the lab. When the software industry says to you: "We will nearly triple your salary, you don't have to work weekends, and you also don't have to feed the mice on a Sunday night." You will 100% take this deal. I was a 10yr+ academic tool maker in biochemistry, built cutting edge microscopes, hardware, and image analysis software. My lab was successful…

As a counterpoint, I know someone who went back to academia after getting disillusioned with tech (academia > tech > academia). The key difference may be that they live in Europe, have no student loans and the pay disparity between a developer in bioscience and in tech is not as large as I imagine it is in the US. They are paid significantly more than the scientists they work alongside but not much less than they wer…

Salaries in Europe are catching up...

Re: Science needs more research software engineers

#158

Post by cycomanic »

toma_caliente wrote: post #9

It's already been said several times over but I'm going to throw it out anyway. I'd love to work at a university as a research software engineer. I find things like bioinformatics fascinating and even interviewed at a research university on the east coast. But, you will get paid SIGNIFICANTLY less than you would doing almost anything else, have zero autonomy, and get no respect. Those 3 things are basically what most…

> But, you will get paid SIGNIFICANTLY less than you would doing almost anything else

Yes definitely true, a junior software engineer in industry earns about as much as a Professor in academia. How would you justify such a salary at the university?

> have zero autonomy,

I've written that before. I would argue it's the complete opposite, autonomy is the one big advantage of academia. If you are a software engineer you likely have even more autonomy than most others, because your work is not directly linked to conference/grant deadlines etc. typically you would be given very rough guidelines on what you should try to achieve and then left to your own devices for long periods of time. Most PIs would neither have the time or expertise to micromanage a software engineer.

> and get no respect.

Again not my experience at all. I know that in my area people would respect a software engineer. I know when my partner was working as a research assistant in a medical lab, I wrote a small script to process some huge spreadsheet that they always processed by hand (work that took sometimes several weeks). They treated me like a god and even wanted to include me on the paper. The thing to remember though is you will not be considered a researcher, so likely not get grant funding and your job progression would always be in more technical roles, not in the academic track. I still don't see any difference in respect.

Re: Science needs more research software engineers

#159

Post by elementalest »

joshvm wrote: post #101
Azrael3000 wrote: post #54

Earlier quoted context omitted.

There is simply no incentive for science to do proper software engineering. It does not directly produce papers or money, so nobody does it. I've seen it several times in my own career in several countries. PhDs having their own version of the code, incompatible with the one of the Postdoc sitting opposite of them. Each slaving away at their niche project but nobody there to bring it all together. This particular uni…

RSEs should not be working on, or assisting, research at the PhD level. By that I mean news/skunkworks stuff. They should be taking the output from a PhD and turning it into core software for the group. That doesn't solve the collaboration issue between PhDs/Postdocs, but there is a particular point in a research project lifecycle where it makes sense to hire an RSE. A bigger challenge is that most PIs are not projec…

RSE's can and often absolutely should be involved at the PhD level. In my experience, collaboration between the scientist and engineer in the process of research iterations almost always produce better results. Each has insights the other may not, likely leading to better outcomes for the research, final product/tool and time taken.

The scientist just wants to focus on their research and once they have a barely working proof of concept, hand it over to the engineer to figure the rest out. The engineer wants a well specified design and prototype that they can lightly refactor to clean up, scale up and turn into a product/tool.

The reality is that approach makes it way harder for both, though most often harder for the engineer as they are generally at the end of the chain in Academia and have little power. For example, the code or spec from the scientist is often terrible, so the engineer needs to start from scratch and keep going back to the scientist to spec out the design as they were not involved at any stage prior. They may even find edge cases or flaws the scientist had not considered that are fundamentally problematic to turning it into a viable product/tool.

This is why the big corporate/industry research labs often have high level RSE that are involved in the research process and get their names in papers (they sometimes have PhD's themselves). They are not optimising for the scientists time, but for the companies resources

Re: Science needs more research software engineers

#160

Post by ohlookabird »

exdsq wrote: post #133
konschubert wrote: post #131

Earlier quoted context omitted.

Are you talking about the letters P, H and D? As in, if you don't have a PhD they don't see you as a peer?

I've honestly yet to meet a research software engineer without a PhD because of the academic bias you will get in, well, academia.

For many years I worked in a high profile research institute (neuroscience) as an RSE without a PhD. Still don't have one, and that's okay (for the path I'm on). Quite a few of the other RSEs in the institute don't have one either. In total I'd say maybe 50% didn't have a PhD.
Post reply on HN