Live data from Hacker News

Science needs more research software engineers

nature.com

111–120 of 366 posts

Re: Science needs more research software engineers

#111
post #55

Science needs more "technicians" of all types, and less PhD students. Good luck making that transformation though.

> Less PhD students PhD students are the backbone of any research project. They interact with postdocs and with undergrads in the research ecossystem. If anything, they will be required more in the years to come, not less .

It depends on the amount of technical debt in the project.

When there's no technical debt, sure, grad students are the backbone. They can happily go about their projects and supervise undergrads, while the postdocs supervise them and apply for jobs.

In a place where there's a lot of technical debt, a postdoc is going to be far more productive than a grad student. In some experiments many grad students spend years floundering around before they learn enough to cut through the cruft and do real research. It's pretty silly to fire all the postdocs just when they've managed to understand everything and replace them with new grad students.

The tricky thing is keeping enough of the technical staff around, and coming up with a system where they are rewarded for eliminating debt, rather than penalized for making their own hard-earned skills obsolete.

Re: Science needs more research software engineers

#112

Earlier quoted context omitted.

> They call it "codes" They/We have probably been calling it that since before you were alive. I wouldn’t be surprised if it comes from the 60s or 70s. Every group of people has their own terminology for stuff. There is no right or wrong.

Yeah. Considering how much time we spend developing and using the damn things, we’ll call them the way we like, thank you very much. We don’t really need condescending software types to just come and explain to us how we’re saying it wrong. That is particularly rich, considering the tendency of CS people to cargo cult and borrow physical terms without understanding them.

Yes, please stop naming your software projects after elementary particles :).

Re: Science needs more research software engineers

#113
So, I was an RSE? Know why I left?

Money. It’s all very well being an RSE but there are two ways of being funded (at least here in the UK). One is centrally funded by the University - universally this means being shoved into the IT hierarchy which is not a good fit. The other is on an ongoing basis by doing 1-2 year contracts. So no job security.

The salaries for both of these are very poor. On leaving I took a 35% pay rise and do basically the same work but in industry. The only difference is that instead of working for engineers in academia, I do work for engineers in companies. Since I jumped, I negotiated another 20% pay rise. If I’d stayed, I’d have got a 3% pay rise. That’s not including the fact I get bonuses, private health insurance, don’t have to pay to park at my own office…

The issue is that the skill set demanded to be an RSE is highly desirable in industry - able to write code well, often mathematical, familiarity with much of devops type work.

On top of that, researchers don’t truly value your contributions. They don’t want to put you on papers that couldn’t have happened without your work. On top of that, they’re hugely demanding about timescales that are unrealistic. I got put onto one project which had contractual commitments to deliver to a major governmental agency to run a live service, and the guy running the project handed me a 5000 line Python 2.7 script that wouldn’t even run and basically said “get it working”. He couldn’t even supply the input files needed to run it for 2 months after that, just ignored emails. I quit before it got any further.

Edit: forgot to add, my boss was a grade higher than me on the national academic salaries scale. He had technical responsibility and also had to line manage 10 people. Even if I could have gotten promoted (impossible - no budget), I didn’t want the line management responsibility. I earn more than the top of that salary band now, and line manage nobody.

Re: Science needs more research software engineers

#115
post #24

I don't think scientists want research software engineers. PIs typically like to assume any and every role on a project, including software design. They don't like handing things off and letting people build modern, best practices software. You start talking about source code control, package managers, a language used in industry that they've never even heard of and don't care enough to learn about, etc., and you jus…

> They call it "codes" This is not really the core issue here, but yes, I've encountered this many times, and I have to admit that in some professional contexts I use "codes" as a shibboleth to identify people who aren't really in touch with the software engineering world.

I always assumed it came from big international physics experiments where English is most people's second language. People who learn to program in an expat community are going to pick up some jargon that will sound weird to native English speakers in California.

Re: Science needs more research software engineers

#116
> What do you envision for the profession over the next 10 years?

> I want to see RSEs as equals in the academic environment. Software runs through the entire research process, but professors tend to get most of the recognition and prestige. Pieces of software can have just as much impact as certain research papers, some of them much more so. If RSEs can get the recognition and rewards that they deserve, then the career path will be that much more visible and attractive.

As a former RSE I think this is unrealistic. Academia just isn't structured to value non-professors (it's far from just RSEs that are undervalued.) For me it wasn't so much the money but the refusal to sponsor permanent residence. When you see your amazing colleagues leave due to dumb departmental policies you realize you need to plan your own exit. I miss working in a team with a mix of scientists and engineers but I don't miss working for a university.

Re: Science needs more research software engineers

#117

Earlier quoted context omitted.

> They call it "codes" They/We have probably been calling it that since before you were alive. I wouldn’t be surprised if it comes from the 60s or 70s. Every group of people has their own terminology for stuff. There is no right or wrong.

Yeah. Considering how much time we spend developing and using the damn things, we’ll call them the way we like, thank you very much. We don’t really need condescending software types to just come and explain to us how we’re saying it wrong. That is particularly rich, considering the tendency of CS people to cargo cult and borrow physical terms without understanding them.

Always annoying to hear machine learning people speak of "tensors"...

Re: Science needs more research software engineers

#118
For this to work academic science would have to start paying significantly more, which many labs simply cannot do, and other labs will refuse to do.

My PI flat out said when we were trying to hire a python developer that they simply could not be paid 2x more than anyone else because it would cause major issues with team cohesion. 2x is the bare minimum to even get close to market rate for a full time developer who won't dance out the door a year later.

By this logic either everyone in science needs to get a massive raise (stop to the PhD slave labor?), or rse's really have to justify their salary, and when everyone else works 60-80 hours a week for half the pay and is also defacto on call, that justification looks like zero work life balance.

Talk to a funder near you, let them know that congress needs to provide more money for basic research. Be sure to sell it as training future employees for big tech too since they will all eventually leave after you finish teaching them.

Re: Science needs more research software engineers

#119

This looks great, and I would have loved to see this when I was in the lab. When the software industry says to you: "We will nearly triple your salary, you don't have to work weekends, and you also don't have to feed the mice on a Sunday night." You will 100% take this deal. I was a 10yr+ academic tool maker in biochemistry, built cutting edge microscopes, hardware, and image analysis software. My lab was successful…

This is pretty much my story as well. I work less and get paid much more after leaving academia. Idealism can last so long before one gives in.

Re: Science needs more research software engineers

#120
post #24

I don't think scientists want research software engineers. PIs typically like to assume any and every role on a project, including software design. They don't like handing things off and letting people build modern, best practices software. You start talking about source code control, package managers, a language used in industry that they've never even heard of and don't care enough to learn about, etc., and you jus…

PI here (Research Professor of AI, former industry R&D director, so I have seen both sides). The money will never be the same in academia, and rewards are outside of the control of the PI in most cases (e.g. in Germany, it's a government-level decision and everyone is paid the same), so there are two options: work with average people or find smart & idealistic ones. So...

...please excuse this shameless plug: I'm interested in people who would like to work with me to build research software (both on-premise and AWS cloud) using best practices, in particular using Rust. If you are interesteed in machine learning, natural language processing and information retrieval and would like to work with me as a research software engineer, please do get in touch (leidner at acm dot org). Our new 2 TB RAM dev research group server running Ubuntu 22.04 LTS is waiting for you (CPU/GPU cluster ressources also available) to set up (a VM for) your CI infrastructure. Your software will be used to push forward the frontier of research in machine learning and its applications to language (summarization, question answering) and information retrieval (vertical search engines, learning to rank, unsupervised topic modeling).

Edit: academia can be poorly funded and bureaucratic. The former can be fixed by grant applications, the latter must be tolerated until politicians/activists fix it.

Post reply on HN