Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Science needs more research software engineers

nature.com

Post reply on HN
Print view View original

181–190 of 366 posts

Re: Science needs more research software engineers

#181

Post by aqme28 »

joshvm wrote: post #101
Azrael3000 wrote: post #54

Earlier quoted context omitted.

There is simply no incentive for science to do proper software engineering. It does not directly produce papers or money, so nobody does it. I've seen it several times in my own career in several countries. PhDs having their own version of the code, incompatible with the one of the Postdoc sitting opposite of them. Each slaving away at their niche project but nobody there to bring it all together. This particular uni…

RSEs should not be working on, or assisting, research at the PhD level. By that I mean news/skunkworks stuff. They should be taking the output from a PhD and turning it into core software for the group. That doesn't solve the collaboration issue between PhDs/Postdocs, but there is a particular point in a research project lifecycle where it makes sense to hire an RSE. A bigger challenge is that most PIs are not projec…

> They should be taking the output from a PhD and turning it into core software for the group.

And what if the PhD’s output relies on software to even exist?

Re: Science needs more research software engineers

#182

Post by YetAnotherNick »

cycomanic wrote: post #158
toma_caliente wrote: post #9

It's already been said several times over but I'm going to throw it out anyway. I'd love to work at a university as a research software engineer. I find things like bioinformatics fascinating and even interviewed at a research university on the east coast. But, you will get paid SIGNIFICANTLY less than you would doing almost anything else, have zero autonomy, and get no respect. Those 3 things are basically what most…

> But, you will get paid SIGNIFICANTLY less than you would doing almost anything else Yes definitely true, a junior software engineer in industry earns about as much as a Professor in academia. How would you justify such a salary at the university? > have zero autonomy, I've written that before. I would argue it's the complete opposite, autonomy is the one big advantage of academia. If you are a software engineer you…

>> and get no respect. > not my experience at all

Respect could mean different things to different people, but I find the things you wrote not to be very respectful, but being nice in my evaluation.

I have worked in a research lab. While the folks I worked with respected my work, what I would actually consider respectful is including me in the design of the work, not giving me my part of work after they designed the thing they want to work on. And while most job industry is not good in this respect, working in research lab is worse than that. Almost all research labs treats SWE like cost center[0].

[0]: https://www.kalzumeus.com/2011/10/28/dont-call-yourself-a-pr...

Re: Science needs more research software engineers

#183

Post by euroderf »

Having a Ph.D sounds like an ANTI-qualification for this kind of position. A PhD would be a diversion from the kind of practical experience that would make a software developer with a sci/tech background & scientific interests useful in an environment full of smart people who possibly can't code worth a damn.

More important than having one's own PhD laser-focus would be an ability to communicate with a variety of people doing their own research, and understand their domains just well enough to write/fix/improve their damn software and maybe even publish it (semi-productised) as open source too.

A couple of application languages, some shell skills, maybe some data wrangling, and some software development common sense borne of experience.

Re: Science needs more research software engineers

#184

Post by evandijk70 »

noobermin wrote: post #180
siva7 wrote: post #172

Earlier quoted context omitted.

seriously? in what kind of alternative reality does academia live to offer 30k?

European countries have lower salaries in general? Although their social safety nets are better. Also, assistant professors (or the equivalent there) generally make less but do probably make more once they get tenure. I'm assuming they meant the tenure-track position itself is ~30K USD, but making tenure usually does mean a pay increase.

In the Netherlands, 30k is the starting salary for a PhD student.

30k for a tenure track position sounds insane to me.

Re: Science needs more research software engineers

#185

Post by epse »

euroderf wrote: post #183

Having a Ph.D sounds like an ANTI-qualification for this kind of position. A PhD would be a diversion from the kind of practical experience that would make a software developer with a sci/tech background & scientific interests useful in an environment full of smart people who possibly can't code worth a damn. More important than having one's own PhD laser-focus would be an ability to communicate with a variety of peo…

Possibly, but at least some experience in academia is terribly helpful, speaking from experience. There's a lot of particuliarities in funding, organisation, terminology, special stacks, ... You can certainly learn them "on the job", but if you already know them, even better

Re: Science needs more research software engineers

#186

Post by euroderf »

epse wrote: post #185
euroderf wrote: post #183

Having a Ph.D sounds like an ANTI-qualification for this kind of position. A PhD would be a diversion from the kind of practical experience that would make a software developer with a sci/tech background & scientific interests useful in an environment full of smart people who possibly can't code worth a damn. More important than having one's own PhD laser-focus would be an ability to communicate with a variety of peo…

Possibly, but at least some experience in academia is terribly helpful, speaking from experience. There's a lot of particuliarities in funding, organisation, terminology, special stacks, ... You can certainly learn them "on the job", but if you already know them, even better

Just wondering then, how much of that kind of knowledge is portable among academic research environments, and how much is peculiar to every individual environment ? Is there something that is dying to be written up for people entering the field ?

Re: Science needs more research software engineers

#187

Post by joshvm »

elementalest wrote: post #159
joshvm wrote: post #101

Earlier quoted context omitted.

RSEs should not be working on, or assisting, research at the PhD level. By that I mean news/skunkworks stuff. They should be taking the output from a PhD and turning it into core software for the group. That doesn't solve the collaboration issue between PhDs/Postdocs, but there is a particular point in a research project lifecycle where it makes sense to hire an RSE. A bigger challenge is that most PIs are not projec…

RSE's can and often absolutely should be involved at the PhD level. In my experience, collaboration between the scientist and engineer in the process of research iterations almost always produce better results. Each has insights the other may not, likely leading to better outcomes for the research, final product/tool and time taken. The scientist just wants to focus on their research and once they have a barely worki…

Yeah let me be clear. PhD students absolutely should get guidance from experienced engineers (so I was a bit over-zealous with "assist" in my parent post). But this should be more like understanding best practices, and they should feel free to ask questions and figure out how to write better code. There are initiatives to do this called Software Carpentry.[0] However, RSEs should not be writing code for students doing PhD level projects in my opinion, for exactly the reasons you mention.

I know some of the big research councils do this in the UK. For example STFC has a program where they'll work with universities and companies to production-ise research code.

> The scientist just wants to focus on their research and once they have a barely working proof of concept, hand it over to the engineer to figure the rest out. The engineer wants a well specified design and prototype that they can lightly refactor to clean up, scale up and turn into a product/tool.

As you say, this is a great idea in principle. In reality I think that it's really difficult to make it work.

[0] https://www.software.ac.uk/programmes-events/carpentries/sof...

Re: Science needs more research software engineers

#188

Post by joshvm »

aqme28 wrote: post #181
joshvm wrote: post #101

Earlier quoted context omitted.

RSEs should not be working on, or assisting, research at the PhD level. By that I mean news/skunkworks stuff. They should be taking the output from a PhD and turning it into core software for the group. That doesn't solve the collaboration issue between PhDs/Postdocs, but there is a particular point in a research project lifecycle where it makes sense to hire an RSE. A bigger challenge is that most PIs are not projec…

> They should be taking the output from a PhD and turning it into core software for the group. And what if the PhD’s output relies on software to even exist?

The challenge is that usually PhDs are not writing software that's designed for production, is very specific (for a single conference or journal paper) and often the utility of the code is nebulous until the end of the project. So what you don't want is for RSE's to spend ages writing code for a PhD project (which could be done by the student) only to have it thrown in the trash when the student leaves, or when they pivot to a new avenue of research.

I'm saying this as someone who did a PhD and who wrote a lot of code, including refactoring legacy codebases in my group.

Where the utility in having an RSE lies is where the group is all using some shared codebase that gets touched by everyone. This is the sort of cruft that I had to work with: legacy frankencode that generations of students and postdocs had added to. It would have made a ton of sense to pay someone to spend a year optimising it (which is ultimately what I did). But you want to make sure that RSE's maximise utility in the group. Having them work on individual student projects is not an effective use of their time IMO.

Re: Science needs more research software engineers

#189

Post by Azrael3000 »

joshvm wrote: post #187
elementalest wrote: post #159

Earlier quoted context omitted.

RSE's can and often absolutely should be involved at the PhD level. In my experience, collaboration between the scientist and engineer in the process of research iterations almost always produce better results. Each has insights the other may not, likely leading to better outcomes for the research, final product/tool and time taken. The scientist just wants to focus on their research and once they have a barely worki…

Yeah let me be clear. PhD students absolutely should get guidance from experienced engineers (so I was a bit over-zealous with "assist" in my parent post). But this should be more like understanding best practices, and they should feel free to ask questions and figure out how to write better code. There are initiatives to do this called Software Carpentry.[0] However, RSEs should not be writing code for students doin…

Well I think the best way would be for RSEs to maintain a project and PhDs should then contribute to that project via pull requests. RSEs can then point towards proper coding styles, test development, etc.

That would ensure that the contributions of the PhDs does not get lost and they learn how to properly contribute to a project.

Post reply on HN