Live data from Hacker News

Science needs more research software engineers

nature.com

161–170 of 366 posts

Re: Science needs more research software engineers

#161
post #133

Earlier quoted context omitted.

Are you talking about the letters P, H and D? As in, if you don't have a PhD they don't see you as a peer?

I've honestly yet to meet a research software engineer without a PhD because of the academic bias you will get in, well, academia.

Depending on the definition of RSE, I may or may not have been one. The company I worked for was a Synchrotron Light Source; I worked on software for data collection on X-Ray beamlines. I would say that only about half of those in the same role as me had a PhD.

Moving away from data collection to analysis, the fraction of PhDs went up, but only reached 1.0 when considering the sub-group specialising in structural biology.

Re: Science needs more research software engineers

#163

This looks great, and I would have loved to see this when I was in the lab. When the software industry says to you: "We will nearly triple your salary, you don't have to work weekends, and you also don't have to feed the mice on a Sunday night." You will 100% take this deal. I was a 10yr+ academic tool maker in biochemistry, built cutting edge microscopes, hardware, and image analysis software. My lab was successful…

Thank you for your comment, it was a joy to read and get a peek.

Re: Science needs more research software engineers

#164
post #17

The other side of "We need software engineers in research" is surely the botched covid model from Imperial College way back at the beginning of the pandemic. The group that produced it had the audacity to complain that the software industry didn't make C++ foolproof enough, and it was our fault that the modelling was wrong. This shows the attitude that some folks in these places have - what you do is easy, and rather…

Having worked in both, software is much easier than producing quality scientific discoveries.

Re: Science needs more research software engineers

#165
I think this doesn't frame the problem correctly. It promotes a separation between software engineers and real scientists which helps nobody. Anyone building software for scientific purposes really should understand the science behind it. Also, since better software allows better scientific outcomes, scientists need to understand software engineering. Both should be considered scientists and be on an equal footing.

Re: Science needs more research software engineers

#166
post #138

The first thing I did for my PhD in experimental quantum computing was to rewrite the lab control equipment in Python. It was written in LabView and while it worked well it was an absolute mess of (literally) tangled spaghetti code. I designed and implemented one of the first MVC frameworks for quantum computing in Python. It had an interactive IDE where you could execute scripts and store snippets (e.g. for qubit ca…

This sounds like a good outcome. From my experience, the usual reason small labs setups use things like LabVIEW is that they are usually built by one PhD student without experience and this is the only way they can get something working in a reasonable time. But then the student moves on and the result is something that may work, but nobody can build upon. Still, the situation you describe is unstable because as soon…

I think the software spread to multiple labs and was even used in a commercial startup, so I think it will live on. Also, Python skills seem to be increasingly taught at universities, so many PhDs have had some exposure to the language already.

Re: Science needs more research software engineers

#167
post #126

Earlier quoted context omitted.

> You don't have enough letters after your name to matter, and you will likely be a pet instead of a peer. This is an underrated point. This is the case for programmers in finance as well, and requires a hefty salary premium to put up with.

Are you talking about the letters P, H and D? As in, if you don't have a PhD they don't see you as a peer?

While informal culture and individuals' self-importance do play a role, it's also down to strict old-fashioned salary scales that many universities have in place (even if your day-to-day colleagues see you as a peer, the administrative systems defining your salary range can't/won't). Salaries are often strictly attached to letters behind your name, at a high level, and largely immovable by individual research departments.

And secondly, while your PHD peers may earn more than you, they also often earn much less than software industry averages.

Re: Science needs more research software engineers

#168
post #126

This looks great, and I would have loved to see this when I was in the lab. When the software industry says to you: "We will nearly triple your salary, you don't have to work weekends, and you also don't have to feed the mice on a Sunday night." You will 100% take this deal. I was a 10yr+ academic tool maker in biochemistry, built cutting edge microscopes, hardware, and image analysis software. My lab was successful…

> You don't have enough letters after your name to matter, and you will likely be a pet instead of a peer. This is an underrated point. This is the case for programmers in finance as well, and requires a hefty salary premium to put up with.

It certainly echoes my experience having just left a professional dev job in academia after an 11 year stretch. Anybody without academic credentials relevant to the subject matter is "the help" no matter how much you contribute, and it's flat-out demoralizing.

I worked on a tech-heavy project large enough to get an NYT feature article covering its launch. For it, I collaborated heavily on the service design and logistics, and singlehandedly designed, built, administered, documented, supported, and provided training for the technical infrastructure and more than a dozen related interfaces and tools. In lines of code, it probably landed somewhere in the low 5 figures, but that was certainly way more than it needed to be. It was hackish but durable and performant. It was an exercise in pure generalism— no individual accomplishment was close to technically innovative enough to warrant a novel white paper, but I was invited to speak at a few related conferences about it.

But the professor overseeing the project didn't even mention me or my role in his launch party speech for the folks in our building, let alone anywhere that would have provided career visibility. He thanked and spoke about the contributions of every other major contributor— even the temp worker who ran the machines (he wouldn't want to appear classist after all)— but I got a hand shake and quiet thank you after his speech for my 5 year effort. I was at every related manager's meeting and largely seen as one of three "go-to" people for the project in general, not just tech stuff.

This sort of gatekeeping is a part of academic culture I just don't get. At least in business there's some predictability to people stepping on each other to get to the top, but what's the purpose of this?

Re: Science needs more research software engineers

#169

Earlier quoted context omitted.

> They call it "codes" They/We have probably been calling it that since before you were alive. I wouldn’t be surprised if it comes from the 60s or 70s. Every group of people has their own terminology for stuff. There is no right or wrong.

Yeah. Considering how much time we spend developing and using the damn things, we’ll call them the way we like, thank you very much. We don’t really need condescending software types to just come and explain to us how we’re saying it wrong. That is particularly rich, considering the tendency of CS people to cargo cult and borrow physical terms without understanding them.

[deleted]

Re: Science needs more research software engineers

#170
Responding to title only, and a little rant:

No, scientists need to stop hand-waving away everything that's not their area as "trivial and not worth talking about" because yeah, I've heard that exact line from three separate individuals. When I shrugged it off and still produced tools for them that shaved off a literal 1-2h out of their day and giving them more time to do the actually creative things... they were so overjoyed they forgot to say thanks and immediately proceeded to act as if the tools were always there. They were extremely careful not to show any attribution or even basic human gratitude. Would it kill them?

Yeah, frak that crap.

A lot of scientists become severely tunnel-visioned, arrogant, conservative, downright disrespectful, and very hard to work with. Pile the extremely subpar payment and you really have to just be a bored college kid (or an adult with a huge financial safety net with a passion for the area) to engage with those people in a professional setting at all.

I won't go into details about why I think many are like that because frankly, even if I was 100% correct (and my analysis is not at all flattering) it still doesn't matter one bit. Lasting change comes only from within. Scientists should start holding each other to a higher behavioral and collegiate standards and expel those who don't comply. Peer pressure works but many are so independent and without any oversight that they gradually fall under the illusion that their insufferable quirks are a personality or a personal touch that helps them excel in their area (and it's often the case that they don't excel in it).

Looking down on people because they don't know your extremely niche physics research -- that has zero experiments attached to it, by the way -- is not helping your cause with people who usually get a lot of money AND respect AND a better working environment for actually improving your workday. In most cases these people will of course go where they are at least not looked down upon.

Post reply on HN