Live data from Hacker News

Science needs more research software engineers

nature.com

261–270 of 366 posts

Re: Science needs more research software engineers

#261

Earlier quoted context omitted.

> But, you will get paid SIGNIFICANTLY less than you would doing almost anything else Yes definitely true, a junior software engineer in industry earns about as much as a Professor in academia. How would you justify such a salary at the university? > have zero autonomy, I've written that before. I would argue it's the complete opposite, autonomy is the one big advantage of academia. If you are a software engineer you…

>> and get no respect. > not my experience at all Respect could mean different things to different people, but I find the things you wrote not to be very respectful, but being nice in my evaluation. I have worked in a research lab. While the folks I worked with respected my work, what I would actually consider respectful is including me in the design of the work, not giving me my part of work after they designed the…

> Respect could mean different things to different people, but I find the things you wrote not to be very respectful, but being nice in my evaluation.

> I have worked in a research lab. While the folks I worked with respected my work, what I would actually consider respectful is including me in the design of the work, not giving me my part of work after they designed the thing they want to work on. And while most job industry is not good in this respect, working in research lab is worse than that. Almost all research labs treats SWE like cost center[0].

I don't quite understand your point, do you mean that you as an SRE want to be involved in the grant being written? That might be appropriate if the grant is directly on the topic of software, but completely inappropriate if the SRE writes the labautomation software to be used in the grant. So if the researcher asks you to write a GUI to some lab-instrument for example is that being disrespectful?

Regarding your citation, I also don't get it. The text says that MBAs consider SWEs as cost centers, so that implies it's worse in industry (there's very few MBAs running research labs). I can also tell you that academics are largely considered cost centres by university admin.

Re: Science needs more research software engineers

#262

Earlier quoted context omitted.

Since this thread is turning into a yet another complaints about academia thread: One of the serious downsides of working in academia is you are basically doing the industry's work for them for less pay and they will one day turn around, pat you on the back, then sell your work for millions of dollars. It gets worse honestly the closer you are to applied fields. There, you already straddle the line between what your…

The next thought should be: why doesn't neo-liberal capitalism fix this problem? And: is my characterization of the problem correct? Why not start a new firm that better compensates researchers (and tool makers) for their valuable work? It seems like big tech (especially Google, and perhaps Microsoft) comes in from the commercial side and invests in R&D at reasonable rates for just this purpose! But surely if workers…

"If it always worked, it would be business. Let's go to the pub." -- Me, consoling a grad student after experiment failure #24.

More seriously, if you're in basic science, your skills are valuable in transforming the work into a more useful thing to be used later. Using your science factory model, you have created a reusable widget that other people can use. The science factory model does work, you can see its results in things like MIAME: https://www.nature.com/articles/ng1201-365 Where large pooled datasets are used to get insights otherwise impossible.

There's not a ton of low hanging fruit in some fields, as time has gone on the edges are harder and more expensive to see to be at the cutting edge. Ex: you spend $2M on a microscope that does a cool thing and two years later the new model is all that, a bag of chips, and a soda for the low price of $750k. You hope you have a good enough relationship with the vendor that they will either mod or upgrade your system, or that those two years were enough for you to get ahead. It probably wasn't. And you now have a not as fast ferrari for more money than the fast ferrari.

There is a massive glut of international students willing to work for basically nothing, beholden to your PI by their visas. I say this not as xenophobia, but I was the only working class American (my parents do not have degrees) in the department. All students/postdocs that I worked with were from other countries, or if they were American, their families were doctors, or a faculty member. More generally, the kind of people that might own horses :D.

No firm would take this work on, as the profits are not clear, and the time scales for success range from two years to never. In this case success is "great job publishing, we'll give your lab another 2-3y of funding." After which, you better get good at writing books and eating pasta.

Re: Science needs more research software engineers

#263

Earlier quoted context omitted.

Since this thread is turning into a yet another complaints about academia thread: One of the serious downsides of working in academia is you are basically doing the industry's work for them for less pay and they will one day turn around, pat you on the back, then sell your work for millions of dollars. It gets worse honestly the closer you are to applied fields. There, you already straddle the line between what your…

The next thought should be: why doesn't neo-liberal capitalism fix this problem? And: is my characterization of the problem correct? Why not start a new firm that better compensates researchers (and tool makers) for their valuable work? It seems like big tech (especially Google, and perhaps Microsoft) comes in from the commercial side and invests in R&D at reasonable rates for just this purpose! But surely if workers…

> surely if workers are systematically undercompensated, there is room for a disruptive firm to come in and take the best talent and still make a profit.

Other good replies here, but this part of the comment reveals some assumptions that need better definition. Having been both, I can comfortably say that academics aren’t “workers” in the same way that industry programmers are “workers”. The parent comment is not correct about the norm; programming for research projects is not usually sold for profit later to industry. It happens occasionally, but most academic work stays academic. Sometimes when it does happen, it’s in the form of a spinoff company that brings the original authors of the software, and so they end up getting some of the proceeds… when the business survives.

Also the top comment didn’t say ‘undercompensated’ - in business this has a clinical meaning that someone is being paid lower than market rates. We know that adademics pays lower, but we do not know that it’s lower than market rates for academics. It’s plenty true in industry alone that you can make a lot of money at Facebook or very little money at a small indie game dev shop. That doesn’t mean the indie game devs are undercompensated, it means they’re in a different market.

Starting firms to compensate researchers better is what pharmaceuticals (for example) are. The problem with your suggestion is that the need for income can undermine the ability to do research that is unbiased, risky, controversial, critical, or just free of agenda. If you pay researchers in line with what Bay Area programmers get, it will put an enormous burden on the PIs to make many multiples more money than their peers, and compete with them using a small fraction of the number of people of peer groups.

Re: Science needs more research software engineers

#264
post #133

Earlier quoted context omitted.

Are you talking about the letters P, H and D? As in, if you don't have a PhD they don't see you as a peer?

I've honestly yet to meet a research software engineer without a PhD because of the academic bias you will get in, well, academia.

I'm one who started with only a BS, and I'm at a top-20 public university in the US. It depends on your PIs, but I've definitely been appreciated on many of the projects I've worked on (e.g., listed with 2nd most ownership percentage on invention disclosures, which also won a campus-wide yearly award).

Admittedly, my path was convoluted; I started as a engineer to help with non-research software at a large lab, and got pulled on to projects via reputation. But I was replacing a Master's student who was essentially at the same academic level as me anyway. It does pay less, but I made the tradeoff for the quality of projects, which was worth more to me at this point in my career. It's still much more than I need, just not at industry levels.

Re: Science needs more research software engineers

#265
I work as a research engineer in Italy. My path was academia (Physics)->industry (Finance)->mix. The "mix" where I am now is a strange case: it is located inside an university, nearly everybody has a PhD and we also have PhD students, but we are not academia, it is a private no profit company doing applied research. I am getting the same money I was getting in the industry but also 7+ weeks of vacation per year, more autonomy and less stress. Plus, I am not working to make rich people richer.

Most of my job is about optimizing numeric code, lots of numpy, pandas, numba but also thinking of new algorithms. I get respect and they put my name in the research papers they publish, even if I did not write a word in the paper and have no knowledge at all of the underlying science ;-) Such companies are rare but they do exist. Much better than academia because there is no bureaucracy. I got in because of a friend of a friend...

Re: Science needs more research software engineers

#266
post #23

I've worked at a top research university for the past 15 years as a research software engineer. Many of these comments have some truth but a lot are definitely not universally true. Poor compensation: Yup, I'll cop to that. I could be making way more in "industry" but I make enough to live and I don't build ad-ware and do take pride in my work. Being a PhD but little respect: I actually have the opposite experience.…

Lets be honest. Searching for https://hn.algolia.com/?q=burnout in hn shows so many people suffering from burnouts working in industry.

My SO is a research software engineer.

1. As you say, she doesn't build ad-ware and do take pride... 2. relaxed and convenient job timings 3. In EU - close to 31 days paid + usually Professors are very generous about going home early etc... 4. Pension, unions, unlimited contracts 5. No one gives a shit when github/gitlab/heroku loses credentials on a Easter weekend. (We were in roadtrip) - chill life.

Re: Science needs more research software engineers

#267
post #152

This looks great, and I would have loved to see this when I was in the lab. When the software industry says to you: "We will nearly triple your salary, you don't have to work weekends, and you also don't have to feed the mice on a Sunday night." You will 100% take this deal. I was a 10yr+ academic tool maker in biochemistry, built cutting edge microscopes, hardware, and image analysis software. My lab was successful…

> at least in bioscience... BINGO! that field is notoriously horrible and interacts extremely badly with a 'when not discovered here, not important' syndrome. Biology is brutal toward bio-physicists, mathematicians as well as people who code who they are forced to work with rather than seek out to help them. I still hold up as an example nonsense discussions around p-values in bio vs actual work going on in statistic…

I would say that is a very strong criticism and very warranted! For note, I witnessed the immolation of two careers over retractions of papers that could not be replicated. You could say that the system worked. That was a while ago, and I'm sure the paper mill phenomenon is in full swing. You get echo chambers of PIs that rubber stamp each others work.

In my case, I was in basic science which hit a crisis near 2008 when the NIH was flat funded. This caused a come to Jesus moment, where suddenly all basic science labs were rebranded as translational medicine. My department was absolutely gutted, down from 15 or so PIs to maybe 8ish in the span of a year. Our field was bioenergetics which at the time was pretty competitive, and easy to link to diseases/metabolic disorders. We didn't work with pharma, some labs received contracts for small work. NIH was by far the biggest funder, followed by DARPA and other smaller health organizations.

Re: Science needs more research software engineers

#268
post #136

Earlier quoted context omitted.

I'm curious which company pays a RS 10x more than a SE/RSE. I've found its usually SE/RSE's that make a bit more than RS's but I've never seen an RSE comp significantly outweigh an SE unless its in ML/Crypto. On that note I wish levels.fyi had RS and RSE roles...

This isn't totally crazy but it's only if you stretch the facts enough. An RSE at Oxford will earn 32k GBP, like a postdoc. In theory, that person could be so good they could get the highest starting salary possible at a Tier 1 paying company like a research engineer at Hudson River Trading, which can be > 320k TC. So it's possible but only for a very very small number of people. 3x-5x is much more likely.

> highest starting salary possible at a Tier 1 paying company like a research engineer at Hudson River Trading, which can be > 320k TC.

You don't have to be at a "tier 1 paying company" to exceed 320k TC, that's easily achievable at any tech company that has gone public in the NYC metro or Bay Area. TC at a FAANG (MANGA or whatever they're calling it) for a more senior IC role can easily cross the $500k mark. If you've been their awhile and have been accruing shares that have increased dramatically in value, crossing the 7 figure mark is not unheard of.

So you don't have to stretch the facts too much to realistically achieve the 10x. If you take two very qualified engineer graduating with a PhD, one chooses to go to Google and stay there the other goes to a university and works as an RSE. Fast forward a decade, you'll definitely be seeing a 10x difference in total comp.

Re: Science needs more research software engineers

#269
No, they need to be willing to give a fair wage and benefits.

I looked into scientific programming, and everyone I offered to work for if given an actual position, not an assigned seat and the perpetual threat of firing if a string of soft money grants ran out, and I got zero takers.

The science field is full of performative narcissists who think if they're trying to "cure cancer" or whatever, the lessons from psychology, behavioral economics etc simply do not apply to them.

Re: Science needs more research software engineers

#270

Earlier quoted context omitted.

Since this thread is turning into a yet another complaints about academia thread: One of the serious downsides of working in academia is you are basically doing the industry's work for them for less pay and they will one day turn around, pat you on the back, then sell your work for millions of dollars. It gets worse honestly the closer you are to applied fields. There, you already straddle the line between what your…

The next thought should be: why doesn't neo-liberal capitalism fix this problem? And: is my characterization of the problem correct? Why not start a new firm that better compensates researchers (and tool makers) for their valuable work? It seems like big tech (especially Google, and perhaps Microsoft) comes in from the commercial side and invests in R&D at reasonable rates for just this purpose! But surely if workers…

>why doesn't neo-liberal capitalism fix this problem?

The whole point of academia is to subsidize research before it gets to an application phase. How can a private firm compete with academia who benefits from government funding and are tax exempt? Trying to pin this problem on "capitalism" is just lazy.

Post reply on HN