Live data from Hacker News

Science needs more research software engineers

nature.com

271–280 of 366 posts

Re: Science needs more research software engineers

#271
post #172
post #145

Earlier quoted context omitted.

This. I have worked for almost 15 years in academic research, but in very close collaboration with the steel industry. The code we write can help steel companies to save millions when developing new products. This is quite complex software, which combines materials science, mechanical engineering and advanced mathematical concepts while requiring high performance and reliability. I found a nice position for a tenure…

seriously? in what kind of alternative reality does academia live to offer 30k?

come to Denmark! you can make about 60k a year as a phd student!

Re: Science needs more research software engineers

#272
I spent a few years as a RSE in the mid-2010's. Supporting people doing actual science is a rewarding job.

The worst parts of that job are the same for every body in research that isn't a PI pulling in lots of grant money: low pay, terrible or non-existent benefits, bureaucracy ran by non-science people, making do with older equipment.

There is also a breaking in period where you have to prove you are not a moron. Since you come from a different educational area, and most likely aren't as educated, this can take some time and effort.

The best part of the job: getting to work with highly educated people, getting to see science be discovered, being able to actually plan and execute projects because everything isn't on fire, regular business hours with no after hours support calls.

Re: Science needs more research software engineers

#273

Earlier quoted context omitted.

The next thought should be: why doesn't neo-liberal capitalism fix this problem? And: is my characterization of the problem correct? Why not start a new firm that better compensates researchers (and tool makers) for their valuable work? It seems like big tech (especially Google, and perhaps Microsoft) comes in from the commercial side and invests in R&D at reasonable rates for just this purpose! But surely if workers…

>why doesn't neo-liberal capitalism fix this problem? The whole point of academia is to subsidize research before it gets to an application phase. How can a private firm compete with academia who benefits from government funding and are tax exempt? Trying to pin this problem on "capitalism" is just lazy.

No, lazy would be straw-manning a stranger's argument for no good reason to elicit an emotional reaction. It's a style of communication that seeks conflict rather than understanding, and there is plenty of it on twitter and reddit, but not here.

Re: Science needs more research software engineers

#274

Earlier quoted context omitted.

Since this thread is turning into a yet another complaints about academia thread: One of the serious downsides of working in academia is you are basically doing the industry's work for them for less pay and they will one day turn around, pat you on the back, then sell your work for millions of dollars. It gets worse honestly the closer you are to applied fields. There, you already straddle the line between what your…

The next thought should be: why doesn't neo-liberal capitalism fix this problem? And: is my characterization of the problem correct? Why not start a new firm that better compensates researchers (and tool makers) for their valuable work? It seems like big tech (especially Google, and perhaps Microsoft) comes in from the commercial side and invests in R&D at reasonable rates for just this purpose! But surely if workers…

>The next thought should be: why doesn't neo-liberal capitalism fix this problem?

Neo-liberal capitalism fixes problems?!

Re: Science needs more research software engineers

#275

Well, then science needs to do more to create an environment where software engineers would want to be an RSE. From my experience "science" does not appreciate RSEs, the compensation is bad, the freedom is bad, and you get all of the big company bureaucracy from the University. It's just a bad environment and to top it off, even if you have a PhD many will see you as "less than."

This last point, and replies to this all seem to talk about the research industry. But it has been my experience that this same dynamic of being “less than” exists in any industry that is not primarily software. I am a mechanical engineer by degree, but a software engineer by 30 years of practice. That has allowed me to thrice straddle the divide where I worked on software that was not just software for softwares sak…

I have to agree that in some environments you can see comp sci majors picking on anyone who isn’t part of traditional “software”.

I switched from engineering into comp sci at my university and I can safely say that the elitism for software was stoked by the elitism of engineering.

While I was in engineering, everyone bragged about how they were going to get “The iron ring” (a thing we give to only engineers in Canada). When I switched over to comp sci, all my old engineering buddies would say to me, “yeah, but you don’t get the ring” like it meant everything.

After I graduated, I noticed the comp sci “software” folks didn’t like engineering folks much (especially new grads). When I asked why, they shared similar stories to mine. The engineers and their pumped up pride at my local university had hurt their relation to other fields by being arrogant.

I’m not trying to justify this kind of prejudice, but the reasons it happens are fairly obvious to me. It’s sad cause all of us are so similar in our trades and will likely end up doing similar work as well.

Re: Science needs more research software engineers

#276
post #229
post #152

Earlier quoted context omitted.

> at least in bioscience... BINGO! that field is notoriously horrible and interacts extremely badly with a 'when not discovered here, not important' syndrome. Biology is brutal toward bio-physicists, mathematicians as well as people who code who they are forced to work with rather than seek out to help them. I still hold up as an example nonsense discussions around p-values in bio vs actual work going on in statistic…

No there isn't good money in physics and chemistry or pure math. PhD chemists almost never make 6 figures even in high cost of living areas serving as a specialist. I made less as a senior scientist or a project manager in chemistry than I do as an entry level software engineer. I don't know how many physicists I've met who work minimum wage jobs, usually call centers, after their PhD/post doc (even finding a PhD is…

>"I bet a third of them once were physical scientists/mathematicians who mostly regret their degrees"

Would mathematicians truly be regretting their degrees, if they decide to work in software? I read that mathematics one of the best degrees for a career in software engineering, as computer science is very closely related to mathematics (to the point where studies of algorithms are largely the same for mathematics and computer science students).

Re: Science needs more research software engineers

#277

I work as a research engineer in Italy. My path was academia (Physics)->industry (Finance)->mix. The "mix" where I am now is a strange case: it is located inside an university, nearly everybody has a PhD and we also have PhD students, but we are not academia, it is a private no profit company doing applied research. I am getting the same money I was getting in the industry but also 7+ weeks of vacation per year, more…

Sounds heavenly, are they hiring?

Re: Science needs more research software engineers

#278

This looks great, and I would have loved to see this when I was in the lab. When the software industry says to you: "We will nearly triple your salary, you don't have to work weekends, and you also don't have to feed the mice on a Sunday night." You will 100% take this deal. I was a 10yr+ academic tool maker in biochemistry, built cutting edge microscopes, hardware, and image analysis software. My lab was successful…

Having worked in academia before the age of 40 (though not in computer science), I can agree with this view. To enjoy the benefits of status, compensation and, to a certain extent, flexibility, one has to reach the level of professor. After failing to reach this level at 40, I switched to software development, without ever regretting it.

A while ago I saw a position for image recognition in astronomy advertised by the university of my town. It sounded all very exciting for someone who enjoys figuring out solutions for a complex task -- until I looked up the pay grade: Less than half of my current hourly rate, without the flexibility I have (working from everywhere I want, even pre-Covid). Well, the problem seems to be that a competitive salary for a software engineer would probably have to be higher than for the professor leading the group.

As an aside, I recently had a discussion with a friend in my country's military about the cyber defence forces wanting to recruit software engineers. There is a similar problem here: if they do not use contractors (whom they can pay what they ask for), they have difficulty finding an appropriate pay grade, since a well-qualified software engineer would have to be paid better than a general.

Re: Science needs more research software engineers

#279
post #152

This looks great, and I would have loved to see this when I was in the lab. When the software industry says to you: "We will nearly triple your salary, you don't have to work weekends, and you also don't have to feed the mice on a Sunday night." You will 100% take this deal. I was a 10yr+ academic tool maker in biochemistry, built cutting edge microscopes, hardware, and image analysis software. My lab was successful…

> at least in bioscience... BINGO! that field is notoriously horrible and interacts extremely badly with a 'when not discovered here, not important' syndrome. Biology is brutal toward bio-physicists, mathematicians as well as people who code who they are forced to work with rather than seek out to help them. I still hold up as an example nonsense discussions around p-values in bio vs actual work going on in statistic…

"Not to criticize too strongly, but given the above, combined with it's reproducabilty crisis, and existential problem of being in the back-pocket of big-pharma, I seriously doubt the professional integrity of a lot of people in the field."

Lack of professional integrity is a very real problem.

Over the past two years I wrote fairly frequently about some of the nonsensical / pseudo-scientific COVID papers that got published, especially the quality problems in epidemiology. Epidemiology isn't bioscience (actually that's one of the problems with it - total lack of biology), but it's adjacent. After that I got contacted by a former research software engineer who worked with top epidemiology teams in the UK. I also got contacted by a member of the SAGE committee.

Both of them told me some absolutely mind-blowing stories of ethical malpractice. I wasn't totally surprised because it was obvious that those sorts of things must have been going on behind the scenes just from reading their model source code, reports, watching their behavior etc. The RSE had become so disgusted at what he'd seen that he actually left the country and switched from working at Oxford to some US university I'd never heard of, switching fields along the way too. Quite the downgrade in prestige but after years of trying to help epidemiologists he concluded the entire field was utterly morally corrupt and he wanted nothing to do with it.

Here are some of the more memorable things I was told by those two scientists:

- The RSE at one point found a bug in a FORTRAN model being used to model malaria outbreaks. It had been used as the basis for hundreds of papers but at critical points was using pointer values as variables instead of dereferencing them. Obviously, a typical pointer has a very different value to most organic things (some FFI bug). He reported this bug to the authors and got a reply back within 30 minutes saying they'd checked the papers (all of them) and it didn't affect the results. This claim was very obviously a lie: not only could they not possibly have checked even one paper in 30 minutes but he already knew fixing the bug did indeed change results! They didn't care and he was shocked that his "colleagues" would bullshit him so directly, especially as they must have known that he would know.

- Same guy flagged code quality issues to some of the scientists and proposed introducing some rules designed to improve quality. He was dismissed with the words: "oh , we're scientists, we don't write bugs".

- The SAGE member told me about some of the internal discussions they had. Criticisms of the methodological validity and accuracies of their models were dismissed with reasoning like this: "that person reads the Spectator so it doesn't matter what they think". Relatedly, he made clear that the supposedly scientific SAGE predictions were sometimes being altered to reduce criticism of the group by left wing media and journalists. The changes were presented as "the science changed" but that wasn't what was going on behind the scenes.

- Malaria research is (supposedly) being badly distorted by the Gates Foundation. Gates only cares about eradication which leads to lots of problems. There are some smaller ones, like many researchers don't genuinely believe that's possible but lie on their grant applications to make mitigation efforts sound like eradication efforts. And then there were unethical experiments on entire populations where e.g. whole areas are blanketed in anti-malarial drugs. If it works, great, you eradicated malaria in that area. If it doesn't you just selected for drug-resistant mosquitos and now the drugs that were being used only to treat the serious cases don't work for anyone. He told me this has actually happened more than once.

- The RSE told me they'd at one point tried to recruit an RSE working with climatologists to help them with their modelling (a belief that climatologists are more rigorous than they are seems to be common in epidemiology). The RSE they interviewed refused to take the job. His reason was he was quitting academia entirely, as he was so disturbed by the practices he'd seen.

A few years ago if you'd told me that a whole research field could be unethical I'd have thought you were crazy because, well, that's a whole lot of people being painted by a very broad brush. Now I've seen it for myself and heard from other former insiders, it's easy to see what happens - the honest ones discover what's happening and leave. Because academia hardly ever penalizes intellectual dishonesty, the pool of people who remain are the ones who are OK with it and have learned that it works / has no consequences. Things steadily become more and more toxic.

Re: Science needs more research software engineers

#280
Most industries outside of the actual tech industry treat software engineers as low level technicians, pay and promote them accordingly, and then complain they can’t find anyone competent. Academia is that plus some of the worst internal culture you’ve ever seen. The work can be fascinating but unless you’ve already got fuck you money and just want something to do, stay away.
Post reply on HN