Science needs more research software engineers • Hacker News Board

Re: Science needs more research software engineers

#191

Post by Azrael3000 » Wed, Jun 01, 2022, 11:17 AM UTC

exdsq wrote: post #134

Azrael3000 wrote: post #54

Earlier quoted context omitted.

There is simply no incentive for science to do proper software engineering. It does not directly produce papers or money, so nobody does it. I've seen it several times in my own career in several countries. PhDs having their own version of the code, incompatible with the one of the Postdoc sitting opposite of them. Each slaving away at their niche project but nobody there to bring it all together. This particular uni…

This isn't completely true. My wife is a postdoc at a medical lab at Stanford and there's a big drive to push good software practices - containerisation, tests, documentation, etc... and a good friend who's a postdoc in medical imaging at Oxford runs training courses for their labs software (along with following all standard engineering practices you'd see at a company). This will be subjective of where you're workin…

Of course my statement wasn't meant to be absolute. I come from an engineering background. I am not surprised that in a medical setting the situation is somewhat different, after all I guess the stakes there are higher. If your CFD code does not perform well, who cares. If your analysis of some medication is crap, chances somebody cares is going to be much higher.

Also from experience, it helps if universities have closer ties to industry. As an industrial player you can't work with software that does not deliver reproducible results.

Re: Science needs more research software engineers

#192

Post by Tade0 » Wed, Jun 01, 2022, 11:21 AM UTC

I guess I fit this category, because I'm working on a project that's based off a scientific paper and is basically a Matlab script turned webapp so that all those calculations don't have to be done locally.

The pay is indeed around 60-70% of what the industry offers, but I would have the same rate if was making a CRUD app[0] for the same large company, so it's more about the employer than the type of work.

[0] which is something I did in the past - a webapp generating documents ensuring regulatory compliance. This organisation likes to make webapps.

Re: Science needs more research software engineers

#193

Post by maccard » Wed, Jun 01, 2022, 11:22 AM UTC

evandijk70 wrote: post #184

noobermin wrote: post #180

Earlier quoted context omitted.

European countries have lower salaries in general? Although their social safety nets are better. Also, assistant professors (or the equivalent there) generally make less but do probably make more once they get tenure. I'm assuming they meant the tenure-track position itself is ~30K USD, but making tenure usually does mean a pay increase.

In the Netherlands, 30k is the starting salary for a PhD student. 30k for a tenure track position sounds insane to me.

In Ireland PhD stipends are closer to 13-17k. It's not a perfect comparison because the PhD stipends are tax free, so your comparative salary would be in the 17-20k mark. That said, Postdoc research positions are much closer to 40k than 30k.

Re: Science needs more research software engineers

#194

Post by Azrael3000 » Wed, Jun 01, 2022, 11:24 AM UTC

w0de0 wrote: post #79

Azrael3000 wrote: post #54

Earlier quoted context omitted.

There is simply no incentive for science to do proper software engineering. It does not directly produce papers or money, so nobody does it. I've seen it several times in my own career in several countries. PhDs having their own version of the code, incompatible with the one of the Postdoc sitting opposite of them. Each slaving away at their niche project but nobody there to bring it all together. This particular uni…

Is there any solution to be found through analogy with established fields' relationship between engineering and research disciplines? How, for instance, do physicists share formal models which rocket engineers might apply? Certainly the physicists aren't out developing unit tests for engines which apply their theory, but nor are the engineers exasperated by physicists rejecting their standardized tools (I imagine?).

Well from experience the interface is conferences and papers. On conferences you get to know the latest stuff that is going on and the implementation details can be found in papers.

What needs to be said though is that reimplementing something from a paper is near impossible. I had to do that a couple of times and I was only fully successful if the paper was accompanied by some open source code as there always are tiny little edge cases or initialization details that won't be covered in the paper.

Re: Science needs more research software engineers

#195

Post by goodpoint » Wed, Jun 01, 2022, 11:24 AM UTC

exdsq wrote: post #134

Azrael3000 wrote: post #54

Earlier quoted context omitted.

There is simply no incentive for science to do proper software engineering. It does not directly produce papers or money, so nobody does it. I've seen it several times in my own career in several countries. PhDs having their own version of the code, incompatible with the one of the Postdoc sitting opposite of them. Each slaving away at their niche project but nobody there to bring it all together. This particular uni…

This isn't completely true. My wife is a postdoc at a medical lab at Stanford and there's a big drive to push good software practices - containerisation, tests, documentation, etc... and a good friend who's a postdoc in medical imaging at Oxford runs training courses for their labs software (along with following all standard engineering practices you'd see at a company). This will be subjective of where you're workin…

> good software practices

> containerisation

Pick one.

Re: Science needs more research software engineers

#196

Post by javajosh » Wed, Jun 01, 2022, 11:25 AM UTC

noobermin wrote: post #174

throwmamatrain wrote: post #60

This looks great, and I would have loved to see this when I was in the lab. When the software industry says to you: "We will nearly triple your salary, you don't have to work weekends, and you also don't have to feed the mice on a Sunday night." You will 100% take this deal. I was a 10yr+ academic tool maker in biochemistry, built cutting edge microscopes, hardware, and image analysis software. My lab was successful…

Since this thread is turning into a yet another complaints about academia thread: One of the serious downsides of working in academia is you are basically doing the industry's work for them for less pay and they will one day turn around, pat you on the back, then sell your work for millions of dollars. It gets worse honestly the closer you are to applied fields. There, you already straddle the line between what your…

The next thought should be: why doesn't neo-liberal capitalism fix this problem? And: is my characterization of the problem correct? Why not start a new firm that better compensates researchers (and tool makers) for their valuable work? It seems like big tech (especially Google, and perhaps Microsoft) comes in from the commercial side and invests in R&D at reasonable rates for just this purpose! But surely if workers are systematically undercompensated, there is room for a disruptive firm to come in and take the best talent and still make a profit.

Perhaps the characterization is wrong and the EV (expected value) of this work is far lower than you think (this seems likely), and/or there are externalities like regulation, or the leverage of prestige that traditional orgs (e.g. universities and publishers) wield, that warp the profit incentive. Or (and this is my cynical view) pure science was always best left to the hobbyists. Historically the most important discoveries have come only rarely and to those who loved doing science in their free time or, more rarely, when a talented individual found a patron. Building a science factory and hiring science factory workers not only sounds distasteful, but it doesn't seem to work very well. (The exceptions being those very capital intensive projects like the LHC which require a large pool of professional scientists and engineers to do the experiment.)

Re: Science needs more research software engineers

#197

Post by dwrodri » Wed, Jun 01, 2022, 11:26 AM UTC

Computer Engineering PhD here. I’m seriously considering transitioning out of my program because I’ve reached a point where I’m just writing tons of code on a graduate student stipend and quite frankly, it seems doors are opening to me which I had previously thought would only be open with the PhD.

I feel like my experience would be very different if I had one full-time research engineer who was paid a respectable sum to just... help me write code. I like working with undergraduate students, but I can’t do expect them to commit to the task like I would, or a full-time employee. They just don’t have the time.

I’m doing my work extending / improving open-source CPU simulators and it’s just such a huge amount of code. I feel like my work would be so much more useful to others if I just had another pair of hands to work on docs, do ops work, go back and forth on decisions. My advisor doesn’t have the time for such things.

- - -

On the topic of academic pay: as a PhD student in Boston, I get paid $2000 a month pre-tax. Studios and 1BR apartments renting for under $1500/mo in a 30-minute commute radius of my campus are basically non-existent, so roommates and/or long commutes are a must. I’m realizing now why so few US citizens participate in graduate CS/CE programs: if you aren’t looking for a path to citizenship, it just doesn’t seem like a great way to spend your mid-to-late 20s.

Re: Science needs more research software engineers

#198

Post by hackernewds » Wed, Jun 01, 2022, 11:33 AM UTC

dwrodri wrote: post #197

Computer Engineering PhD here. I’m seriously considering transitioning out of my program because I’ve reached a point where I’m just writing tons of code on a graduate student stipend and quite frankly, it seems doors are opening to me which I had previously thought would only be open with the PhD. I feel like my experience would be very different if I had one full-time research engineer who was paid a respectable su…

this inadvertent details why the immigration path to citizenship is almost modern day labor exploitation. don't get me started on how the labor certification process for the green card furthers that in a timeline that is decades

Re: Science needs more research software engineers

#199

Post by hackernewds » Wed, Jun 01, 2022, 11:34 AM UTC

javajosh wrote: post #196

noobermin wrote: post #174

Earlier quoted context omitted.

Since this thread is turning into a yet another complaints about academia thread: One of the serious downsides of working in academia is you are basically doing the industry's work for them for less pay and they will one day turn around, pat you on the back, then sell your work for millions of dollars. It gets worse honestly the closer you are to applied fields. There, you already straddle the line between what your…

The next thought should be: why doesn't neo-liberal capitalism fix this problem? And: is my characterization of the problem correct? Why not start a new firm that better compensates researchers (and tool makers) for their valuable work? It seems like big tech (especially Google, and perhaps Microsoft) comes in from the commercial side and invests in R&D at reasonable rates for just this purpose! But surely if workers…

They expected value theory is very plausible. there are a lot of r&D projects that basically produce zero output for decades. high risk high reward

Re: Science needs more research software engineers

#200

Post by sseagull » Wed, Jun 01, 2022, 11:39 AM UTC

rsfern wrote: post #171

bmitc wrote: post #90

Earlier quoted context omitted.

I have never heard or read anyone, beside physicists, say codes for software code. It doesn't even make sense. Where have you seen that used?

It’s used fairly often in the US national lab community (Not just among physicists). I think the point is that the physicists were among the very early adopters of computing systems. I’m not sure about the history of the term “code” used this way, but it might literally predate compiled software since Ulam and von Neumann invented the Monte Carlo method in 1948 during the Manhattan Project. Personally I think it just…

Now I do wonder about the history of the term.

The tendency is to call computational science software “codes”, but I’ve never heard a scientist call, ie a web browser, a code.

There’s a different connotation. Like how theres a different connotation between a program, an application, and and app. The are largely synonymous, but have slightly different uses.