Live data from Hacker News

Why we need to create careers for research software engineers

scientific-computing.com

41–50 of 79 posts

Re: Why we need to create careers for research software engineers

#41
post #34

Earlier quoted context omitted.

ORNL? Maybe we could get lunch sometime, I've been curious about the computational climate stuff since I got here.

LLNL, actually :) We collaborate with the folks in that group, though!

Oh cool, I was out there for a summer in grad school making Sankey diagrams and working on some conceptual stuff for cities research. I definitely miss the bay area. Haha, I figured at the very least you'd be in contact with some people here, lots of climate stuff going on.

Re: Why we need to create careers for research software engineers

#42
post #13

I've interviewed for a number of these types of jobs in the last year or so. The pay is just too poor to even think about it. One large university with a super well-funded and growing medical school in a huge mega-city was looking to hire someone with 5+ years of deep experience in both Python and C/C++, as well as some familiarity or past education in statistics and machine learning, to be the software development l…

This is a really good analysis. I think that as it stands, the best hope for filling these jobs is to get a highly talented postdoc who will accept this role in exchange for assistance with his or her academic career aspirations (another way to do it is more insidious, which is where a PI stretches out a doctoral student's completion time to hang on to his or her talented and cheap programmer a few more years).

Of course, this means getting authorship (doesn't have to be first, but it needs to be there), recommendation letters, and so forth. It is also a temporary situation, since what is happening here is that the postdoc is doing software development for well below market rate in order to get a boost in climbing an academic career ladder.

Unfortunately, this scenario makes it even less appealing for a software developer who isn't part of the academic ladder. Because they aren't getting any special boost, career-wise, it just becomes a demanding job with long hours and low pay.

The setup is virtually guaranteed to result in very low quality software. Post docs may be brilliant people, but it's unlikely they will learn to write high quality, maintainable software in a transitional period between grad school and a faculty position, especially since it's something they'd like to leave behind as quickly as possible. And it's equally unlikely that someone talented at software development will stick around in an organization like this where they will never get the chance to lead a project and make a decent salary, especially when those opportunities exist outside universities.

The idea of a research programmer role is a really good one but I'm not sure I can really see faculty accepting it. It would be a major cultural transition for them.

Re: Why we need to create careers for research software engineers

#43
post #38

Earlier quoted context omitted.

> I wrote the majority of the analysis code for one of the chapters of the IPCC 5th Assessment and yet was told my name couldn't be put anywhere as I wasn't a 'scientist'. Can you be more specific about the nature of your contributions? Writing code would seem analogous to performing experiments, which alone is insufficient to warrant authorship (but should get an acknowledgment).

You raise an important question. It is important to detail the actual contribution, since programming may be perceived as purely implementing an idea with no creative input. That said… in my career so far, I don't think I've ever seen a clean example of coding someone else's idea to spec. If you successfully take an idea from concept to working software, or if a research project requires complex software to be writte…

Isn't this simialar to what experimental physics is. Just having an idea is different from implementing and testing it.

Re: Why we need to create careers for research software engineers

#44
post #43
post #38

Earlier quoted context omitted.

You raise an important question. It is important to detail the actual contribution, since programming may be perceived as purely implementing an idea with no creative input. That said… in my career so far, I don't think I've ever seen a clean example of coding someone else's idea to spec. If you successfully take an idea from concept to working software, or if a research project requires complex software to be writte…

Isn't this simialar to what experimental physics is. Just having an idea is different from implementing and testing it.

Sorry, I really don't know anything about experimental physics, so I couldn't comment.

eI see software development as somewhat similar to writing a novel - someone could give you an idea, a general outline, even a fairly detailed outline, but the writer who puts it into words will almost inevitably make a meaningful creative contribution.

Here's one way to think about it - if you had two different writers, or software developers, how different would the end product look?

Re: Why we need to create careers for research software engineers

#45

I think this is done much better by e.g. Google already today. Academia has have their 50 year run with software research and the end result is quite disappointing. I have been peripherically involved with some recent academic research projects in Europe. The way I see it it's basically a way of keeping senior academics occupied and also diverting tax revenues to EU companies for no good reason. It's also a breeding…

> Academia has have their 50 year run with software research and the end result is quite disappointing.

You might want to tell that to the students and faculty at MIT, Berkeley and CMU who wrote the papers and much of the software powering your shiny new macbook and iphone.

Re: Why we need to create careers for research software engineers

#46
post #33

Working in research as a software engineer can be very frustrating. I wrote the majority of the analysis code for one of the chapters of the IPCC 5th Assessment and yet was told my name couldn't be put anywhere as I wasn't a 'scientist'. I also consistently see people who have never written much code put in charge of projects where the majority of the work/research is writing software. These projects end up with no c…

Some would argue that no OO use is a good thing (assuming you are referring to object oriented programming). Expanding on this, writing research code is quite a bit different than writing production code. In particular, requirements can change quickly and drastically. This means that very open designs where you don't hide data (e.g. as is common in OO programming) are often the best designs.

just no.

Re: Why we need to create careers for research software engineers

#48

I think this is done much better by e.g. Google already today. Academia has have their 50 year run with software research and the end result is quite disappointing. I have been peripherically involved with some recent academic research projects in Europe. The way I see it it's basically a way of keeping senior academics occupied and also diverting tax revenues to EU companies for no good reason. It's also a breeding…

> Academia has have their 50 year run with software research and the end result is quite disappointing. You might want to tell that to the students and faculty at MIT, Berkeley and CMU who wrote the papers and much of the software powering your shiny new macbook and iphone.

Most of them were students, not academic researchers . The sole reason they were there was that it was the only way to get to computers.

Since you mention MIT: I guess one example is how much effort Marvin Minsky's influence wasted in AI research and how he single-handedly killed more relevant research in that area for decades.

What I was trying to get at though is what I have seen in the EU. Basically they don't even attempt to do basic research - most of the money goes to building product-like software systems, but without any realistic customer feedback loop (they already got the funding from oblivious tax payers thousands of miles away!). It's just wanking.

Re: Why we need to create careers for research software engineers

#49
post #20

Earlier quoted context omitted.

Does this also apply to large industrial labs, e.g. MSR, IBM Research, Google research etc.? Or are there software developers who work together with researchers in those places, perhaps resulting in better quality code?

Microsoft Research has RSDEs ... research software design engineers. IBM Research also has a research software engineer position, with its own career progression track. These folks are generally fantastic developers who have been professional software engineers some point in their careers. Moreover, I have never personally seen a case where a research engineer was not put on a paper for being just a coder. It is prob…

Correct me if I'm wrong, but my understanding was that being an RSDE at MSR was still a dead-end position.

FWIW product engineers usually get bonuses for patents, so there's the obvious incentive of $5k in your pocket.

Post reply on HN