Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Software engineering research is a train wreck

buttondown.email

Post reply on HN
Print view View original

121–130 of 174 posts

Re: Software engineering research is a train wreck

#121

Post by jdlshore »

qznc wrote: post #57

Greg Wilson is more positive. Though he does agree that nobody cares. https://third-bit.com/2021/07/17/software-engineerings-great...

I'm skeptical of Wilson's reporting. One of his quoted studies looked interesting (slide 20, Fucci 2016, "An External Replication on the Effects of Test-driven Development Using a Multi-site Blind Analysis Approach"), so I looked it up.

Wilson describes that study as involving 39 professionals on real projects.

But when I looked at the actual study [1], it involved 21 graduate students working on two toy problems, the Bowling Game kata and "Mars Rover API." This is a disappointing misrepresentation of the study.

A charitable interpretation is that he was referring to a different study, and accidentally put the wrong reference on that slide. Or maybe the actual talk explains the difference. I'm not sure—I didn't look deeper.

[1] http://people.brunel.ac.uk/~csstmms/FucciEtAl_ESEM2016.pdf

Re: Software engineering research is a train wreck

#122

Post by madhadron »

jeddy3 wrote: post #75
ftio wrote: post #32

I worked on engineering productivity research and measurement at Google for two years until about a month ago. (Opinions my own.) Compared to my former colleagues, I'm an infant in this area, so take this with a heaping of salt. In general, I think the author's cynicism about productivity research is justified, but I think it could have been directed more productively. (NB: the following comments say nothing about ar…

I seems like many posters in this thread try to classify software enineering as either creative or "mindless factory-work". Where actual enineering disciplines has the risk of removing the creative part. I think this classification is wrong. There IS NO mindless factory-work. Just as in other enineering disciplines, our work is not manufacturing. It's just that the actual manufacturing does not exist (or rather is do…

> It's just that the actual manufacturing does not exist (or rather is done by compiler)

In good conditions, yes, the drudgery is all done by the compiler. There's still a terrifying number of cases where something hasn't been automated and still qualifies as factory work.

Re: Software engineering research is a train wreck

#123

Post by mym1990 »

lifeisstillgood wrote: post #79

My two cents: Software is a form of literacy not engineering. One good bridge looks and acts almost exactly like every other bridge in the world. Good software that does a given job can be unrecognisable compared to other good software that does exactly the same job. Software is more like a novel than a bridge. Maybe, maybe we can apply engineering terms to machine code or assembly. But the abstract levels we all wor…

But the risk is that bad software or a bad bridge can pose as good software or a good bridge until something catastrophic happens.

Software might be more like a novel if you are consuming the code/design, but to write a good novel one must be far more than just literate.

Re: Software engineering research is a train wreck

#124

Post by ska »

JimTheMan wrote: post #119
ska wrote: post #23

Earlier quoted context omitted.

> while no large team (that I'm aware of) has done anything similar. In areas like avionics, medical devices etc., there are formal production engineering methods applied to software as a matter of course. It can very work well, but it is definitely expensive compared to industry average.

I have my suspicion that medical devices aren't as rigidly programmed as the general public would like. Ie programmed by a biomedical/electrical engineer who isn't really a software specialist. But I'd be keen to hear from someone in the trenches.

The industry changes slowly, and definitely regulators like the FDA were too hardware focused for a long time. Less so in the last 10-20 years.

They've always been pretty good about critical systems but less so about "secondary" ones, and there are old devices out there where it shows.

These days most of the industry has reasonable guidance and standards to follow. Characterizing "the industry" is hard; it includes cloud based data handling products and embedded devices with minimal software, etc. - a huge range.

In both these areas (and others,to be sure), there is a formal engineering process that looks quite similar - and you need to embrace it in your software development or there will be a lot of friction. The goals of it are pretty sensible, but the approaches may seem odd to some with more mainstream software backgrounds.

Re: Software engineering research is a train wreck

#125

Post by anonymousDan »

mcguire wrote: post #95

" Here’s the only technique I’ve found that works, which I call scrobbling even though that means something totally different: " 1. Search seed terms you know, like “cost of bugs”, in an appropriate journal (here’s one). " 2. Find papers that look kinda relevant, skim their abstracts and conclusions. " 3. Make a list of all the papers that either cite or are cited by these papers and repeat. " 4. Find more useful ter…

If you're really in a hurry, filter out all those not published at top venues (e.g. ICSE or FSE as top tier for SW engineering, or ASE/ISSTA the tier below), or at the very least cited by several papers at top venues. In the vast majority of cases everything else is likely to be noise, incremental, or preliminary work. Of course consider the above venues as (perhaps) necessary but not necessarily sufficient condition for quality. You have to realise that academia is hyper competitive. The best work tends to get published at top venues because they are very difficult to get published at, and offer hiring and promotion committees an easy filter. Use that in your favour to filter out much of the dross and/or poorly executed ideas.

Re: Software engineering research is a train wreck

#126

Post by Rd6n6 »

moksly wrote: post #29
SebastianFish wrote: post #6

Testing different methods of development in terms of speed, cost and quality is really hard. The most convincing approach to me would be a single blind experiment to hire two software development teams and have them build to the same set of requirements in two different ways. But then it is hard to know whether you are really comparing the method of software development or the quality of the software teams. So two so…

It’s not that hard is it? There recently was a study on the value of testing and best practices linked here on HN, that I of course can’t find now, where the researchers looked a thousands of projects. Over all there was no scientific proof that testing and best practices lead to better results than just making spaghetti without a recipe. Having worked in an public enterprise organisation that buys a lot of different…

> Over all there was no scientific proof that testing and best practices lead to better results than just making spaghetti without a recipe.

Which best practices? I can’t see how a team without certain practices could be that effective. Eg, version control, having good backups, a good communication culture, code review, config management to prevent “it works in my machine” problems, and many other things.

For code review specifically, I read a paper years ago claiming a dramatic decrease in defect counts in companies who practice this

Re: Software engineering research is a train wreck

#128

Post by Rd6n6 »

hwayne wrote: post #96
NohatCoder wrote: post #17

Earlier quoted context omitted.

"A specification that can be implemented using formal methods". That is just source code. If the specification can completely define arbitrary programs it is necessarily Turing complete on its own, and as such prone to the same type of bugs as any other program.

Purely from an industrial perspective, interest in formal methods tends to split two ways: 1. Verifying really nasty algorithms, the kind you see in cryptography and embedded systems and stuff where the bugs are triggered by horrific race conditions or incredibly specific malicious inputs that even experts won't think of testing 2. High level specifications of requirements and abstract machines and stuff, where the s…

What is rust most commonly used for? If Somebody learned to program with it, what sorts of projects would they work on?

Re: Software engineering research is a train wreck

#129

Post by petra »

svachalek wrote: post #91
ftio wrote: post #45

Earlier quoted context omitted.

Oof. I have no idea. But since it'll be fun to guess: Absent some kind of magical (from today's POV) AGI, I think I'd probably say yes, but I'm not sure that means software engineering will continue to look the way it does today, at least not universally. My expectation is a pretty linear extrapolation of history. I think new tools and higher levels of abstraction will emerge that will make certain types of tasks unn…

That's what tools like Visual Basic and Delphi were meant to do, and actually were pretty good at for their time. As far as I can tell, we still haven't nearly caught up to the browser/cloud equivalent of these tools. Things need to sit still for a while longer I guess, but I think there's also some industry changes to blame. In the 90s there was big money in making tools and components for software engineers, while…

Look at low code tools. There's money, market demand, and many tool vendors.

Re: Software engineering research is a train wreck

#130

Post by frickinLasers »

titzer wrote: post #108
mcguire wrote: post #95

" Here’s the only technique I’ve found that works, which I call scrobbling even though that means something totally different: " 1. Search seed terms you know, like “cost of bugs”, in an appropriate journal (here’s one). " 2. Find papers that look kinda relevant, skim their abstracts and conclusions. " 3. Make a list of all the papers that either cite or are cited by these papers and repeat. " 4. Find more useful ter…

My impression on reading this list was, uh yeah, there's no avoiding that. Except if you know an expert. Then just ask an expert. Search engines for papers suck, bad. At the fringes of human knowledge, you need an actual brain to process cutting-edge research. Either your own non-expert brain, or an expert brain to help guide you. That process is the same in every field. Older fields have textbooks with knowledge dis…

| Search engines for papers suck, bad.

This screams the need for an intelligent search engine, with summarizing features that point out relevant parts of the paper, and (imho) a UI and algorithms that reach across disciplines and encourage discovery. So, for instance (I'm making this up), someone looking for a way to understand interferometry data might stumble on a useful regularization technique from image processing.

I imagine academic libraries would pay really good money for that. Of course, it would help if a mostly complete body of scientific literature was available to crawl--like SciHub.

Post reply on HN