Software engineering research is a train wreck • Hacker News Board

Re: Software engineering research is a train wreck

#141

Post by cardosof » Wed, Jul 21, 2021, 1:36 AM UTC

ctvo wrote: post #133

Hermitian909 wrote: post #132

Earlier quoted context omitted.

The poor ontology backing the term software engineer has always seemed to me like the primary culprit for this problem. I've often heard people doing any of the following for their day job referred to as software engineers: - people who write basic static sties - people who write wordpress sites - people who write simple CRUD apps - people who write hardware drivers - people who write compilers - people who write dat…

Do you think other engineers think this way? Your bridge is only 20 yards over a creek, you're not a civil engineer? I'm sure the complexity, depth, and expected maintenance of some bridges vary too.

This also applies to other fields - medicine for instance.

Re: Software engineering research is a train wreck

#142

Post by joppy » Wed, Jul 21, 2021, 1:41 AM UTC

mcguire wrote: post #95

" Here’s the only technique I’ve found that works, which I call scrobbling even though that means something totally different: " 1. Search seed terms you know, like “cost of bugs”, in an appropriate journal (here’s one). " 2. Find papers that look kinda relevant, skim their abstracts and conclusions. " 3. Make a list of all the papers that either cite or are cited by these papers and repeat. " 4. Find more useful ter…

Yeah I was going to say, this sounds like a normal way of getting around the literature. We were taught to start mostly with review/summary papers from high-quality places, and work our way out from there.

Re: Software engineering research is a train wreck

#143

Post by kreelman » Wed, Jul 21, 2021, 2:23 AM UTC

dalbasal wrote: post #140

ftio wrote: post #32

I worked on engineering productivity research and measurement at Google for two years until about a month ago. (Opinions my own.) Compared to my former colleagues, I'm an infant in this area, so take this with a heaping of salt. In general, I think the author's cynicism about productivity research is justified, but I think it could have been directed more productively. (NB: the following comments say nothing about ar…

>> I do not believe in the possibility of a "General Theory of Productivity." Important frame. I think any effort that doesn't start with this statement is probably headed for frustration. I reckon this is true also for education, aspects of economics and many other fields. Besides sloppy and/or cynical work, I believe a lot of the "replication crisis" relates to this. Do we really expect that a relationship between…

I like what you've said here. I'm going to think more about it. Some quite important points. There are parts of dealing with people that can't be science based. I would find it hard to disagree with that. We do try to use science in this realm quite a lot though....

...I think this conversation around this item is a good example of how sometimes the surrounding conversation is more valuable than the original topic.

Re: Software engineering research is a train wreck

#144

Post by derekbaker783 » Wed, Jul 21, 2021, 2:51 AM UTC

mattchamb wrote: post #107

nradov wrote: post #72

Earlier quoted context omitted.

Those are not separate phases, they take place simultaneously. No one does waterfall development anymore.

Waterfall development is still pervasive in a lot of industries and companies.

Definitely still out there.

Re: Software engineering research is a train wreck

#145

Post by Ostrogodsky » Wed, Jul 21, 2021, 2:55 AM UTC

ftio wrote: post #32

I worked on engineering productivity research and measurement at Google for two years until about a month ago. (Opinions my own.) Compared to my former colleagues, I'm an infant in this area, so take this with a heaping of salt. In general, I think the author's cynicism about productivity research is justified, but I think it could have been directed more productively. (NB: the following comments say nothing about ar…

> Commercial software engineering is a creative endeavor; it is not a science,

And do you think Science it is not a creative endeavor? I know that you are coming from the "art/science dichotomy" where the terms are used metaphorically, to me it is an useless distinction still.

> nor is it a manufacturing process?

Why not? Software production IS a manufacturing process.

Re: Software engineering research is a train wreck

#146

Post by feoren » Wed, Jul 21, 2021, 2:58 AM UTC

bob1029 wrote: post #138

Software engineering can only exist with an incredible amount of discipline. We routinely start new implementations in excel and perform formal normalization checks over models before writing any code. You can get away without having a strong type system as long as your tables and relations are clean. This stuff is so powerful. Figuring out how to funnel your domain instances into a SQL db and writing all your comple…

Software engineering is so weird -- what you've just described sounds awful to me. I believe almost the exact opposite of everything you've said. Yet I have no doubt* you've seen good results from your approach, and I get good results from mine. It's surprising how personal software engineering really is.

* I do actually have doubts, because 99.9% of software is utter garbage. But I'm trying to be more positive and the point stands whether you're in the 99.9% or the 0.1%.

Re: Software engineering research is a train wreck

#147

Post by carlosf » Wed, Jul 21, 2021, 3:01 AM UTC

dalbasal wrote: post #140

ftio wrote: post #32

I worked on engineering productivity research and measurement at Google for two years until about a month ago. (Opinions my own.) Compared to my former colleagues, I'm an infant in this area, so take this with a heaping of salt. In general, I think the author's cynicism about productivity research is justified, but I think it could have been directed more productively. (NB: the following comments say nothing about ar…

>> I do not believe in the possibility of a "General Theory of Productivity." Important frame. I think any effort that doesn't start with this statement is probably headed for frustration. I reckon this is true also for education, aspects of economics and many other fields. Besides sloppy and/or cynical work, I believe a lot of the "replication crisis" relates to this. Do we really expect that a relationship between…

Well written!

Re: Software engineering research is a train wreck

#148

Post by Hermitian909 » Wed, Jul 21, 2021, 3:17 AM UTC

ctvo wrote: post #133

Hermitian909 wrote: post #132

Earlier quoted context omitted.

The poor ontology backing the term software engineer has always seemed to me like the primary culprit for this problem. I've often heard people doing any of the following for their day job referred to as software engineers: - people who write basic static sties - people who write wordpress sites - people who write simple CRUD apps - people who write hardware drivers - people who write compilers - people who write dat…

Do you think other engineers think this way? Your bridge is only 20 yards over a creek, you're not a civil engineer? I'm sure the complexity, depth, and expected maintenance of some bridges vary too.

Based on civil engineers I know I think it's more likely they'd simply consider the engineer making 20 yard bridges more junior, but doing similar work. As I understand it there aren't the same difference in kind that I'm describing here.

Re: Software engineering research is a train wreck

#149

Post by jodrellblank » Wed, Jul 21, 2021, 3:44 AM UTC

jeffreyrogers wrote: post #12

I was reading this thesis the other day[0], which is on precision machine design. It got me comparing precision machine design to software engineering. A big part of why we're able to design extremely precise machines (the author worked on a lathe used for machining optical parts for the National Ignition Facility) is because we can characterize exactly where errors will come from (e.g. structure isn't rigid enough,…

> "where you tracked what sort of bugs were caused by what sort of constructs, and design your program in such a way to minimize their chance of being introduced. I've never heard of anyone except Dan Bernstein actually doing anything approximating this"

Not sure if I completely understand but here[1] is a talk by Jules May called "If considered harmful: How to eradicate 95% of all your bugs in one simple step" where he discusses that they analyzed what was causing a lot of bugs in their programs, and found it was code which keeps state synchronised using `if` in far distant places in the codebase, many places with "if (stateFlag) {}" which then gradually fell out of sync with each other as people changed some but not all of them.

By redesigning the programs so that instead of call points dotted around the codebase all saying `if (condition) { function1() } else { function2() }` they all say `obj.doWork()` any the condition checking code only exists in one place inside the class definition, the talk title claims they removed 95% of bugs in their codebase. Inside the talk he spends all the time explaining the idea and doesn't actually talk about the effect it had on their bug tracker database, bugs encountered, etc. which is a bit of a shame.

Of course, the comments are suitably sneery for a programming talk: "having double checking being your main source of bugs means that you really try your best to write bad code. The whole talk boils down to: double checking conditions is bad and we found it 20 years after everybody else." - even if you agree with that, the analysis of their bug tracker to find what was causing most bugs and redesigning how they code based on that, appears to be an example of what you describe.

[1] https://www.youtube.com/watch?v=z43bmaMwagI

Re: Software engineering research is a train wreck

#150

Post by routerl » Wed, Jul 21, 2021, 4:27 AM UTC

deregulateMed wrote: post #10

routerl wrote: post #4

Earlier quoted context omitted.

There's another strategy that works better than searching for a keyword, which is to read the article. It's about the difficulty/expense of systematically and scientifically studying the software development process (e.g. finding out what data actually exists to support the claim that "a bug is cheaper to squash when it is caught during the specification process"). The title expresses frustration not at science itsel…

Despite insistence of programmers wanting to be scientists and engineers, it's not true to the definition. Maybe safety critical C or assembly is engineering. Maybe testing code for speed can be considered science. By the time any useful program is finished, there's significantly more tradition and art than Science and engineering.

Sure, but I think the underlying instinct behind "evidence based software engineering" is precisely about turning those "maybes" into measurements.

In principle, I'm in favor of this endeavor, and the article at hand expresses frustration about apparently arbitrary barriers towards accomplishing these measurements.

I don't know about you, but I'm generally always in favor of "let's measure more things and build more models from that data, to make sure we're actually improving"