Live data from Hacker News

Software engineering research is a train wreck

buttondown.email

51–60 of 174 posts

Re: Software engineering research is a train wreck

#51
post #27

Wow, thanks for the link to sci-hub.st - this is awesome! I can finally access Elsevier's "walled" content again.

https://whereisscihub.now.sh/ Currently redirects to: https://whereisscihub.vercel.app/

Oh hey, I made that, but it was taken down by what was then called Zeit. Looks like that didn't persist through their Vercel rename.

Note that it just fetches the current URL from Wikidata.

Re: Software engineering research is a train wreck

#52
post #32

I worked on engineering productivity research and measurement at Google for two years until about a month ago. (Opinions my own.) Compared to my former colleagues, I'm an infant in this area, so take this with a heaping of salt. In general, I think the author's cynicism about productivity research is justified, but I think it could have been directed more productively. (NB: the following comments say nothing about ar…

>> I'm highly skeptical of attempts to quantify the precise relationship between error discovery stage and cost in a way that is generalizable...

I would say universally that bugs found prior to shipping are lower cost (not just cost to fix) than those found after. I've heard from an auto industry friend that over-the-air update capabilities are becoming mandatory for more components. That sounds good because critical fixes can be issued without the cost of a recall (very expensive - if you're a tier 2 supplier you may be out of business). The down side is that the software teams are starting to think they don't actually need to be "done" by launch day, which leads to a bunch of harder to quantify costs.

Another example from a different industry: a bug that caused file import compatibility issues between two versions of the same software. Had that been caught before shipping it would have been no problem to fix. But instead the fix involved trapping an error and rereading the file with a different code path. Also, we didn't realize the change mattered so the file header info was not updated, so we couldn't tell in advance if a file was "old" or "new" when reading it. Once files of both "versions" were in the wild with our customers the simplest (most correct) fix was not possible without braking compatibility.

Re: Software engineering research is a train wreck

#53

Earlier quoted context omitted.

I happened to also delved into machining tools and work as SWE full time. This comparison to me, is not workable between machine building and software. So the difference is that machines do simple things and not changing. But software do complicated things and always changing. You can easily understand which parts and/or ways of operating that cause machine misbehaving. Because you have full grasp of the parts and ho…

> For example, no one cares about screws, as long as they are machined according to spec, and torqued correctly during installation. The chance that a random screw to bring down a whole machine is close to zero. This sounds like people do care about screws. You identify two specific failure modes: they could be machined incorrectly, or they could be torqued incorrectly during installation. In either case, it is entir…

Depending on your application, you learn to care very, very deeply about screws and bolts. Screw and bolt failures kill people, and they usually fail in conjunction with other design mistakes. Like software, it's not just one thing and you do need to sweat the details.

Re: Software engineering research is a train wreck

#54
post #37

Earlier quoted context omitted.

> Commercial software engineering is a creative endeavor; it is not a science, nor is it a manufacturing process. Do you think software engineering will remain a creative endeavour in the future? We've only been programming in high level languages for 60-70 years. All the while Moore's law has been in effect.

I think doing the engineering design of structures and machines (buildings, bridges, engines, etc.) is still a creative endeavor after many hundreds of years, and I think this is the closest analogue to what software engineers do. The difference is that once we have the blueprints drawn (ie. the code written), a computer can just execute them rather than requiring a lot more material and labor inputs. Program executi…

I agree with this, but those things have also moved from essentially being artisanal to being more formalized and scientific. I don't think we're at that point in software engineering.

Re: Software engineering research is a train wreck

#55
post #42

> ...“bugs found in requirements are 100x cheaper than bugs found in implementations.” ... There’s one tiny problem with the IBM Systems Sciences Institute study: it doesn’t exist It's funny to think how much of an effect this chart had on software engineering as a whole. I remember learning it at university and until today I thought it had some basis in science.

It "feels right" and is therefore "truthy."

Re: Software engineering research is a train wreck

#56

Earlier quoted context omitted.

"A specification that can be implemented using formal methods". That is just source code. If the specification can completely define arbitrary programs it is necessarily Turing complete on its own, and as such prone to the same type of bugs as any other program.

As soon as you need to interface with arbitrary external components, you see the value in good specifications. If JPEG-2000 was just a reference implementation and not a spec, that would work fine if only one team ever had to develop a JPEG library, and every application that read or wrote JPEG files used exactly that one library. Since that isn't the case, having something sit at a higher level of abstraction than t…

I understand your last point, but my question is:

In order for domain experts to contribute to a formal specification, this specification must be, well, formal, and also serve as a very precise shared language among all domain experts and implementors (i.e. the people who are going to read the spec and build something out of it). Once you go down this road, the specification language becomes as complex as any programming language -- or maybe even more! -- and must be learned by all involved, just like any given programming language. Some people will find it easier to learn, some will struggle or find it bizarrely unfamiliar -- again, just like any given programming language. Any sufficiently expressive specification language will also be subject to the kinds of bugs and complexity that affect programming languages.

So my question is: isn't learning a shared formal specification language more or less as difficult as learning an unfamiliar programming language?

Re: Software engineering research is a train wreck

#58
post #54

Earlier quoted context omitted.

I think doing the engineering design of structures and machines (buildings, bridges, engines, etc.) is still a creative endeavor after many hundreds of years, and I think this is the closest analogue to what software engineers do. The difference is that once we have the blueprints drawn (ie. the code written), a computer can just execute them rather than requiring a lot more material and labor inputs. Program executi…

I agree with this, but those things have also moved from essentially being artisanal to being more formalized and scientific. I don't think we're at that point in software engineering.

We're definitely earlier in the timeline with software engineering, but I personally think it seems like structural and mechanical engineering designs are a lot more "artisanal" than is the general conception. Certainly the primitives involved are very well understand scientifically (and this is where we're definitely a long way behind them), but the specific combinations of those primitives strike me as remaining very bespoke and creative, as does the process for creating the designs. All of this looks familiar to me, just further down the maturity timeline.

Re: Software engineering research is a train wreck

#60

Earlier quoted context omitted.

> For example, no one cares about screws, as long as they are machined according to spec, and torqued correctly during installation. The chance that a random screw to bring down a whole machine is close to zero. This sounds like people do care about screws. You identify two specific failure modes: they could be machined incorrectly, or they could be torqued incorrectly during installation. In either case, it is entir…

Depending on your application, you learn to care very, very deeply about screws and bolts. Screw and bolt failures kill people, and they usually fail in conjunction with other design mistakes. Like software, it's not just one thing and you do need to sweat the details.

Yep. I think we have a tendency to assume that our field is somehow unique, because we know it so intimately. "Everyone else has it simpler."

The actual fact is that everyone else has it just as complicated, but we never see that complexity because it's not what we are experts in.

Post reply on HN