Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Software engineering research is a train wreck

buttondown.email

Post reply on HN
Print view View original

61–70 of 174 posts

Re: Software engineering research is a train wreck

#61

Post by narag »

shusson wrote: post #37
ftio wrote: post #32

I worked on engineering productivity research and measurement at Google for two years until about a month ago. (Opinions my own.) Compared to my former colleagues, I'm an infant in this area, so take this with a heaping of salt. In general, I think the author's cynicism about productivity research is justified, but I think it could have been directed more productively. (NB: the following comments say nothing about ar…

> Commercial software engineering is a creative endeavor; it is not a science, nor is it a manufacturing process. Do you think software engineering will remain a creative endeavour in the future? We've only been programming in high level languages for 60-70 years. All the while Moore's law has been in effect.

My two cents: is that really the problem? I don't think so.

I mean that the non-creative part is the problem. Why would anybody want to eliminate the creative part and leave the mindless factory-like part?

And there is a huge non-creative, tedious work in programming. A lot of workarounds, minutiae, leaky abstractions, kludges and plain simple idiocy that doesn't get removed because "it's too much work".

Re: Software engineering research is a train wreck

#62

Post by snidane »

jeffreyrogers wrote: post #12

I was reading this thesis the other day[0], which is on precision machine design. It got me comparing precision machine design to software engineering. A big part of why we're able to design extremely precise machines (the author worked on a lathe used for machining optical parts for the National Ignition Facility) is because we can characterize exactly where errors will come from (e.g. structure isn't rigid enough,…

Software errors which are traceable usually get dealt with with ease. The remaining majority are unknown and governed by power law - eg. a bug uncovers a major redesign, but there is no time for rewrites so we have to roll along with it, complicating the design until next big rewrite if it ever happens. Power law phenomena are quantitatively untraceable. Averages mean nothing.

Ergo software engineering is largely untraceable and better treated as a discovery or research process. To make things efficient, focus on removing impediments and continual simplification, but don't expect to predict much in advance. Unless it is an instance of a previous task, in which case why isn't there a function/library/framework for it yet?

Re: Software engineering research is a train wreck

#63

Post by WalterBright »

phkahler wrote: post #52
ftio wrote: post #32

I worked on engineering productivity research and measurement at Google for two years until about a month ago. (Opinions my own.) Compared to my former colleagues, I'm an infant in this area, so take this with a heaping of salt. In general, I think the author's cynicism about productivity research is justified, but I think it could have been directed more productively. (NB: the following comments say nothing about ar…

>> I'm highly skeptical of attempts to quantify the precise relationship between error discovery stage and cost in a way that is generalizable... I would say universally that bugs found prior to shipping are lower cost (not just cost to fix) than those found after. I've heard from an auto industry friend that over-the-air update capabilities are becoming mandatory for more components. That sounds good because critica…

> over-the-air update capabilities are becoming mandatory

Looking forward to the mass injection of malware into cars exploiting the usual bugs. How about ransomware to get your car started? Lots of fun!

> the cost of a recall

Mail me a USB stick with the update.

Re: Software engineering research is a train wreck

#64

Post by justanotherguy0 »

spaetzleesser wrote: post #40
shusson wrote: post #37

Earlier quoted context omitted.

> Commercial software engineering is a creative endeavor; it is not a science, nor is it a manufacturing process. Do you think software engineering will remain a creative endeavour in the future? We've only been programming in high level languages for 60-70 years. All the while Moore's law has been in effect.

“Do you think software engineering will remain a creative endeavour in the future?” I would think so. If something can be automated or be done with a reproducible process it will get automated or performed by lower paid people. But I can easily see how software and automated processes will reach a point of “good enough” for most purposes sometime in the future and there will be much less software engineers doing crea…

I would expect the opposite. As the mechanical stuff becomes cheaper and more automated, the creative work decreases in proportion but increases in absolute terms.

Re: Software engineering research is a train wreck

#65

Post by blunte »

> But first you gotta find the primary sources.

Yeah, and the real primary sources are the companies building and running software. They don't have time or interest in the long term research. And likely they aren't even concerned with their own technical situation 12 months in advance. They just need to "deliver", they think.

So unless you can find some clever way to gather all their data (Sentry, AppSignal, ...?), you will never know what's really happening in most places. Your research will be artificial without that real world data.

Re: Software engineering research is a train wreck

#66

Post by jkaptur »

nonameiguess wrote: post #43
NohatCoder wrote: post #17

Earlier quoted context omitted.

"A specification that can be implemented using formal methods". That is just source code. If the specification can completely define arbitrary programs it is necessarily Turing complete on its own, and as such prone to the same type of bugs as any other program.

As soon as you need to interface with arbitrary external components, you see the value in good specifications. If JPEG-2000 was just a reference implementation and not a spec, that would work fine if only one team ever had to develop a JPEG library, and every application that read or wrote JPEG files used exactly that one library. Since that isn't the case, having something sit at a higher level of abstraction than t…

Is there a machine-readable spec for JPEGs? Did it find interesting bugs and oversights?

Re: Software engineering research is a train wreck

#67

Post by Karrot_Kream »

ftio wrote: post #32

I worked on engineering productivity research and measurement at Google for two years until about a month ago. (Opinions my own.) Compared to my former colleagues, I'm an infant in this area, so take this with a heaping of salt. In general, I think the author's cynicism about productivity research is justified, but I think it could have been directed more productively. (NB: the following comments say nothing about ar…

> do not believe in the possibility of a "General Theory of Productivity." I'm highly skeptical of attempts to quantify the precise relationship between error discovery stage and cost in a way that is generalizable, although I think it might be possible given a large group of engineers using a highly homogenous process, tools, and accounting. Google is pretty close to this (common dev infrastructure across tens of thousands of engineers), and even across Google this kind of generalization would be extremely difficult.

I don't think you are incorrect, but I think a lot of the aspirants behind ESE just want to have a better sense of what works and what doesn't; I'd even welcome negative results! The current state of things is to read 100 opinionated people and their blog posts. And given enough time, you'll encounter someone who swears that after drinking their morning coffee and jumping on one foot for 1 min, they enter a VRChat standup with their team and hit max flow. There's just so little knowledge right now about what works and what doesn't that I'd welcome more clarity, especially negative results.

> As a result, academic research into productivity can be difficult to generalize

I think defects are what we should measure for, not productivity because of the subjectivity of measuring productivity. But even measuring defects is complicated. The best way I see to measure defects is to ask a Team Under Test to document bugs that they encounter along with resolution times, but this is not only expensive, but something I doubt most corporations will be willing to share outside of their walls. Perhaps open source projects can try to store this data, like curl's stats [1].

[1]: https://github.com/curl/stats

Re: Software engineering research is a train wreck

#68

Post by concernedtroll »

WalterBright wrote: post #63
phkahler wrote: post #52

Earlier quoted context omitted.

>> I'm highly skeptical of attempts to quantify the precise relationship between error discovery stage and cost in a way that is generalizable... I would say universally that bugs found prior to shipping are lower cost (not just cost to fix) than those found after. I've heard from an auto industry friend that over-the-air update capabilities are becoming mandatory for more components. That sounds good because critica…

> over-the-air update capabilities are becoming mandatory Looking forward to the mass injection of malware into cars exploiting the usual bugs. How about ransomware to get your car started? Lots of fun! > the cost of a recall Mail me a USB stick with the update.

But think of how easy OTA updates will make apprehending criminals!

For instance, if a government needs to institute a lockdown to prevent spread of a novel virus, they can just disable all affected citizens' ability to drive anywhere inessential, and the original software doesn't have to support it

Re: Software engineering research is a train wreck

#69

Post by jamincan »

star-trek-fleet wrote: post #26
jeffreyrogers wrote: post #12

I was reading this thesis the other day[0], which is on precision machine design. It got me comparing precision machine design to software engineering. A big part of why we're able to design extremely precise machines (the author worked on a lathe used for machining optical parts for the National Ignition Facility) is because we can characterize exactly where errors will come from (e.g. structure isn't rigid enough,…

I happened to also delved into machining tools and work as SWE full time. This comparison to me, is not workable between machine building and software. So the difference is that machines do simple things and not changing. But software do complicated things and always changing. You can easily understand which parts and/or ways of operating that cause machine misbehaving. Because you have full grasp of the parts and ho…

A random screw can have a huge difference. Just pull up McMaster-Carr and look up the hundreds of different types of fasteners that all have different properties that make them suitable in some cases or others. Someone has to choose the screw, and it's not necessarily a straightforward choice.

Re: Software engineering research is a train wreck

#70

Post by musicale »

ftio wrote: post #32

I worked on engineering productivity research and measurement at Google for two years until about a month ago. (Opinions my own.) Compared to my former colleagues, I'm an infant in this area, so take this with a heaping of salt. In general, I think the author's cynicism about productivity research is justified, but I think it could have been directed more productively. (NB: the following comments say nothing about ar…

> Commercial software engineering is a creative endeavor; it is not a science, nor is it a manufacturing process

Exactly. And it's less like movie production and more like 4,000 people trying to collaborate to produce a million-page novel.

It does bear some resemblance to design and engineering, but with custom materials and components that have never been used before and need to be created specifically for the project.

Post reply on HN