Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Software engineering research is a train wreck

buttondown.email

Post reply on HN
Print view View original

71–80 of 174 posts

Re: Software engineering research is a train wreck

#71

Post by jeffreyrogers »

shusson wrote: post #46
jeffreyrogers wrote: post #12

I was reading this thesis the other day[0], which is on precision machine design. It got me comparing precision machine design to software engineering. A big part of why we're able to design extremely precise machines (the author worked on a lathe used for machining optical parts for the National Ignition Facility) is because we can characterize exactly where errors will come from (e.g. structure isn't rigid enough,…

> It got me comparing precision machine design to software engineering. I think the problem is that the failure of physical systems are often caused by random failures, which can be tested for and probabilities determined e.g how long before a bolt breaks. This as opposed to software, which most often fail due to systematic errors and cannot be tested completely.

In precision machine design the failures are typically not statistical (like in fatigue failures) but due to errors of design. E.g. the tool spindle can maintain the desired tolerance when running from a cold start but after several minutes thermal effects cause it to be out of tolerance.

Re: Software engineering research is a train wreck

#72

Post by nradov »

AnimalMuppet wrote: post #31
nradov wrote: post #11

Earlier quoted context omitted.

Requirements analysis is an integral part of the development process. Giving teams fully specified requirements at the beginning of the experiment wouldn't be realistic.

Depends. You could then run experiments on team performance in the requirements-to-code phase, separate from the generating-requirements phase. That has its place. And then you could experiment with teams trying to convert informal requirements to formal requirements. That might let you learn some things about the parts that you couldn't learn if you dealt with the whole.

Those are not separate phases, they take place simultaneously. No one does waterfall development anymore.

Re: Software engineering research is a train wreck

#73

Post by foobiekr »

IshKebab wrote: post #20

This tallies with my experience. It isn't early bugs that are expensive to fix. It's early design mistakes. For example choosing the wrong language, framework or architecture. Dropbox using Python is a good example. Or Python's GIL.

The GIL is a good example because it removed optionality from Python by having its semantics leak out. Using a poorly performing tool is a different problem entirely.

Most of the really horrible errors in SW design involve choices where the semantics above will be visible and encoded as a subtle (or not so subtle) dependency that cannot later be fixed without an infeasible level of effort. I deal with these regularly and they are the true original sin in most sizable software projects.

Re: Software engineering research is a train wreck

#74

Post by jeffreyrogers »

snidane wrote: post #62
jeffreyrogers wrote: post #12

I was reading this thesis the other day[0], which is on precision machine design. It got me comparing precision machine design to software engineering. A big part of why we're able to design extremely precise machines (the author worked on a lathe used for machining optical parts for the National Ignition Facility) is because we can characterize exactly where errors will come from (e.g. structure isn't rigid enough,…

Software errors which are traceable usually get dealt with with ease. The remaining majority are unknown and governed by power law - eg. a bug uncovers a major redesign, but there is no time for rewrites so we have to roll along with it, complicating the design until next big rewrite if it ever happens. Power law phenomena are quantitatively untraceable. Averages mean nothing. Ergo software engineering is largely unt…

This sort of thinking precludes the possibility of ever developing better methods though. And this statistical argument was actually quite common in machine design until people started actually building machines that were more precise than many believed possible. The thesis I linked has examples of arguments analogous to what you're saying for why precision machine design is impossible.

Edit: I don't mean to be too harsh, you could be right, but software engineering is a young discipline. Mechanical engineering is much older and it was only recently that precision machine design became possible (and, just as importantly, teachable). I believe software engineering has the potential for a similar change in how it is done, at least for projects that warrant it.

Re: Software engineering research is a train wreck

#75

Post by jeddy3 »

ftio wrote: post #32

I worked on engineering productivity research and measurement at Google for two years until about a month ago. (Opinions my own.) Compared to my former colleagues, I'm an infant in this area, so take this with a heaping of salt. In general, I think the author's cynicism about productivity research is justified, but I think it could have been directed more productively. (NB: the following comments say nothing about ar…

I seems like many posters in this thread try to classify software enineering as either creative or "mindless factory-work".

Where actual enineering disciplines has the risk of removing the creative part.

I think this classification is wrong. There IS NO mindless factory-work.

Just as in other enineering disciplines, our work is not manufacturing. It's just that the actual manufacturing does not exist (or rather is done by compiler)

Software enineering can (just like other enineering) be:

  - more scientific
  - more pragmatic
  - helped by formal methods
without removing the creativity.

Just as "other" enineering (like software):

  - is highly creative
  - can be artisanal (if wanted, rarely in all projects)
I REALLY feel we can mature in SWE without being afraid of losing creativity.

Re: Software engineering research is a train wreck

#76

Post by NohatCoder »

SebastianFish wrote: post #28
NohatCoder wrote: post #17

Earlier quoted context omitted.

"A specification that can be implemented using formal methods". That is just source code. If the specification can completely define arbitrary programs it is necessarily Turing complete on its own, and as such prone to the same type of bugs as any other program.

I think you are speaking to one of the core tensions in formal methods. The difference between a specification and an implementation can get blurry. Where formal methods get interesting is statically proving properties about the specification. Take a simple example of a sorting algorithm. The two most commonly proved properties of these algorithms are that they 1) return a permutation of the input list (no items remo…

Sorting is one of the more favorable tasks for being specified this way, for much code there is no simpler way to verify the output than running the same or equivalent code again.

If your specification language is not Turing complete then there is simply stuff you cannot specify. Of course, just because it isn't Turing complete doesn't mean it isn't perfectly adequate for writing bugs.

Re: Software engineering research is a train wreck

#77

Post by IshKebab »

klysm wrote: post #49
IshKebab wrote: post #20

This tallies with my experience. It isn't early bugs that are expensive to fix. It's early design mistakes. For example choosing the wrong language, framework or architecture. Dropbox using Python is a good example. Or Python's GIL.

They aren’t really exclusive at all. Early design mistakes can also be really expensive, but a really scary bug that nukes all your data could also suck.

They mean the cost of fixing the defect, not the severity of bugs.

Re: Software engineering research is a train wreck

#78

Post by hwayne »

star-trek-fleet wrote: post #26
jeffreyrogers wrote: post #12

I was reading this thesis the other day[0], which is on precision machine design. It got me comparing precision machine design to software engineering. A big part of why we're able to design extremely precise machines (the author worked on a lathe used for machining optical parts for the National Ignition Facility) is because we can characterize exactly where errors will come from (e.g. structure isn't rigid enough,…

I happened to also delved into machining tools and work as SWE full time. This comparison to me, is not workable between machine building and software. So the difference is that machines do simple things and not changing. But software do complicated things and always changing. You can easily understand which parts and/or ways of operating that cause machine misbehaving. Because you have full grasp of the parts and ho…

One of my favorite online discoveries is Fastenal labs, where they talk about properties of screws. Here's one that says your machine can break because you put a stainless steel screw through an aluminum plate: https://www.fastenal.com/en/70/corrosion

Re: Software engineering research is a train wreck

#79

Post by lifeisstillgood »

My two cents: Software is a form of literacy not engineering. One good bridge looks and acts almost exactly like every other bridge in the world. Good software that does a given job can be unrecognisable compared to other good software that does exactly the same job.

Software is more like a novel than a bridge.

Maybe, maybe we can apply engineering terms to machine code or assembly. But the abstract levels we all work at - not a hope.

I mean this is a good thing. It changes how we think about software - engineering has changed the world - and it's in the hands of a few professionals. Imagine software in the hands of ... everyone. It's like Eric Idle gather mud in the Dark Ages imagining what a literate world that reads the New york times will be like

Re: Software engineering research is a train wreck

#80

Post by hwayne »

jeddy3 wrote: post #75
ftio wrote: post #32

I worked on engineering productivity research and measurement at Google for two years until about a month ago. (Opinions my own.) Compared to my former colleagues, I'm an infant in this area, so take this with a heaping of salt. In general, I think the author's cynicism about productivity research is justified, but I think it could have been directed more productively. (NB: the following comments say nothing about ar…

I seems like many posters in this thread try to classify software enineering as either creative or "mindless factory-work". Where actual enineering disciplines has the risk of removing the creative part. I think this classification is wrong. There IS NO mindless factory-work. Just as in other enineering disciplines, our work is not manufacturing. It's just that the actual manufacturing does not exist (or rather is do…

Agreed; a while back I interviewed of ex-trad, now-software engineers and found out that

    1. Engineering is a lot more personal and creative than we think
    2. A large amount of software development is very similar to trad engineering
    3. Never walk over a bridge.
Post reply on HN