Live data from Hacker News

Software engineering research is a train wreck

buttondown.email

11–20 of 174 posts

Re: Software engineering research is a train wreck

#11

Testing different methods of development in terms of speed, cost and quality is really hard. The most convincing approach to me would be a single blind experiment to hire two software development teams and have them build to the same set of requirements in two different ways. But then it is hard to know whether you are really comparing the method of software development or the quality of the software teams. So two so…

Requirements analysis is an integral part of the development process. Giving teams fully specified requirements at the beginning of the experiment wouldn't be realistic.

Re: Software engineering research is a train wreck

#12
I was reading this thesis the other day[0], which is on precision machine design. It got me comparing precision machine design to software engineering.

A big part of why we're able to design extremely precise machines (the author worked on a lathe used for machining optical parts for the National Ignition Facility) is because we can characterize exactly where errors will come from (e.g. structure isn't rigid enough, temperature variation causes parts to expand by different amounts, parts not round or flat enough, etc.). Once we know what errors we need to control and their rough order of importance we can start improving the machine design to control them better.

In theory, something similar could be done in software engineering (formal methods are part of this, but not a full solution). Rather than an error budget, you'd have some sort of bug budget, where you tracked what sort of bugs were caused by what sort of constructs, and design your program in such a way to minimize their chance of being introduced. I've never heard of anyone except Dan Bernstein[1] actually doing anything approximating this. Probably because the perceived level of effort is too high.

I actually don't think it would take that much effort, but it would require quite a bit of organization to track where bugs are introduced and what their root causes are. This is probably why Bernstein, an individual, is able to do this, while no large team (that I'm aware of) has done anything similar.

Of course, just like your toaster doesn't need to use precision machine design techniques (an over-engineered toaster is a bad toaster), most software doesn't need the effort/expense of a rigorous design process either, but some would benefit from it.

[0]: https://dspace.mit.edu/handle/1721.1/9414

[1]: https://cr.yp.to/djb.html

Re: Software engineering research is a train wreck

#13
post #4

Earlier quoted context omitted.

There's another strategy that works better than searching for a keyword, which is to read the article. It's about the difficulty/expense of systematically and scientifically studying the software development process (e.g. finding out what data actually exists to support the claim that "a bug is cheaper to squash when it is caught during the specification process"). The title expresses frustration not at science itsel…

But specifically with respect to software engineering, it's really hard to do solid, repeatable experiments. It's not like physics, it's like sociology or psychology - which also have trouble doing precise, repeatable science. Why is it like sociology or psychology? Because it's about people , not about things . Software engineering is not just about languages and programming techniques; it's about how people interac…

I think at the present time, software can be more like science than sociology.

Both suffer from abstraction.

I think psychology when closer to biology is similar to programming like running the programming language C experiments.

Re: Software engineering research is a train wreck

#17

Testing different methods of development in terms of speed, cost and quality is really hard. The most convincing approach to me would be a single blind experiment to hire two software development teams and have them build to the same set of requirements in two different ways. But then it is hard to know whether you are really comparing the method of software development or the quality of the software teams. So two so…

"A specification that can be implemented using formal methods". That is just source code. If the specification can completely define arbitrary programs it is necessarily Turing complete on its own, and as such prone to the same type of bugs as any other program.

Re: Software engineering research is a train wreck

#19
post #18

I don't know what "FM" and other initializations are. Please when writing an article, expand them in parenthesis the first time they are mentioned.

It’s expanded the first time it’s used (in the sentence preceding the one with the abbreviation).
Post reply on HN