Software engineering research is a train wreck • Hacker News Board

Re: Software engineering research is a train wreck

#131

Post by zmmmmm » Tue, Jul 20, 2021, 11:28 PM UTC

JimTheMan wrote: post #119

ska wrote: post #23

Earlier quoted context omitted.

> while no large team (that I'm aware of) has done anything similar. In areas like avionics, medical devices etc., there are formal production engineering methods applied to software as a matter of course. It can very work well, but it is definitely expensive compared to industry average.

I have my suspicion that medical devices aren't as rigidly programmed as the general public would like. Ie programmed by a biomedical/electrical engineer who isn't really a software specialist. But I'd be keen to hear from someone in the trenches.

It's a different version of rigorous to what a normal software industry interpretation would be.

It's about controlled processes, highly auditable, very sharply optimised focused managing the medical (and medico-legal) risk at the expense of every other type of problem.

Generic software specialists I think actually aren't very good at it and take a while to acclimatise to it.

Re: Software engineering research is a train wreck

#132

Post by Hermitian909 » Tue, Jul 20, 2021, 11:39 PM UTC

ftio wrote: post #32

I worked on engineering productivity research and measurement at Google for two years until about a month ago. (Opinions my own.) Compared to my former colleagues, I'm an infant in this area, so take this with a heaping of salt. In general, I think the author's cynicism about productivity research is justified, but I think it could have been directed more productively. (NB: the following comments say nothing about ar…

The poor ontology backing the term software engineer has always seemed to me like the primary culprit for this problem. I've often heard people doing any of the following for their day job referred to as software engineers:

- people who write basic static sties

- people who write wordpress sites

- people who write simple CRUD apps

- people who write hardware drivers

- people who write compilers

- people who write databases

- people who write google-scale distributed systems

- people who write graphics engines

- people who write physics simulation systems

- people who write video games

The complexity, depth of abstractions, and expected maintenance lifetimes of these tasks vary wildly, but because we all write source code we're all counted as doing the same job! Sort of an obvious lunacy, different mistakes will punish each of these jobs differently than others.

Re: Software engineering research is a train wreck

#133

Post by ctvo » Tue, Jul 20, 2021, 11:54 PM UTC

Hermitian909 wrote: post #132

ftio wrote: post #32

I worked on engineering productivity research and measurement at Google for two years until about a month ago. (Opinions my own.) Compared to my former colleagues, I'm an infant in this area, so take this with a heaping of salt. In general, I think the author's cynicism about productivity research is justified, but I think it could have been directed more productively. (NB: the following comments say nothing about ar…

The poor ontology backing the term software engineer has always seemed to me like the primary culprit for this problem. I've often heard people doing any of the following for their day job referred to as software engineers: - people who write basic static sties - people who write wordpress sites - people who write simple CRUD apps - people who write hardware drivers - people who write compilers - people who write dat…

Do you think other engineers think this way? Your bridge is only 20 yards over a creek, you're not a civil engineer?

I'm sure the complexity, depth, and expected maintenance of some bridges vary too.

Re: Software engineering research is a train wreck

#134

Post by BadInformatics » Wed, Jul 21, 2021, 12:41 AM UTC

aeturnum wrote: post #117

Software engineering research is immature and, as such, has few (no?) strongly-supported claims that the entire field relies on. I agree this is a frustrating state of the field, but the way this is written makes the author appear to be unfamiliar with the state of things. For example: > Did I mention that all three of those papers use different definitions of “defect”? This kind of thing is really common in academia…

As mentioned in TFA, the author has both written and spoken about software engineering research and related topics in some depth before. Though fiery, I think it does its job as a more informal cautionary tale on accepting research in the field without a critical gaze.

Analyses like [1] and [2] (from the same author) are extraordinarily valuable because they help us wade through the firehose of published studies to find what (if any) conclusions may be gleaned. Without such a filter, we are fated to a) repeat work by slogging through the torrent ourselves, or b) taking stuff at face value. Certainly there is no shortage of the latter happening on forums like this, and I wonder how many misinformed decisions are made as a result.

[1] https://danluu.com/empirical-pl/ [2] https://www.hillelwayne.com/post/are-we-really-engineers/

Re: Software engineering research is a train wreck

#135

Post by johncena33 » Wed, Jul 21, 2021, 12:49 AM UTC

Hermitian909 wrote: post #132

ftio wrote: post #32

I worked on engineering productivity research and measurement at Google for two years until about a month ago. (Opinions my own.) Compared to my former colleagues, I'm an infant in this area, so take this with a heaping of salt. In general, I think the author's cynicism about productivity research is justified, but I think it could have been directed more productively. (NB: the following comments say nothing about ar…

The poor ontology backing the term software engineer has always seemed to me like the primary culprit for this problem. I've often heard people doing any of the following for their day job referred to as software engineers: - people who write basic static sties - people who write wordpress sites - people who write simple CRUD apps - people who write hardware drivers - people who write compilers - people who write dat…

It's really good point. I usually use this analogy in the context of unit tests. Everyone understands that the review process for a nuclear power generation site shouldn't be same as foot-bridge over a stream on a trail. However, this basic common sense is missing in all these clean code, TDD, unit tests etc. conversation.

I've seen a startup where the front-end is a very simple web-site (just some basic forms, charts and tables). But the team follows all the clean-code, TDD, unit test and whatever latest buzzword you can think of. It takes one front end developer to sort a list alphabetically more than a day, which probably takes a decent developer less than 10 minutes to finish. IMO, the whole clean-code seminars/books industrial complex if responsbile for the amount of time, effort and money wasted following these clean code trends without any real business value.

Re: Software engineering research is a train wreck

#136

Post by gilbetron » Wed, Jul 21, 2021, 1:16 AM UTC

krisoft wrote: post #83

eschneider wrote: post #55

Earlier quoted context omitted.

It "feels right" and is therefore "truthy."

Do you guys doubt that it is right? What kind of information would you require to show you that it is true? I'm one of the lucky ones who tends to work on safety critical "complex cyber-physical systems". Is that maybe the difference? There is the case where a developer runs unit tests before a commit. Some of the tests are unhappy. Developer fixes it and commits it up. Maybe hours spent? Then there is the case where…

Imagine a requirements document for Hacker News - such documents traditionally have a tremendous amount of change control around it, with tons of review for every change. Remember, people like you believe it is 100x easier to catch a bug in requirements and 100x gives you lots of budget to spend to make sure you catch bugs in requirements.

So, there's a bug in the requirements document. Specifically, the 'logout' link has a spelling error. It says 'lugout'. If someone were to catch this bug in the requirements doc, it would trigger a bunch of costly review, as we established earlier.

However, in the implementation phase, once the entire site is up and running, it would take maybe 5 minutes to fix.

Trivial example, but there it is.

Re: Software engineering research is a train wreck

#137

Post by Swizec » Wed, Jul 21, 2021, 1:20 AM UTC

jeddy3 wrote: post #75

ftio wrote: post #32

I worked on engineering productivity research and measurement at Google for two years until about a month ago. (Opinions my own.) Compared to my former colleagues, I'm an infant in this area, so take this with a heaping of salt. In general, I think the author's cynicism about productivity research is justified, but I think it could have been directed more productively. (NB: the following comments say nothing about ar…

I seems like many posters in this thread try to classify software enineering as either creative or "mindless factory-work". Where actual enineering disciplines has the risk of removing the creative part. I think this classification is wrong. There IS NO mindless factory-work. Just as in other enineering disciplines, our work is not manufacturing. It's just that the actual manufacturing does not exist (or rather is do…

> It's just that the actual manufacturing does not exist (or rather is done by compiler)

Here’s the kicker though: The part that is done by compilers used to be the bulk of software engineering.

In his Art and Science book Hamming talks about how programmers rejected the idea of even just automated address assignment. They took great pride in manually managing absolute addressing. Only a sissy who doesn’t know real programming would ever use something so silly as symbolic addressing, to say nothing of compiling from assembly. Ugh!

Now we don’t even think about that. Too boring, too solved, too uncreative.

There is a lot of engineering that we currently do, which is completely mechanical, mindless, and can be automated away.

Re: Software engineering research is a train wreck

#138

Post by bob1029 » Wed, Jul 21, 2021, 1:22 AM UTC

Software engineering can only exist with an incredible amount of discipline.

We routinely start new implementations in excel and perform formal normalization checks over models before writing any code.

You can get away without having a strong type system as long as your tables and relations are clean. This stuff is so powerful.

Figuring out how to funnel your domain instances into a SQL db and writing all your complex logic in terms of basic queries is how you can build an ecosystem in which determinism and formal proofs of correctness are possible.

For me, software engineering is about correctness for the business end. There is certainly still a layer of craftsmanship below the engineering realm which enables all the magic. This is where things like performance and reliability live. Interacting with the outside world is an icky thing, and the specific implementation always depends on the use case.

Re: Software engineering research is a train wreck

#139

Post by rhizome » Wed, Jul 21, 2021, 1:26 AM UTC

jeddy3 wrote: post #75

ftio wrote: post #32

I worked on engineering productivity research and measurement at Google for two years until about a month ago. (Opinions my own.) Compared to my former colleagues, I'm an infant in this area, so take this with a heaping of salt. In general, I think the author's cynicism about productivity research is justified, but I think it could have been directed more productively. (NB: the following comments say nothing about ar…

I seems like many posters in this thread try to classify software enineering as either creative or "mindless factory-work". Where actual enineering disciplines has the risk of removing the creative part. I think this classification is wrong. There IS NO mindless factory-work. Just as in other enineering disciplines, our work is not manufacturing. It's just that the actual manufacturing does not exist (or rather is do…

At the very least, I imagine a true engineering version of software development would be fundamentally driven by Big-O. Everything justified and implemented in terms of calculable facts. Repeatability might emerge from this.

I think of it this way: what would a CRC manual[1] for software engineering comprise?

1. https://en.wikipedia.org/wiki/CRC_Handbook_of_Chemistry_and_...

Re: Software engineering research is a train wreck

#140

Post by dalbasal » Wed, Jul 21, 2021, 1:26 AM UTC

ftio wrote: post #32

I worked on engineering productivity research and measurement at Google for two years until about a month ago. (Opinions my own.) Compared to my former colleagues, I'm an infant in this area, so take this with a heaping of salt. In general, I think the author's cynicism about productivity research is justified, but I think it could have been directed more productively. (NB: the following comments say nothing about ar…

>> I do not believe in the possibility of a "General Theory of Productivity."

Important frame. I think any effort that doesn't start with this statement is probably headed for frustration.

I reckon this is true also for education, aspects of economics and many other fields. Besides sloppy and/or cynical work, I believe a lot of the "replication crisis" relates to this. Do we really expect that a relationship between watching TV at dinnertime and marital sex life observed in 1986 Tokyo to replicate in 2021 Berlin? If it doesn't, does it mean that it doesn't exist, or teach us anything?

Scientific academia is sort of premised on the idea that we start from a blank slate, and build up an understanding of a field based strictly in science. It doesn't matter that you already know gravity pulls thing to the ground or that water boils when hot. It must be stated scientifically, tested and theorized.

Well... realistically, from this perspective, we know almost nothing about managing people, teaching, lots of things. Yet, we do manage to do them somehow. People all over the world created domesticated cultivars before Gregor Mendel. Science isn't the only framework for knowing things.

Writers working on their process don't generally go to "empirical research." They read Stephen King, or one of many other authors that speak about their process. They find one that feels compelling. Often, they describe things in terms like "earning the respect of the muse." Usually, it's a poorly supported framework made of a smattering of methodology, random bits of advice, unique terminology and some innovative semantics. Sound familiar?