Why Development Teams Struggle to Deliver on Time, on Budget, or at All

I think that is a myth right up there with “If I was rich, I’d be happy.” Massively funded software projects fail all the time.

Diseases vary just like reasons for misbehaviour in code. While coding also has a bit of construction, that surgery doesn't, a lot of coding time is spent on extending or fixing existing behaviour. As such, the simile is better than the bridge-construction one.

It would also allow discussing leadership; a strong lead handles the 1-12 hours the surgery takes. He/She's expected to know the human body as a system, by being able to diagnose adverse conditions that occur during the operation, instructing the people around her as she goes.

Operations can't take too long, or fatigue gets you; the corollary being that you don't have slippage like you do in s/w dev. You can't be too unskilled, or else you can't perform the work. You have to have lots of training before you're the one leading.

Contrasting education; a lot of physicians go through much rote memorisation when they start their education. Then they continue with laborations; this could be useful for software engineers and operations folk, by letting them try their hand at diagnosing production systems having problems. Such training could be done by a 20-questions approach, with each question being answered a metric of a category of log entries. In the end you should know what the problem was. Labs at uni are much more constrained; they don't teach how large production systems behave and don't teach mental tools to debug them, they only teach the basics of programming. Furthermore, what they teach of programming is never geared towards what real systems look like, because most teachers have never been close to one.

There's no-place in the world that you can be education like what a surgeon would get; comp-sci is more like training everyone to be a psychologist (because we want to get the full picture!). Apprenticeship programs are more like training to be a nurse.

- The formal education people receive is not applied towards bettering industry performance, it's geared towards inventing and academia - Shorter education people receive is not about understanding the system and how to construct and fix it; it's about working next to it - Similar hard rote-memorisation tests could be coupled with in depth debugging/operations sessions and experienced teachers active in large production systems, like understudies preparing to be expert surgeons - In the light of this; we need a career ladder that doesn't end with the same title you start with "engineer"/"senior engineer". One that is structured.

I've done this 50 times before (10 times in this project!) and these kinds of issues still occasionally occur. How does anyone predict for this kind of thing?

Waterfall gave the impression that planning and analysis is costly by insisting it should be carried down to the lowest levels of abstraction, before doing anything else.

I disagree. Even when you're doing mundane everyday stuff you have done a thousand times before, one trivial variation may throw of your estimate completely. Maybe an edge-case in this specific project triggers a bug in a 3rd party's library, and you're losing weeks waiting for the 3rd party to roll out the fix (been there, done that). You just don't know in advance

> you can do analysis and planning at a much higher level of abstraction than development

Of course, but then you'll get an inaccurate estimate.

> Waterfall gave the impression that planning and analysis is costly by insisting it should be carried down to the lowest levels of abstraction

That's only necessary if you want an accurate estimate. I think accurate estimates are pipe dreams.