The economics of software teams: Why most engineering orgs are flying blind

Nothing much to tell. About 10 years ago on another job, I wanted more context in what I was building, so I suggested shadowing a user so I could see what they actually did, what value the software provided, and where the pain points were. A business analyst attached to my team became somewhat upset because she felt that impinged on her job. So there went that idea.

So what you are saying is that writing is easy, but figuring out what to write is hard?

The hospital information system Millenium from Oracle was bought by two regions in south of Sweden for around 400 million USD. It turned out to be unusable for Swedish healthcare and had to be shut down after just three days. Actual users of the software (doctors and nurses) were not involved in the procurement.

> If you had decided up front to not ship until all these features were in place it wouldn't change the work at all in most cases

In my experience (of primarily web dev), this is not true, and the reasons it is not true are not limited to software architecture conflicts like you describe (although they happen too). Instead the problems I usually encounter are that:

* once you have shipped something and users are relying on it, it limits the decisions you are allowed to make about what features the system should have. You may regret implementing feature X because it precludes more valuable features Y and Z, but now that X is there, the cost of ripping it out is very high due to the backlash it will cause.

* once you have shipped an application, most of the time when you add new features you are probably slightly changing at least some UI, and so you need to think about how that's going to confuse experienced users and how to address that in a way you wouldn't have to when implementing something de novo. For an internal LOB app, that might mean creating announcements and demos and internal trainings that wouldn't be necessary for greenfield work.

* the majority of professional web dev involves systems with databases, and adding features frequently involves database migrations, and sometimes figuring out how to implement those database migrations without losing data or causing downtime is difficult and complicated.

* as web applications grow their userbase, the scale of the business often introduces new problems with software performance, with viability of analysing business-relevant data from the system, or with moderation or customer support tasks associated with the system, and these problems often demand new features to keep the broader business surrounding the software afloat that weren't needed at launch.

* software that has actually launched and become embedded in existing business processes inherently tends to have many more stakeholders in the business that care about it than pre-launch software, and those stakeholders naturally want to get involved in decision-making about their tools, and that creates meeting and communication overhead - sometimes to such a degree that stakeholder management and negotiating buy-in ends up being an order of magnitude more work than actually implementing the damn feature being argued about.

To the extent that the amount of work involved in implementing a new feature is inflated by these kind of factors relative to what would have been involved in doing it de novo, I personally conceive of that as "maintenance" work; and in my experience my work on big teams at successful businesses has on average been inflated severalfold by those factors. (I also count work mandated by legal/compliance considerations that arise only after a successful launch as "maintenance". My rough conception of "software maintenance" is that the delta between "the work involved in building a product de novo with the same customer-pleasing features that ours has" and "the work we actually had to do to incrementally build the product in parallel to it being used" as "maintenance".)

Would most people agree with my broad notion of maintenance? I reckon they roughly would, but it's hard to say since people who talk about maintenance rarely attempt to define it with any precision. You give a precise but extremely narrow definition above. Wikipedia likewise gives a precise but extremely broad definition - that maintenance is "modification of software after delivery", under which definition surely over 99.999% of professional software development labour is expended on maintenance! I guess my definition puts me somewhere in the middle.

I see what you're saying, and our labor isn't the bottleneck from the perspective of engineers. However, I'm pretty sure that business leadership would beg to differ. They see engineering as a necessary evil to getting their products out the window, and it can never be done fast enough. The solution is the bottleneck in their eyes. Engineers have a better understanding of why they do what they do, hence they get that their work isn't inherently a bottleneck in spite of how challenging it can be.

But yes, in reality, you're correct that programming itself is not (or should not) be a bottleneck, but the process around developing a product definitely can be. The irony here is that this bottleneck usually gets worse the more corporate a business becomes and the more they try to treat programming as if it were a bottleneck. Not a day goes by that my job isn't made more difficult because the business wants greater agility and efficiency.