Why writing software is not like engineering (2008)

In a billion dollar construction project, everyone understands that if you want to change the design or significantly expand the scope halfway through, you're going to have a heck of a lot of increased costs & time.

In software, it's very common to add scope or major design changes at any stage without understanding that the same cost and time increases will apply.

For example saying 90% through the project:

-Software: "we need a 2-way interface added between the complex software we're building and some other complex software"

-Building: "We need an enclosed concourse built between our new building and the next building over across the street."

It is obvious in the later case that there will be a lot of extra time & cost. While in the former, higher level decision makers on a software project don't understand the comparable difficulties.

If construction companies figured out they could outsource labor to countries with far lower currency values and thereby not-pay for extra labor, they'd do the same.... oh wait...

- It takes several years for someone to prove themselves as a developer among their peers and this reputation does not usually spread outside the confines of a single company. People have to prove themselves all over again each time they change company.

- Big, high-exposure, high-influence software companies like Facebook, Google etc... force arbitrary tests on prospective developers (candidates); but these tests say nothing about a person's skill as a software developer. These tests are reliable at selecting for status-oriented individuals who can regurgitate solutions to common programming puzzles and present themselves as fast-talking and quick-thinking (I.e. Ben Shapiro archetype). These companies have a huge impact on global software development tools and philosophies; their recruitment decisions have a huge impact on deciding industry best-practices. The huge influx of superficial, fast-thinking, fast-talking software developers within the centers of power is making the software industry look like the medical industry 200 years ago when all reputable doctors insisted that blood-letting, electroshock therapy and lobotomies were reasonable treatments for most ailments. Their arguments relied on superficial, smart-sounding observations and appealed to the authority of large medical institutions. Modern developers like to leverage their associations with financially successful corporations 'sources of authority' to project themselves as top experts and to promote their agendas.

- Too many people who have no sense of curiosity at all joined this industry purely for financial reasons over the past couple of decades; these status-oriented people will end up running all the big tech companies and enforcing their ill-conceived philosophies onto others, silencing inquisitive minds. It's a numbers game; the voices of the many high-status, fast-talking, loud-mouthed idiots will drown out those of the few quiet, low-status geniuses.

We are entering the dark ages of software development.

In medical, automotive, aerospace, military applications, the software that goes into the device is kind of considered part of the object. It needs this validation for obvious reasons, such as the long lifecycle and critical nature of these products.

But in cellphone handsets, PC software and firmware such as in IoT home devices, and even IT infrastructure like SoHo routers especially but also other devices, there is no agreed upon standards that I am aware of. Even in payment applications, there isn't much there. Large institutions and banks get by often by using "legacy" infrastructure like mainframes to keep the COBOL stuff running, as it has been vetted over decades.

Going further, the architecture of infrastructure, firmware and software should be examined more closely. Recently, IT architecture has been exploited over and over (Kaseya, SolarWinds, MTA) having quite asymmetric effects. In the past it was worms and drive-by malware from janky websites, today it is extremely costly disasters that make the news but roll on bye until the next turd is flipped over.

The problem space is very difficult to define, since "software" and "architecture" are such general terms encompassing basically anything. I rate the difficulty in analyzing these problems for sufficient solutions as very high. When I consider neural-net AI applications, it seems to get even more murky.

Is anybody aware of other software and infrastructure standards that exist or are in research which could apply to more general fields, or to corporate/business IT infrastructure, or to home devices, etc?