Why Is There So Much Crap Software in the World

Not gatekeeping. That would be preventing folks from coding by law. A ridiculous idea.

More like, enforcing certain standards. ie "When you hire from X, you can assume they know Y and are proficient with Z". It already happens with recruiting. When I see resumes from X I know what to expect.

If someone decides to engage in a race to the bottom and hires "engineers" from something other than X then, good luck. Maybe you found a diamond in the rough, but likely not. In that case don't whine that it's impossible to get nice software.

This article seems to be based on several assumptions that feel a bit off in my experience.

Splitting the level of developers in three equal groups makes it look like it will take the same amount of time to get from noob to mediocre as it takes to get from mediocre to good. If I were to draw these lines the first one would be after a few weeks. You don't need more time to write useful software. You maybe can't solve hard engineering problems after that time, but it's enough to work in a team and be productive.

I'm not sure where to draw the second line, though. I don't know if there's a point where you simply stop making stupid problems. You get better at solving problems and might shoot yourself in the foot less often, but to write bug-free software, you really need dedicated QA who will concentrate on that. I don't think it's a problem that can be solved by individuals, but only in a team.

And this leads me to the conclusion that software quality is mostly driven by the company and economics. If the project doesn't pay for solid quality assurance or has too much time pressure on developers, the software you get might be bad at least to some degree. Even good developers can't make up for that.

I'm not sure this makes any sense outside my current bubble, so please add your 5 cents.

> And this leads me to the conclusion that software quality is mostly driven by the company and economics. If the project doesn't pay for solid quality assurance or has too much time pressure on developers, the software you get might be bad at least to some degree. Even good developers can't make up for that.

I think this is absolutely the case. As an extreme example, an organization that uses formal methods is probably going to ship higher quality software because they've meticulously worked out design bugs before they've even written a line of code and, once the program is implemented, they've rigorously verified its correctness – the economics of this methodology, however, are not really feasible in most circumstances and so instead we opt for duct tape fixes and crunch time.

The author also mentions the lack of gatekeeping for software engineering – this is often pointed to as one reason for bad software existing. I tend to agree, since "good code" and "bad code" that both do what they're supposed to under the common case can appear the same to the user and the manager who signs off on it (excluding bad UI). Point being, if someone can code just well enough to get by in their field, they will probably remain employed. This is a tricky problem to solve though, because so much of the practice of software engineering is qualitative (bordering on aesthetic, I would argue) and involves designing abstractions. While much of software engineering is analytical and must obey a certain set of axioms, higher-level systems and software design is really more akin to art and architecture than civil or mechanical engineering.

I say this in a kind manner — it's spelled "stakeholder" and not "steak holder."

I think the situation can be described pretty accurately like this:

1. A developers job is more complex than most, and takes a very long time to learn, because you have to become proficient in so many aspects of the job. e.g.

- technical stack which is huge (languages, protocols, how hardware works, networking etc),

- plus technical tooling (ides, source control, platforms such as clouds, programming frameworks and so on and on)

- programming models (desktop ui, web ui, state management, apis consumption, api creation, security models, and so on)

- teamwork (project management systems, soft skills, team dynamics, professional writing and so on)

- user level software (operating systems, office software, communication software, anything users use that you need to interact with)

- understanding your industry (how all the above changes and evolves over time)

- one or more business domains - to know what software to write and the nuances of how to create it to be truly useful

- all the other areas, such as high performance, or high quality, or design for testability, or how systems fit into companies management frameworks, or designing a system for manageability over the span of it's lifetime, or for easy extensibility and so on and so on

2. Skill levels

- This is a hard ask, many people don't have the kind of mind to ever do this well, and will be mediocre forever

- Learning time, it tends to take 5-10 years for talented people to become proficient in ALL the above areas they need to combine to truly be a journeyman developer

- True mastery requires a commitment to learn to a deep level on so many area, and 10-20 years is probably realistic for smart people with the right mindset, to truly get to a level of masters of so many related disciplines

3. Skills triangle

- The industry has been growing so fast. There are ten or a hundred times more recently started developers than there are developers with decades of experience because as an industry software development is growing by orders of magnitude as every company becomes an IT company where in the past it was rare.

- People often move up and out. Of the rare developers with the capacity to become real masters, who then have the 10-20 years of experience to have time to learn it all deeply - many of them have left the field by that time. This is for various reasons such as: having made their fortune and moving to investing, getting sick of computing or the industry and wanting a change, moving across to management, not updating their skills to keep up with technology etc

- Quick napkin guesses to put metrics to these -- 1/10 developers with more than 10 years serious development experience (90% less than 10 years, because of fast growing industry) -- 1/3 developers with capacity to really master it -- 1/3 of those remaining in development instead of moving up and out

= on these completely made up but semi realistically guessed figures, 1/10 * 1/3 * 1/3 calculates out to having 90 non master developers for every master developer in the industry

So a true master developer is very, very, very rare.

~

As well as being very rare, they are other factors which mean their experience and guidance is not always utilised:

- management that often suffers from lack of experience technically

- peverse commercial incentives as one of the other comments here mentioned

- fashion storms of technology that occur due to the hype of a significant proportion of the hundred times as many less experienced developers filling the blogsphere and management mindshare with hype regarding the latest thing they've re-discovered that is already known if they cared to try to learn more deeply instead of try to 'thought lead'

- ageism and overemphasis on the 'new shiny' vs tried and true techniques

Last points might seem a bit bitter but please stay on topic, they are mentioned because they are significant, real reasons why there is so much crap software in the world!