Why Software Developers Suck at UX

I'll add:

6. Not every software project even has a UI

7. Some software projects have a very specialized set of users or barely interact with users and solve very technical problems or data problems (the software I work on now runs 24/7 in perpetuity, and as long as it works, they interact with it only once).

Also, we need consider design as not something that needs to be trendy or has a finite time. A lot of design choices made in early 90’s are totally sound - technology has changed ofcourse so we should adapt but the way things are today - it’s chasing trends after trends without giving any thought. Design is not fashion. Changes should be made only when they’re needed, not because designers like a particular aesthetic or want to make things flat, or remove borders because it’s sexy, or go after magenta/purple themes today that are rampant cliches.

Have you tried searching for anything UX/UI on the web and only to find shallow articles, blog spam, nothing authoritative?

I have a hard time finding a good textbook in this area. Why aren’t there any textbooks one may ask?

the reason is because they love complexity. they love complexity because it's stimulating like a puzzle. they create dozens of flags for $ls because they think it's cool. they are in love with minutia but don't know how to get a girlfriend. they spend all their mental energy creating complexity because they think it's a sign of intelligence. they don't get user interfaces just like they don't get haircuts and deoderant.

Composer/musician here. I think one of the reasons, as other comments have pointed out, that even designers sometimes (often?) suck at UX is because UX and the constraints of Capitalism don't mix, much in the same way that art in general and Capitalism don't mix.

UX is generalized optimization at the intersection of multiple specific needs. The software has to work, the appearance/disappearance of features needs to make sense within a workflow or many workflows (but probably without too heavy-handedly forcing a specific workflow). Features need to be discoverable, the UI needs to be information-dense without being cluttered. The application of the application needs to feel inseparable from the app's design. It's a tough problem, and unfortunately it takes a lot of time, trial and error, iteration, and refinement -- all things that Capitalism tends to not like, especially w/r/t things that don't clearly demonstrate a contribution to the bottom line.

If engineering is applied science, then UX is a form of applied art, just as music-as-a-job is. "Art" in this case meaning "something that requires a novel approach for each separate problem and can't really have too many rules-of-thumb or best practices applied to it without strangling the thing in the process."

The science -> engineering conversion, while fraught with its own problems, works because best practices are more readily able to be applied to logical problems. Optimization in this case is identical to Capitalism's goal -- to make more money in less time while spending less money -- so any and all optimizations in the realm of engineering tend to be met with open arms and a corresponding fatter bottom line.

Capitalism -- or rather, those that tend to run Capitalism -- expect that everything is like this. But applied art does not follow the same rules as applied science, one reason being that what works today may not work tomorrow. Styles change, novelty (something highly optimization-resistant) tends to sell. Every app needs a bespoke approach in order for its function to be optimally eXperienced by the largest audience of users possible. UX being a kind of meta-optimization means that, like composing and producing music, you're often discovering the thing in the sea of not-that-things that is the act of creation, sort of like sculpting. Every time I try to optimize the music itself or the process of making it (which is notably separate from the tools that I use to make it -- these should very definitely be optimized), I end up not reaching my goal of making good, impactful, novel music. Instead, I get what always happens when you try to optimize art: generic, uninspired shit.

Whereas every time I allow the process to be what it by definition is -- obtuse, messy, non-linear -- and stumble around in the dark, slowly discovering the "solution" (analogous to "the right approach" or something like that, in the case of music), I end up with a whole that is greater than the sum of its parts, full of thematic and conceptual connective tissue that is as invisible as it is essential.

Unfortunately, deadlines tend to disagree with the ivory tower approach.

This same phenomenon is my theory as to why so many apps break the "prime directives" of UX as outlined above -- they're somehow not information dense while still cluttered, seemingly essential features are non-obvious or non-discoverable, one way of using an app becomes preferred by the app's own internal logic (so, just in the same way that "the medium is the message," "the app is the medium" -- something that can really hamstring creative software for me), etc. On the design front, this tends to result in wasted space in the name of "minimalism," unimpactful and unopinionated color/type choices, and so on. The Windows 10 effect, in a nutshell.

You can't optimize design and experience like you can a supply chain or a pipeline. You just can't.

“I didn't have time to write a short letter, so I wrote a long one instead.” - Mark Twain

TLDR; The real answer is because they are first taught to code applications with a command line interface.