I cannot prove it, but I think some of these UI and UX horrors are caused by the ever present need for software to change in order to demonstrate it is alive. Imagine you reach peak usability in $YourApp, in the year of our lord 2023. It's maybe not perfect , but there's no conceivable way your app's usability can improved past this point. Users are reasonable proficient, everything mostly clicks into place past the…
My ideal is to create a product that is well loved and used, automate the hell out of every conceivable customer support and operational concern, and either/both move on to a different product or kick back on the beach.
Nobody wants change. I only ever see hordes of people complaining whenever any UI changes. There is never an equivalent mob of true believers on the other side of the line.