Earlier quoted context omitted.
Scrum won out because it was waterfall in disguise. Commitments and sprints become terribly destructive over time.
Waterfall was coined by a paper discussing why waterfall wasn't a good way to do software projects. You aren't wrong, either. I found Scrum to be a series of waterfalls that weren't well thought out. "Iterative Design" essentially meant, "We aren't sure what the button should do exactly, but we know we need it there and it kinda has to do this and we'll figure the rest out for the next iteration." That caused so many…
The tricky bit is having a solution and implementing it.
Having something well thought out is hard work. Is hard work rewarded? Is initiative to even try rewarded? I'd say the answer is no.
The answer becomes yes in small teams that are given time, freedom and resources, or in times of crisis (same thing as small team really, minus the time bit), as far as I can tell. Otherwise, most people's natural tendency (including me) is to phone it in. We'd be wise to let people in software development and many other professions that are not manual labor, to switch careers after 30-35 if they haven't done anything worthwhile by then.
Working with people who have to phone it in for the next 30 years and know it is modern day hell for anyone with an iota of ability.
It's permanent compromise and insincerity, or being an outcast that's temporarily tolerated for doing 5-10x the work of the person next to you.