I've seen quite a lot of this too, though from other vantage points.
I don't disagree with anything you say, and would probably go down a similar route. These projects have enormous and unacknowledged risk. One of those risks (the most common pone) is that the project does actually arrive at completion, is good enough to go into use, but it's still pretty crap. Not better than what could have been achieved at a fraction of the cost. You should (as CEO, CIO) take this as a given, and act accordingly. IE, minimize these projects and find alternatives.
So, this isn't a criticism...
Here's the problem as I see it: Custom, "enterprise^" software is important, very. A lot of things could work a lot better if we had better enterprise software, much better. Especially for government, the ability to produce good software that works is tremendous. Transport systems could be better, welfare systems could be better, democratic systems could be much higher information, education.... Google maps really raised the minimum standard for public transport "timetables," especially buses. This changes public transport, enabling multi-stop routes (you would never find them otherwise), casual use, tourist use...
So... How do we do this better? Personally, I think the recurring failure details are probably a red herring. You imply problems like not having a designer, just a dump of requirements from anyone who has the juice to make them. Problems like cargo-cult design around existing processes, rather than adapting to new tools. IMO these are symptoms of bad meta-systems, the "economy" around making decisions, buying services, designing solutions, etc. Also, naturally, the incentives.
Will this be a problem forever?
^I'm at a loss for a good general term. Hopefully everyone understands what I mean.