I think one of the primary reasons that it is such a dumpster fire is there traditionally hasn't been an "open" ecosystem in the hardware world, though now they are being forced towards that direction kicking and screaming. Every part of the hardware ecosystem has traditionally been done in closed, NDA ridden environments and only over the last 5-10 years has that even started to change. Designing chips has required…
Is it, though? Software thrives because it has an open ecosystem. But it's no less of a dumpster fire of complexity, being slowly buried under the technical debt. We literally have decades-old system designs wrapped into multiple layers of progressively newer systems, and openness doesn't help here. Most software is write-only.
The overarching reason might be the lack of refactoring and feedback loops. It's more related to the production cycles and incentives than to the second-order effects of openness.