Perhaps part of the problem was a premature switch from exploration to exploitation? When Facebook bought Oculus for $$$$, the collaborative community which had been pushing the envelope, exploring for possibilities, rapidly died. People took it as a hint to switch modes. And eye-tracking rent seekers, already in low-volume high-cost exploitation mode, weren't then incentivized to support exploration. Gaming had doll…
Facebook has already released two mass market targeted VR headsets with moderate success. It's not console numbers yet, but then again, there's still plenty of low-hanging fruit to grab when it comes to technical improvements (and unlike early video game consoles, the market always has plenty of mature ecosystems for interactive games).
A less concentrated industry might have been more interested, creative, and effective, at gathering such fruit, and exploring for accessible markets. Perhaps yielding a less slow takeoff. A counter argument is that Intel's RealSense does exist to build on, and Leap Motion's hand tracker and Project NorthStar. A counter^2 argument is these are outliers, with LM's work only remaining available, because they dropped IIRC three separate acquisition attempts by Apple. Most work hasn't remained available to use and build on.
But perhaps a healthy commercial-diy blended market was never an option. Given a patent regime optimized for pharma-shaped industry. There seemed a long-term pre-concentration pattern of patent dodging via non-commercial software, and hardware "kits". Leaving many capabilities already unavailable, because a bit of commercially-motivated effort was needed, but discouraged by a too-small community market and/or associated patents.
The current concentration of effort may well get us more quickly to eventual phone-scale broad consumer adoption of XR. It has motivated the supply and R&D tech chain in a way a small market never could. But perhaps it has also contributed to a less-diverse flatter takeoff. And perhaps, if XR encounters difficulties with mass consumer adoption, it maybe have added a fragility risk of larger delays.