Things like this make me think that it's a shame humanity is duplicating all this work instead of collaborating to do it once. One of the awkward failures of capitalism, I guess. At least when your product is something physical you end up with twice as many outputs. When it's just data you just do the same work twice.
Not quite. I think "competition" is overly glorified, but it does some things well. In this context, routing around path dependence is a key benefit. The upstart competitor in a field doesn't have to "waste" a decade of investment & infrastructure to bypass a methodology that has reached its maximum utility. The old method was cheaper for the incumbent to maintain, no amount of good will and collaborative spirit would convince them to turn their back on it for an unproven alternative. Any industry that has ever been "disrupted" is an industry shaken out of path dependence