I don't understand why was the next version of IP not just identical to IPv4 but with more bits in address space? Were they trying to do too many things at once in the 90's?
https://cr.yp.to/djbdns/ipv6mess.html
There still hasn't been a coherent answer. The best approximation is that a lot of networking researchers saw their once-in-a-lifetime opportunity to bikeshed the lowest layer of the public network stack, and couldn't resist the urge.
All of this could have been avoided by making NAT64 part of the original IPv6 standard (instead of wasting a decade pretending like it wasn't necessary) and making it a mandatory service provided by every IPv6 router, unless all downstream routers already provide the service. This would have forced an invisible-to-endpoints IPv6 transition, starting at the backbone ("no default route" region). Carriers would have very quickly pushed the NAT requirement downstream (away from the default-free region) in order to offload the burden it creates. Each step of the transition would have been a strictly local change between two peers, one of which is paying the other, and can therefore can be incentivized ("hey, if you run NAT64 for your downstreams so we don't have to, we will charge you less per month"). No global coordination, and the local coordination is always across a commercial relationship where a carrot can be dangled.
But instead we got what is basically a flag day, so "the transition" will never ever happen. We'll still be talking about it in twenty more years.