For me, I think the biggest issue with large scale deployment of HTTP 3 is that it increases the surface area of potentially vulnerable code that needs to be kept patched and maintained. I'd far rather have the OS provide a verified safe socket layer, and a dynamically linked SSL library, that can be easily updated without any of the application layer needing to worry about security bugs in the networking layer. Addi…
A "few extra ms" is up to 3 roundtrips difference, that's easily noticeable by humans on cellular. For all the CPU optimisations we're doing, cutting out a 50ms roundtrip for establishing a HTTP connection feels like a great area to optimize performance.
HTTP/3 makes a meaningful difference for machines that need to work with HTTP endpoints, which is what Google needed it for: it will save them (and any other web based system similar to theirs) tons of time and bandwidth, which at their scale directly translates to dollars saved. But it makes no overall difference to individual humans who are loading a web page or web app.
There's a good argument to be made about wasting round trips and HTTP/3 adoption fixing that, but it's not grounded in the human experience, because the human experience isn't going to notice it and go "...did something change? everything feels so much faster now".