> But if you want to swap the source of the video to your 240p version if the video is buffering 'too slowly'?
This sounds like something that should be a feature of the protocol used for streaming (in fact, I'd be surprised if most streaming protocols didn't support such functionality). Even without, this should be possible to do even with a native media player, and many streaming providers offer streams with different bitrates and encodings and such (for an example, check out soma.fm).
> YouTube-like annotations?
Most video players have subtitle and even captioning support; it shouldn't be hard to extend this to arbitrary annotations, be it as part of the streaming protocol, part of the media encoding, or even as a separate stream or file.
> Show suggested videos after?
Could be hypothetically done with an API call on the player's part to the video source (or some other source that indexes videos), which then provides the list. The player could then display said suggestions.
All three of these things could be worked around with a standard "metadata" path that such a video player would access and fetch an HTML page or JSON/XML document or somesuch.
> Developers aren't going to want to support all environment configurations
Nor should they; they should instead support agreed-upon standards, like how web browsers support agreed-upon standards for HTML/CSS/Javascript files and HTTP(S) 1.x/2.0. I don't use separate web browsers for different websites, after all.
> Additionally, there's a lot of great research going into browser sandboxing and user-driven permission granting. Allowing content-compatible-but-unsandboxed XYZPlayer to take over content playback negates a lot of the potential benefits.
Part of the issue is that operating-system-level sandboxing has historically been insufficient. Bringing some modern server-grade sandboxing techniques (containers, VMs, chroots, etc.) to the desktop and mobile realms in an easy-to-use manner would alleviate the need for browsers to try and implement this themselves.