I think it mostly has to do with:
1) the large "implementation surface" of three components (TLS+HTTP+WebSocket), each of which by itself requires an API more complicated, if provided by a userspace library, than a kernel-provided TCP socket, and maybe
2) the fact that non-Web servers are rare enough, and WebSockets are still recent enough, that no "standard" library has emerged and had its edges honed down over time to support multiple applications, especially when the current era and funders of open source are probably less incentivized to create application-independent libraries than in earlier eras where "let's work together to create a free OS with minimal effort" was a larger share of the driving forces.
- With TLS you can certainly link with OpenSSL, but the API is more complicated than a kernel-provided TCP socket, and async/nonblocking TLS requires an API much more complicated. TLS sometimes requires a write in response to a read, and to do that in an apparently nonblocking fashion either (a) the application needs to include callsites back into library in its event loop to tell the library when the underlying socket is writeable just in case the library had something buffered it was hoping to write, (b) the library needs to run its own thread that blocks on the underlying socket, or (c) the library can only be used with languages that support async behavior in a more composable way, which is not C. None of those are good options.
- Parsing the incoming HTTP request is tricky and there's no "standard choice" for this either, e.g. a library that's been distributed in Debian/RedHat/Homebrew for >10 years and is depended-on by a bunch of applications.
- The WebSocket protocol requires that a server write a pong in response to an incoming ping. As with TLS, this means a nonblocking implementation requires a thread or integration with the application's event loop, but it's arguably even worse because WebSocket wants the server to respond soon to a ping. (By contrast, TLS-on-TCP is mostly designed so that an app can ignore the read or write direction as long as it wants.) So you don't just need to possibly queue up that pong and later call into the library when the socket becomes writeable; you need to make sure no other event is going to run or block for a long time in the meantime.
So I think the comparison here may not be, "Why isn't there a library that provides an API for WebSockets that's almost as simple as a kernel-provided TCP socket?" (where the kernel basically runs its own thread and does the async work behind the scenes), but maybe more like, "Why isn't there a user-space library that implements QUIC [or nonblocking TLS, or user-space TCP] with a simple API?"
We have implemented a nonblocking C++ WebSocket/TLS server in the cleanest fashion we could (https://github.com/stanford-stagecast/audio/tree/main/src/ht...), also for a low-latency audio project, but it's still a ton of code and has to make its own assumptions/demands on how it gets invoked. If you wanted to adopt a WebSocket implementation into Ardour, I'd be happy to help you make that happen, but it sounds like you very reasonably were looking to outsource this to a library where your application isn't the only user.