Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Video Live Streaming: Notes on RTMP, HLS, and WebRTC

daily.co

Post reply on HN
Print view View original

61–70 of 84 posts

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#61

Post by rektide »

Karrot_Kream wrote: post #18
Sean-Der wrote: post #9

Earlier quoted context omitted.

I am heavily biased toward WebRTC. Here is my take on it though! > It's incredibly complex as a specification What is complex about it? I can go and read the IETF drafts, webrtcforthecurious.com, https://github.com/adalkiran/webrtc-nuts-and-bolts and multiple implementations. QUIC/WebTransport seems simple because it doesn't address all the things WebRTC does. > has limitations and numerous issues that set limits in…

> QUIC/WebTransport seems simple because it doesn't address all the things WebRTC does. Partially agree here, but the design of QUIC(/WebTransport/TCPLS) make some of the features in WebRTC unnecessary: 1. No need for STUN/TURN/ICE. With QUIC you can have the NATed party make an outbound request to a non-NATed party, then use QUIC channels to send/receive RDP from the sender and receiver. 2. QUIC comes with encryptio…

> I just wish WebRTC wasn't so prescriptive of DTLS/SRTP.

There was a webrtc-webtransport spec, but it got renamed/retasked to p2p-webtransport[1]. It got renamed/rebuild ~1 year ago[2]. Feels like a pretty strong indicator of webrtc being deconstructed, but whose to say this goes anywhere. We'd also need webcodecs.

It's somewhat scary & also somewhat exciting thinking of the one good, working, browser supported standard being ripped into pieces (p2p-webtransport, webcodecs, more) & being user-implemented. Having the browser & servers have a well-known target is both great but also perhaps confining. If we leave it up to each site/library to DIY their solution, figure out how to balance the p2p feeds, it'll be a long long time before the Rest of the World (other than the very big few) have reasonable tech again. WebRTC is quite capable & a nice even playing field, with lots of well-known rules to enable creative interopation. We'd be throwing away a lot. I'd hoped for webrtc-webtransport, to at least keep some order & regularity, but that seems out, at the moment. But Webrtc-nv is still ultra-formative; anything could happen.

The rest of the transport stack is also undergoing massive seismic shifts. I feel like we're in for a lot of years of running QUIC or HTTP3 over WebRTC Data-Channels and over WebTransport, so we can explore solutions the new capabilities while not having to ram each & every change through with the browser implementers. It feels like a less visible but far more massive Web Extensibility Manifesto moment ("Browser vendors should provide new low-level capabilities that expose the possibilities of the underlying platform as closely as possible."), only at sub-HTML levels[3]. The browsers refused to let us play with HTTP Push, never let appdevs know realtime resources had been pushed at the browser, so we're still debating terrible WebSocket vs SSE choices; terrible. I think of gRPC-web & what an abomination that is, how sad & pointless that effort is; all because the browser is a mere glimmer of the underlying transport. I feel like a lot of experimentation & exploration is going to happen if we start exploring QUIC or HTTP3 over WebTransport. Attempts to reimagine alternatives to WebRTC are also possible if we had specs like p2p-webtransport, or just did QUIC over DataChannels. Running modern protocols in the client, not the browser, seems like a semi-cursed future, but necessary, at least for a while, while we don't yet know what we could do. The browsers are super laggy, slow to expose capabilities.

[1] https://github.com/w3c/p2p-webtransport

[2] https://github.com/w3c/p2p-webtransport/commit/63370be4bb61a...

[3] https://github.com/extensibleweb/manifesto

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#62

Post by jhugo »

aidenn0 wrote: post #13

I'm a bit surprised to not see RTP mentioned anywhere. It was a major standard for real-time voice last time I was looking at these things, and it was starting to get some use for videoconferencing.

RTP is the fundamental media transport protocol in WebRTC (which is not a protocol, but rather a suite of protocols working together in a defined way). Basically all web videoconferencing uses RTP already.

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#63

Post by numpad0 »

Karrot_Kream wrote: post #60
vr000m wrote: post #50

Earlier quoted context omitted.

All the listed protocols came after HTTP. RTSP, SIP borrowed heavily (albeit badly in retrospect) from HTTP. I do not have all the historical context (early 90s), but for WebRTC, the idea was to not define any new protocol(s) or do a clean slate design. but rather to just agree on the flavors of the various protocols, and then to universally implement those. We already had SDP, RTSP, RTP, SAP, etc. And the idea was t…

I know they came later, so I'm still confused why RTSP and SIP weren't implemented atop HTTP. I realize that RTSP and SIP can push server to client, but there's ways around that, though perhaps long polling and Websockets weren't conceivable when RTSP and SIP were invented. I mean, in a pinch, I have an HTTP server serving a folder where SDP files are generated, and I've written clients that just look for a well-know…

To me, the whole typical IP Multimedia stack screams telco. They prefer to remove and reattach headers upon passing interfaces, separate control and data plane, and rely on synchronization for session integrity. Great when there's a phone line to HQ and a heavily metered satellite link to do a live, I guess...

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#64

Post by kwindla »

difosfor wrote: post #52

We're using DASH and experimenting with LL-HLS to get reliable and affordable (using CDNS) end to end latency of 3 seconds with which you can already interact live with your audience without too much trouble. Latencies down to about 1 second are also feasible already at the cost of some client side buffering. So I would have expected to see more information about DASH than HLS here. It's usable on all platforms excep…

Nice!

I am a bit skeptical about "down to about 1 second" being achievable with DASH or LL-HLS reliably. Of course, I could be wrong. And a lot depends on the definitions of "about" and "reliably," as well as your user cohort (where they are in the world, etc). :-)

The reason I didn't write much about DASH is that the basic concepts are the same for both DASH and HLS. And my sense for the last couple of years has been that most of the momentum in the ecosystem as taking place around HLS/LL-HLS. But I could be wrong about that, too.

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#65

Post by aidenn0 »

jhugo wrote: post #62
aidenn0 wrote: post #13

I'm a bit surprised to not see RTP mentioned anywhere. It was a major standard for real-time voice last time I was looking at these things, and it was starting to get some use for videoconferencing.

RTP is the fundamental media transport protocol in WebRTC (which is not a protocol, but rather a suite of protocols working together in a defined way). Basically all web videoconferencing uses RTP already.

Okay it's funny because I had vaguely remembered something like that, but a "C-f rtp" on the wikipedia page for WebRTC didn't yield any hits, even though SIP (over websockets) was prominently mentioned, so I just figured the similarity in RTC and RTP had caused me to misremember.

I assume WebRTC includes STUN/TURN/ICE (negotiated over SIP?) then for traversing NATs? The last time I was really into networking was 2001-ish so that stuff was still around the corner, but I kept up with my reading for a few years after that. I also had some of these acronyms refreshed when setting up Jingle, which uses XMPP instead of SIP, but establishes an RTP connection much like traditional VOIP would use.

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#66

Post by kwindla »

mchusma wrote: post #43
kwindla wrote: post #7

Earlier quoted context omitted.

[post author here] In addition to what vr000m said above, I'll just add that when you make HLS chunks smaller, you're reducing the leverage you get from HLS's core design decisions. I tried to cover some of this in the post. One way to think about this intuitively is that HLS and WebRTC are opposite ends of one important trade-offs axis. HLS is about delivering media streams in a way that scales as cost-effectively a…

I appreciated your post so thank you. I would be more interested in understanding why you don't see <1s HLS chunk sizes as working in most cases? I feel like p99 real world latency stuff would show some natural buffer sizes.

The smaller the chunk (file) sizes, the less benefit you're getting from pushing the chunks through a CDN. More requests will come back to your origin server. And there's a lot of complexity in the pipeline from encoder -> origin server -> CDN that's mostly hidden (which is good) until you hit big performance cliffs (which is bad).

This is something that I think most people trying to implement low latency HLS and DASH have struggled with. It's not only the connection to the client that can stall. Your CDN can stall internally, too, waiting on chunks. And, in fact, if your CDN never ever has any internal performance issues, that's probably an indication that it's configured in such a way as to make the costs of delivering video through the CDN pretty much the same as the costs of delivering that same data through cascading WebRTC media servers!

Also, the CDN -> client link is TCP, so you're giving up the ability to just drop packets. TCP is going to do its lossless/ordered thing for you, which again is great for most of what we do on the Internet but starts to actively work against you when you're trying to get down to very low latencies.

Does that make sense? I tried to cover some of this in the footnotes. Apologies for not doing a better job.

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#67

Post by jhugo »

aidenn0 wrote: post #65
jhugo wrote: post #62

Earlier quoted context omitted.

RTP is the fundamental media transport protocol in WebRTC (which is not a protocol, but rather a suite of protocols working together in a defined way). Basically all web videoconferencing uses RTP already.

Okay it's funny because I had vaguely remembered something like that, but a "C-f rtp" on the wikipedia page for WebRTC didn't yield any hits, even though SIP (over websockets) was prominently mentioned, so I just figured the similarity in RTC and RTP had caused me to misremember. I assume WebRTC includes STUN/TURN/ICE (negotiated over SIP?) then for traversing NATs? The last time I was really into networking was 2001…

WebRTC doesn't proscribe a signalling mechanism at all. SIP is sometimes used, Jingle (XMPP) sometimes, or sometimes it's just a custom protocol exchanging SDPs (or equivalent structures) over WebSockets or a REST API.

WebRTC itself is RTP (DTLS-SRTP), ICE (incl. STUN/TURN), codecs & related parameters, capture mechanisms, all bundled up into a Web API.

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#68

Post by kwindla »

dbrueck wrote: post #54

Great article, thank you! Minor nit about footnote #10 (the very bad network failure mode): it really depends on where the bottleneck is and how the client has been implemented to react - there's a huge spectrum of client implementations out there, ranging from nearly dumb to those having wizard level heuristics. If it's the user's last mile connection (between e.g. their home and their ISP), then the big HLS/DASH/et…

That's a really great point about multi-CDN architectures.

Partly because big parts of WebRTC are not standardized (session setup signaling, of course, but also in practice lots of necessary state management) it's a little bit hard to imagine how to build an equivalent for WebRTC.

Relatively recently, I would have said that our experience running large-scale WebRTC stuff in production made "core" infrastructure failure relatively low on our list of concerns. The two components of the last mile connection, on the other hand, are always a huge pain point because of the long tail of bad ISPs and bad Wifi setups.

However ... many of us who try to deliver always-available video services got something of a wakeup call in November and December last year when AWS had two pretty big outages two months in a row.

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#69

Post by kwindla »

vdnkh wrote: post #56

Twitch latency in low-latency mode is ~2s (you can check for yourself in the settings cog menu -> advanced -> video stats) because of "prefetch segments", which are delivered via HTTP 1.1 streaming body responses. The client requests this segment and long polls while frames are delivered as they're being encoded. It's simpler and cheaper than WebRTC or Apple's low-latency HLS parts, and scales to hundreds of thousand…

The Twitch stuff is really, really good.

The reason I didn't talk about it in the post is because the basic ideas that go into reducing the latency of any HLS/DASH implementation are the same. Smaller segment sizes, chunked transport, and smart encoding and prefetch/buffering implementations.

But ... while there's lots of terrific engineering underpinning the Twitch approach, there's no way around the real-world limits to that approach. As a result, on good network connections you'll usually get ~2s latency. But not so much on the long tail of network connections. If your use case can gracefully accommodate a range of latencies for the same shared session across different clients, that's fine. If it can't, it's not fine. Plus, I don't think you can get down very much below 2s.

(I don't have access to any Twitch/IVS internal data, so I don't know what latencies they see globally across their user base. But I've done a lot of testing of this kind of stuff in general.)

And, yeah, Apple. Agreed.

Post reply on HN