Video Live Streaming: Notes on RTMP, HLS, and WebRTC • Hacker News Board

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#11

Post by kwindla » Mon, Jun 06, 2022, 5:58 PM UTC

fiestajetsam wrote: post #4

WebRTC is not the future of low latency live streaming... at least not outside of video conferencing. It's incredibly complex as a specification, has limitations and numerous issues that set limits in how scalable it can be. Conversely, for HTTP segment based formats like HLS and DASH have limits due to their design (and that of HTTP) which set how low latency can actually go. Where's the future? A likely candidate m…

I agree that RTP over QUIC [1] is closer to what we'd build today if we were starting from scratch than WebRTC is. (Partly benefiting from the lessons learned getting to WebRTC 1.0, of course.)

It's worth noting that QUIC is also a very complex specification and is only going to get more complex as it continues through the standardization process. In parallel, there's ongoing work on the next generation of the WebRTC spec. [2] (WebRTC-NV also adds complexity. Nothing ever gets simpler.)

My guess is that we're at least three years away from being able to use anything other than HLS and WebRTC in production. And -- pessimistically because I've worked on video for a long time and seen over and over that new stuff always take _forever_ to bake and get adoption, maybe that's going to be more like 10 years.

[1] https://github.com/mengelbart/rtp-over-quic-draft [2] https://www.w3.org/TR/webrtc-nv-use-cases/

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#12

Post by vr000m » Mon, Jun 06, 2022, 6:11 PM UTC

jalino23 wrote: post #5

vr000m wrote: post #3

Earlier quoted context omitted.

Low latency HLS is creating partial segments by bucketing 200ms of frames instead of 6s segments in standard HLS. Whereas in webrtc, the endpoint is sending the frame as soon as it is ready. The apples apples comparison here is 0ms (in webrtc, no send side buffering) vs 200ms (in low latency HLS) or 6s (in standard HLS). This is independent of latency of the endpoint from CDN or source. Another distinction is playbac…

real world around: HLS: 10-30s LL-HLS: 2-6s WebRTC: 0-2s. add couple more startup seconds depending slow is your signaling

Agreed. My calculations are totally skipping finding/updating the manifests or exchanging the SDP Offer/Answers.

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#13

Post by aidenn0 » Mon, Jun 06, 2022, 6:13 PM UTC

I'm a bit surprised to not see RTP mentioned anywhere. It was a major standard for real-time voice last time I was looking at these things, and it was starting to get some use for videoconferencing.

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#14

Post by vr000m » Mon, Jun 06, 2022, 6:21 PM UTC

aidenn0 wrote: post #13

I'm a bit surprised to not see RTP mentioned anywhere. It was a major standard for real-time voice last time I was looking at these things, and it was starting to get some use for videoconferencing.

A future blogpost will talk about all the networking work that went into optimizing for these large size participation.

Webrtc uses a few protocols. RTP is very central to it, ICE and SDP are also very important protocol for NATs/firewalls and capability selections. While we use webrtc as a shortcut to refer to the suite of protocols and didn’t call out RTP specifically for that reason.

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#15

Post by at_a_remove » Mon, Jun 06, 2022, 6:24 PM UTC

This all looks like a lot of Very New Design Decisions. I tend not to be an early adopter on these, although it is mostly moot because I am no longer in the video delivery business. I have been considering streaming sets of short films, commercials, trailers, and movies to friends in a live stream via a pipeline of comparatively hoary old tech:

1) A playlist of multiple different video files in VLC as a source for ...

2) OBS Studio, which produces RTMP to be consumed by ...

3) nginx, which calls ffmpeg to produce multiple birates and resolutions to be rebundled as HLS to be sent to ...

4) a "live TV" channel of my own specification as an input to JellyFin, which can be read by ...

5) various clients on Roku, Apple TV, Firestick, Chromecast "apps"

At this point, I don't think the industry will ever really settle down to something manageable.

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#16

Post by wuyishan » Mon, Jun 06, 2022, 6:41 PM UTC

at_a_remove wrote: post #15

This all looks like a lot of Very New Design Decisions. I tend not to be an early adopter on these, although it is mostly moot because I am no longer in the video delivery business. I have been considering streaming sets of short films, commercials, trailers, and movies to friends in a live stream via a pipeline of comparatively hoary old tech: 1) A playlist of multiple different video files in VLC as a source for ..…

I do something like this by using https://github.com/ffplayout/ffplayout-engine

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#17

Post by sumy23 » Mon, Jun 06, 2022, 6:51 PM UTC

Sean-Der wrote: post #9

fiestajetsam wrote: post #4

WebRTC is not the future of low latency live streaming... at least not outside of video conferencing. It's incredibly complex as a specification, has limitations and numerous issues that set limits in how scalable it can be. Conversely, for HTTP segment based formats like HLS and DASH have limits due to their design (and that of HTTP) which set how low latency can actually go. Where's the future? A likely candidate m…

I am heavily biased toward WebRTC. Here is my take on it though! > It's incredibly complex as a specification What is complex about it? I can go and read the IETF drafts, webrtcforthecurious.com, https://github.com/adalkiran/webrtc-nuts-and-bolts and multiple implementations. QUIC/WebTransport seems simple because it doesn't address all the things WebRTC does. > has limitations and numerous issues that set limits in…

Having attempted to WebRTC as a generic video transport, I can say that WebRTC has insurmountable problems. The two biggest issues are:

1) Lack of client-side buffering. This is a benefit in real-time communication, but it limits your maximum bitrate to your maximum download speed. It’s also incredibly insensitive to network blips.

2) Extremely expensive. To keep bitrate down, video codecs only send key frames every so often. When a new client starts consuming a video stream they need to notify the sender that a new key frame is needed. For a video call, this is fine because the sender is already transcoding their stream so inserting a key frame isn’t a big deal. For a static video, needing to transcode the entire thing in real time with dynamic key frames is expensive and unnecessary.

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#18

Post by Karrot_Kream » Mon, Jun 06, 2022, 6:52 PM UTC

Sean-Der wrote: post #9

fiestajetsam wrote: post #4

WebRTC is not the future of low latency live streaming... at least not outside of video conferencing. It's incredibly complex as a specification, has limitations and numerous issues that set limits in how scalable it can be. Conversely, for HTTP segment based formats like HLS and DASH have limits due to their design (and that of HTTP) which set how low latency can actually go. Where's the future? A likely candidate m…

I am heavily biased toward WebRTC. Here is my take on it though! > It's incredibly complex as a specification What is complex about it? I can go and read the IETF drafts, webrtcforthecurious.com, https://github.com/adalkiran/webrtc-nuts-and-bolts and multiple implementations. QUIC/WebTransport seems simple because it doesn't address all the things WebRTC does. > has limitations and numerous issues that set limits in…

> QUIC/WebTransport seems simple because it doesn't address all the things WebRTC does.

Partially agree here, but the design of QUIC(/WebTransport/TCPLS) make some of the features in WebRTC unnecessary:

1. No need for STUN/TURN/ICE. With QUIC you can have the NATed party make an outbound request to a non-NATed party, then use QUIC channels to send/receive RDP from the sender and receiver.

2. QUIC comes with encryption so you don't need to mess with DTLS/SRTP

3. Scaling QUIC channels is much more similar to scaling a stateless service than scaling something heavily stateful like a videobridge and should be easier to manage with modern orchestration tools.

4. For simple, 1:1 cases, QUIC needs a lot less signaling overhead than a WebRTC implementation. For other VC configurations, a streaming layer on QUIC will probably need to implement some form of signaling and will end up looking just like WebRTC signaling.

---

I just wish WebRTC wasn't so prescriptive of DTLS/SRTP. I'm often fiddling around with VC and video feeds on private networks (for example IPSec or an encrypted VPN like Zerotier), and having to opt into the whole CA system there makes it a bit of a pain. There's also the background that having the browser read from a video or voice source isn't always very low-latency even if the DTLS/SRTP comms is going as fast as the network can, which leads to slower glass-to-glass latency, though there are non-browser ways to use WebRTC and many language frameworks as you indicated.

All-in-all small complaints for a good technology stack though.

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#19

Post by at_a_remove » Mon, Jun 06, 2022, 6:54 PM UTC

wuyishan wrote: post #16

at_a_remove wrote: post #15

This all looks like a lot of Very New Design Decisions. I tend not to be an early adopter on these, although it is mostly moot because I am no longer in the video delivery business. I have been considering streaming sets of short films, commercials, trailers, and movies to friends in a live stream via a pipeline of comparatively hoary old tech: 1) A playlist of multiple different video files in VLC as a source for ..…

I do something like this by using https://github.com/ffplayout/ffplayout-engine

Thank you, that looks like that could replace steps one, two, and three with a single step!

Re: Video Live Streaming: Notes on RTMP, HLS, and WebRTC

#20

Post by irq-1 » Mon, Jun 06, 2022, 6:55 PM UTC

Sean-Der wrote: post #9

fiestajetsam wrote: post #4

WebRTC is not the future of low latency live streaming... at least not outside of video conferencing. It's incredibly complex as a specification, has limitations and numerous issues that set limits in how scalable it can be. Conversely, for HTTP segment based formats like HLS and DASH have limits due to their design (and that of HTTP) which set how low latency can actually go. Where's the future? A likely candidate m…

I am heavily biased toward WebRTC. Here is my take on it though! > It's incredibly complex as a specification What is complex about it? I can go and read the IETF drafts, webrtcforthecurious.com, https://github.com/adalkiran/webrtc-nuts-and-bolts and multiple implementations. QUIC/WebTransport seems simple because it doesn't address all the things WebRTC does. > has limitations and numerous issues that set limits in…

WebRTC is used by phenixrts as the delivery from server to client. The promise of WebRTC was P2P direct connections for video/data transport, and server/client for coordination and fallback.

https://phenixrts.com/en-us/faqs.html

> The scalability of Phenix’s platform does not come from the protocol itself, but from the systems built and deployed to accept WebRTC connections and deliver content through them. Our platform is built to scale out horizontally. In order to serve millions of concurrent users subscribing to the same stream in a short period of time, resources need to be provisioned timely or be available upfront.

https://webrtc.org/

> With WebRTC, you can add real-time communication capabilities to your application that works on top of an open standard. It supports video, voice, and generic data to be sent between peers...