Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to stream media using WebRTC and FFmpeg, and why it's a bad idea

blog.maxwellgale.com

Post reply on HN
Print view View original

11–20 of 41 posts

Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea

#11

Post by dimes »

tatersolid wrote: post #4
mandis wrote: post #2

>And finally, we encounter a large issue without a good solution. In encoded videos, a key frame is a frame in the video that contains all the visual information needed to render itself without any additional metadata. These are much larger than normal frames, and contribute greatly to the bitrate. Ideally, there would be as a few keyframes as possible. However, when a new user starts consuming a stream, they need at…

H.264 and most other modern codecs support “intra refresh” to avoid this problem, at the cost of a marginally higher bitrate overall. Think of this as a “rolling keyframe slice” which marches across the screen every few seconds. http://www.chaneru.com/Roku/HLS/X264_Settings.htm#intra-refr...

I was not aware of this at the time of writing, but it solves a large problem we've been having. Thank you so much for pointing that out.

Edit: I've just tried using intra refresh, and it works pretty well, but the key frame interval is still required.

Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea

#12

Post by wwweston »

mandis wrote: post #2

>And finally, we encounter a large issue without a good solution. In encoded videos, a key frame is a frame in the video that contains all the visual information needed to render itself without any additional metadata. These are much larger than normal frames, and contribute greatly to the bitrate. Ideally, there would be as a few keyframes as possible. However, when a new user starts consuming a stream, they need at…

Thanks for the easy summary.

One thing to consider: for some IRL performances, it's not uncommon that if you arrive late, you might be seated at the timing discretion of an usher. I understand digital experiences may carry different expectations, but I could see building an experience around this, perhaps starting with audio-only and maybe even a countdown to a next keyframe event (every minute?) while a "please wait to be seated" is shown.

Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea

#13

Post by legohead »

mandis wrote: post #2

>And finally, we encounter a large issue without a good solution. In encoded videos, a key frame is a frame in the video that contains all the visual information needed to render itself without any additional metadata. These are much larger than normal frames, and contribute greatly to the bitrate. Ideally, there would be as a few keyframes as possible. However, when a new user starts consuming a stream, they need at…

Still sounds better than a FIR. If you consider a big streamer with thousands of users. Users are constantly arriving and leaving, so the keyframe requests are going to be so constant that I can see keyframes being generated much more often than 4 seconds (assuming I understand it all correctly).

Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea

#14

Post by gregoriol »

amelius wrote: post #9
mandis wrote: post #2

>And finally, we encounter a large issue without a good solution. In encoded videos, a key frame is a frame in the video that contains all the visual information needed to render itself without any additional metadata. These are much larger than normal frames, and contribute greatly to the bitrate. Ideally, there would be as a few keyframes as possible. However, when a new user starts consuming a stream, they need at…

Ok, so only a problem in live streams? (And I suppose also when seeking inside a stream)

It seems to me that you can't seek with a webrtc stream, as it is at least.

Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea

#15

Post by Dylan16807 »

tatersolid wrote: post #4
mandis wrote: post #2

>And finally, we encounter a large issue without a good solution. In encoded videos, a key frame is a frame in the video that contains all the visual information needed to render itself without any additional metadata. These are much larger than normal frames, and contribute greatly to the bitrate. Ideally, there would be as a few keyframes as possible. However, when a new user starts consuming a stream, they need at…

H.264 and most other modern codecs support “intra refresh” to avoid this problem, at the cost of a marginally higher bitrate overall. Think of this as a “rolling keyframe slice” which marches across the screen every few seconds. http://www.chaneru.com/Roku/HLS/X264_Settings.htm#intra-refr...

I would say intra refresh solves a different problem. You still have to wait for the intra refresh to cover the frame before you can start watching properly. That takes just as long as waiting for a keyframe, and needs slightly more bytes.

The benefit of intra refresh is that you avoid having any particularly large frames. If you're using a sub-second buffer, then intra refresh makes your maximum frame size much smaller without sacrificing quality. It's a godsend for getting latency down to tiny amounts. But if you have 1 or 2 seconds of buffer then it's no big deal if a keyframe is an order of magnitude bigger than other frames, and intra refresh is pointless.

Also it's not really a codec thing, it's a clever encoder trick that you can do on basically anything.

Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea

#16

Post by SahAssar »

amelius wrote: post #9
mandis wrote: post #2

>And finally, we encounter a large issue without a good solution. In encoded videos, a key frame is a frame in the video that contains all the visual information needed to render itself without any additional metadata. These are much larger than normal frames, and contribute greatly to the bitrate. Ideally, there would be as a few keyframes as possible. However, when a new user starts consuming a stream, they need at…

Ok, so only a problem in live streams? (And I suppose also when seeking inside a stream)

It's a problem when playing from a non-start, non-keyframe point (which in practice means any arbitrary point). I'm guessing that's what you meant.

Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea

#17

Post by Feoj »

kuter wrote: post #8

For the purpose of one to many type of live streaming you would probably want to use HLS. Twitch uses it's own transcoding system. Here is a interesting read from their engineering blog [0] [0] https://blog.twitch.tv/en/2017/10/10/live-video-transmuxing-...

If you want to acheive something approaching the latency advantages of WebRTC with HLS its well worth checking out the low latency HLS work by Apple and the wider video-dev community.

https://developer.apple.com/documentation/http_live_streamin... https://tools.ietf.org/html/draft-pantos-hls-rfc8216bis-08

Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea

#18

Post by SahAssar »

gregoriol wrote: post #14
amelius wrote: post #9

Earlier quoted context omitted.

Ok, so only a problem in live streams? (And I suppose also when seeking inside a stream)

It seems to me that you can't seek with a webrtc stream, as it is at least.

webrtc is just a stream but you can absolutely tell the sending side to seek to a certain point.

If webrtc is your TV then the sending side is your VHS. You can tell the VHS to rewind or forward, but telling your TV to do the same is impossible. It just shows you what it gets.

Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea

#19

Post by dimes »

Dylan16807 wrote: post #15
tatersolid wrote: post #4

Earlier quoted context omitted.

H.264 and most other modern codecs support “intra refresh” to avoid this problem, at the cost of a marginally higher bitrate overall. Think of this as a “rolling keyframe slice” which marches across the screen every few seconds. http://www.chaneru.com/Roku/HLS/X264_Settings.htm#intra-refr...

I would say intra refresh solves a different problem. You still have to wait for the intra refresh to cover the frame before you can start watching properly. That takes just as long as waiting for a keyframe, and needs slightly more bytes. The benefit of intra refresh is that you avoid having any particularly large frames. If you're using a sub-second buffer, then intra refresh makes your maximum frame size much smal…

Yes, this is a good point. Intra refresh does reduce variability of the bitrate, but the bitrate is still higher than it would need to be if rtcp was supported.
Post reply on HN