How to stream media using WebRTC and FFmpeg, and why it's a bad idea
blog.maxwellgale.com
How to stream media using WebRTC and FFmpeg, and why it's a bad idea
1–10 of 41 posts
Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea
#2>And finally, we encounter a large issue without a good solution. In encoded videos, a key frame is a frame in the video that contains all the visual information needed to render itself without any additional metadata. These are much larger than normal frames, and contribute greatly to the bitrate. Ideally, there would be as a few keyframes as possible. However, when a new user starts consuming a stream, they need at least one keyframe to view the video. WebRTC solves this problem using the RTP Control Protocl (RTCP). When a new user consumes a stream, they send a Full Intra Request (FIR) to the producer. When a producer receives this request, they insert a keyframe into the stream. This keeps the bitrate low while ensuring all the users can view the stream. FFmpeg does not support RTCP. This means that the default FFmpeg settings will produce output that won’t be viewable if consumed mid-stream, at least until a key frame is received. Therefore, the parameter -force_key_frames expr:gte(t,n_forced*4) is needed, which produces a key frame every 4 seconds.
in case someone was wondering why it was a bad idea
Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea
#3What about the WebRTC part?
The post ends at RTP out from FFMPEG. Maybe I’m supposed to know how to consume that with WebRTC but in my investigation it’s not at all straightforward... the WebRTC consumer needs to become aware of the stream through a whole complicated signaling and negotiation process. How is that handled after the FFMPEG RTP stream is produced?
Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea
#4>And finally, we encounter a large issue without a good solution. In encoded videos, a key frame is a frame in the video that contains all the visual information needed to render itself without any additional metadata. These are much larger than normal frames, and contribute greatly to the bitrate. Ideally, there would be as a few keyframes as possible. However, when a new user starts consuming a stream, they need at…
H.264 and most other modern codecs support “intra refresh” to avoid this problem, at the cost of a marginally higher bitrate overall. Think of this as a “rolling keyframe slice” which marches across the screen every few seconds.
http://www.chaneru.com/Roku/HLS/X264_Settings.htm#intra-refr...
Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea
#5What about the WebRTC part? The post ends at RTP out from FFMPEG. Maybe I’m supposed to know how to consume that with WebRTC but in my investigation it’s not at all straightforward... the WebRTC consumer needs to become aware of the stream through a whole complicated signaling and negotiation process. How is that handled after the FFMPEG RTP stream is produced?
I use MediaSoup to bridge between ffmpeg and WebRTC. It works pretty well, and I like that it’s all node based.
Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea
#6> -bsv:v h264_metadata=level=3.1
This should be `-bsf:v` and it's not required since this command encodes and the encoder has been informed via `-level`.
Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea
#7To get it into the browser check out rtp-to-webrtc[0]
Another big piece missing here is congestion control. It isn’t just about keeping bitrate low, but figuring out what you can use. It is a really interesting topic to measure RTT/Loss to figure out what is available. You don’t get that in ffmpeg or GStreamer yet. The best intro to this is the BBR IETF doc IMO [1]
[0] https://github.com/pion/webrtc/tree/master/examples/rtp-to-w...
[1] https://tools.ietf.org/html/draft-cardwell-iccrg-bbr-congest...
Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea
#8For the purpose of one to many type of live streaming you would probably want to use HLS.
Twitch uses it's own transcoding system. Here is a interesting read from their engineering blog [0]
[0] https://blog.twitch.tv/en/2017/10/10/live-video-transmuxing-...
Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea
#9>And finally, we encounter a large issue without a good solution. In encoded videos, a key frame is a frame in the video that contains all the visual information needed to render itself without any additional metadata. These are much larger than normal frames, and contribute greatly to the bitrate. Ideally, there would be as a few keyframes as possible. However, when a new user starts consuming a stream, they need at…
Ok, so only a problem in live streams?
(And I suppose also when seeking inside a stream)
Re: How to stream media using WebRTC and FFmpeg, and why it's a bad idea
#10> -bsv:v h264_metadata=level=3.1 This should be `-bsf:v` and it's not required since this command encodes and the encoder has been informed via `-level`.
Thanks for the feedback. I've removed it.