An Introduction to WebRTC Simulcast
blog.livekit.io
An Introduction to WebRTC Simulcast
1–10 of 15 posts
Re: An Introduction to WebRTC Simulcast
#2This approach of the broadcasting "peer" sending multiple streams (with varying bitrates) to a relaying server is a neat solution. This is done to avoid transcoding on the relay servers, so it makes the servers cheap. It is superior to:
> Lower[ring] the bitrate of everyone’s streams so it doesn’t overwhelm the slow user (i.e. the lowest common denominator)
However, it's still a lowest common denominator solution with respect to what codec is being used. The broadcasting "peer" has to send using a codec that every other peer supports. Essentially that means you're stuck with the lowest common denominator codec.
If the relay server was to instead transcode, then the broadcasting peer could submit a single stream (lower bandwidth) in the best codec it supports. Then the relay server would generate additional streams in various bitrates and codecs (something this solution is trying to avoid).
So unfortunately there's still a trade-off.
The article does mention "Scalable video codecs" as a future improvement. However, using the approach described in this article, we're a long way off taking advantage of them, because whilst there's no transcoding every participant would need to support VP9/AV1.
Re: An Introduction to WebRTC Simulcast
#3This approach of the broadcasting "peer" sending multiple streams (with varying bitrates) to a relaying server is a neat solution. This is done to avoid transcoding on the relay servers, so it makes the servers cheap. It is superior to: > Lower[ring] the bitrate of everyone’s streams so it doesn’t overwhelm the slow user (i.e. the lowest common denominator) However, it's still a lowest common denominator solution wit…
How often is that really a problem though?
Re: An Introduction to WebRTC Simulcast
#4This approach of the broadcasting "peer" sending multiple streams (with varying bitrates) to a relaying server is a neat solution. This is done to avoid transcoding on the relay servers, so it makes the servers cheap. It is superior to: > Lower[ring] the bitrate of everyone’s streams so it doesn’t overwhelm the slow user (i.e. the lowest common denominator) However, it's still a lowest common denominator solution wit…
How often is that really a problem though?
Codec support is a very annoying issue on mobile. Hardware support for VP9 (let alone AV1) codecs is relatively recent on Android devices, and Apple doesn't provide HW acceleration for Google's codecs at all (or at least not when I was trying to do it).
Re: An Introduction to WebRTC Simulcast
#5This approach of the broadcasting "peer" sending multiple streams (with varying bitrates) to a relaying server is a neat solution. This is done to avoid transcoding on the relay servers, so it makes the servers cheap. It is superior to: > Lower[ring] the bitrate of everyone’s streams so it doesn’t overwhelm the slow user (i.e. the lowest common denominator) However, it's still a lowest common denominator solution wit…
I would really like to see some projects that leverage SVC, as I haven't seen a single example (and ffmpeg doesn't support them).
Re: An Introduction to WebRTC Simulcast
#6This approach of the broadcasting "peer" sending multiple streams (with varying bitrates) to a relaying server is a neat solution. This is done to avoid transcoding on the relay servers, so it makes the servers cheap. It is superior to: > Lower[ring] the bitrate of everyone’s streams so it doesn’t overwhelm the slow user (i.e. the lowest common denominator) However, it's still a lowest common denominator solution wit…
I would really like to see some projects that leverage SVC, as I haven't seen a single example (and ffmpeg doesn't support them).
I believe Galene does[0] Juliusz was working on pion/RTP for it. You can see all the details here [1]
[0] https://github.com/jech/galene
[1] https://github.com/pion/rtp/blob/master/codecs/vp9_packet.go
Re: An Introduction to WebRTC Simulcast
#7LiveKit is an amazing project. If you haven’t had a chance check it out. It’s the first time I have seen WebRTC approached by people who know Cloud/Scaling and it’s all a open source!
Re: An Introduction to WebRTC Simulcast
#8Earlier quoted context omitted.
How often is that really a problem though?
Codec support is a very annoying issue on mobile. Hardware support for VP9 (let alone AV1) codecs is relatively recent on Android devices, and Apple doesn't provide HW acceleration for Google's codecs at all (or at least not when I was trying to do it).
Android devices in some cases don’t have H264 HW. Libwebrtc also doesn’t support SW H264.
That has added lots of complexity/confusion for people building stuff also. Quite a few footguns in this area :)
Re: An Introduction to WebRTC Simulcast
#9When I click on get started it sends me to medium, which is now asking me to create an account. Not the best experience.
Re: An Introduction to WebRTC Simulcast
#10Earlier quoted context omitted.
Codec support is a very annoying issue on mobile. Hardware support for VP9 (let alone AV1) codecs is relatively recent on Android devices, and Apple doesn't provide HW acceleration for Google's codecs at all (or at least not when I was trying to do it).
Android devices in some cases don’t have H264 HW. Libwebrtc also doesn’t support SW H264. That has added lots of complexity/confusion for people building stuff also. Quite a few footguns in this area :)
If we want to get really pedantic...WebRTC will support SW H264 if you enable it explicitly in the compilation, since you're technically on the hook for royalties if you compile it yourself. Footguns galore :)