OMG YEEEEES. I'm building web based remote control and if this allows me to do ffmpeg gdigrab, have that become a WebRTC stream and be consumed by a client without the ExpressJS gymnastics I do right now, I'll be over the moon.
FFmpeg merges WebRTC support
171–180 of 204 posts
Re: FFmpeg merges WebRTC support
#172Earlier quoted context omitted.
So it's only the receiving part of WebRTC, now being able to use WHIP in order to ask a server for a stream?
Currently only the sending part! WHIP is 'pushing media via WebRTC' - https://www.ietf.org/archive/id/draft-ietf-wish-whip-01.html WHEP is 'pulling media via WebRTC' - https://github.com/wish-wg/webrtc-http-egress-protocol/blob/... WHEP isn't standardized/still changing a bit. After it lands I will try and get it into OBS/FFmpeg (and more)
Hmm, what does that mean for instance for workloads that use gstreamer's whepsrc? Is there a risk of incompatibility of a whep server running today with next year webrtc?
Re: FFmpeg merges WebRTC support
#173Earlier quoted context omitted.
WebRTC excels at sub-second latency peer to peer, so you can do near-realtime video, so anywhere that is useful. Say you wanted to do a virtual portal installation connecting views from two different cities with live audio, you could have ffmpeg feed off a professional cinema or DSLR camera device with a clean audio feed and stream that over WebRTC into a webpage-based live viewer. Or say you wanna do a webpage that…
The irony is that you don't _actually_ need WebRTC to get subsecond latency; you can fairly reliably get ~100–200ms (plus network latency) with a completely normal TCP stream.[1] But since browsers have effectively standardized on HLS, whose design is completely antithetical to low-latency (you _can_ do low-latency HLS, but only with heroic efforts), low-latency streaming video has never really been part of their bre…
I saw this also. WebRTC just is the path of least resistance/highest adoption at this point.
We could go make a better/simpler standard for video streaming that is TCP. What a giant battle though that would never seen adoption .
I have accepted/embraced the challenge of making WebRTC as accessible as possible. Stuff like WebRTC for the Curious in hopes of making it less painful for everyone dealing with the complexity :)
Re: FFmpeg merges WebRTC support
#174I have a beginner question - Can WebRTC be used as an alternative to sending base64-encoded images to a backend server for image processing? Is this approach recommended?
Depends on your needs!
https://github.com/pion/webrtc/tree/master/examples/save-to-... here is an example of a server that does what you need on the backend
Re: FFmpeg merges WebRTC support
#175Earlier quoted context omitted.
What sort of infrastructure do you need for scaling WebRTC multicast? Are we entering an era where you don't need Amazon's budget to host something like Twitch?
Yes we are :) When OBS merges the PR [0] things are going to get very interesting. Before you needed to run expensive transcoding jobs to be able to support heterogenous clients. Once we get Simulcast the only cost will be bandwidth. With Hetzner I am paying $1 a TB. With AV1 or H265 + Simulcast I am getting 4K for hundreds of users on just a single server. We will have some growing pains, but I am not giving up unti…
> I am not giving up until I can make this accessible to everyone.
I like this guy.
Re: FFmpeg merges WebRTC support
#176Earlier quoted context omitted.
>I assume autoexec is referring to the plethora of WebRTC vulnerabilities which have affected browsers, messengers, and any other software which implements WebRTC for client use. Its full implementation is seemingly difficult to get right. Like what? I did a quick search and most seem to be stuff like ip leaks and fingerprinting, which isn't relevant in ffmpeg.
Here's a (very) small sample gathered from a search for "webrtc" on cve.org and picking high-severity CVEs affecting browsers: * CVE-2015-1260 * CVE-2022-4924 * CVE-2023-7010 * CVE-2023-7024 * CVE-2024-3170 * CVE-2024-4764 * CVE-2024-5493 * CVE-2024-10488 Of course, I agree that it's not relevant to ffmpeg. But seeing "WebRTC" triggers the same part of the brain that looks out for unescaped SQL statements. Good oppor…
So you searched “WebRTC”, and then took the extraordinary step of… not actually reading any of them while simultaneous using them as supposed points? Quick question since you seem to know a lot about these CVEs and have spent a fair amount of time understanding them: how many of those were browser implementation issue?
This is like searching CVE for “node” and then claiming Node is terrible because some node packages have vulnerabilities. Low effort and intended to fit evidence to an opinion instead of evaluating evidence. “Linux” has 17,000 results; using your critical lens, all Linux is insecure.
Re: FFmpeg merges WebRTC support
#177Earlier quoted context omitted.
It could very well be legit, but if you "have not tested its answer yet" the fact that it can generate something that looks plausible doesn't really tell you much. Generating plausible-sounding but incorrect answers is like the #1 most common failure mode for LLMs.
in recent usage, that only happens 10% of the time for me. Usually the results are grounded and so work usually fine.
Could you imagine if any other software failed silently and plausibly 10% of the time? It would never get off the ground. VC money is a hell of a drug
Re: FFmpeg merges WebRTC support
#178A commit that was "co-authored-by" 6+ people and has three thousand lines of code: this is a total wreck of a development workflow. This feature should have been implemented with a series of about 20 patches. Awful.
(long time FFmpeg dev here) You are being downvoted, but you are entirely correct. This is also explicitly not allowed in FFmpeg, but this was pushed after many months, with no heads up on the list, no final review sign off, and with some developers expressing (and continuing to express) reservations about its quality on the list and IRC.
That's really unfortunate to hear. I'm a huge fan of Webrtc and Pion, and was very excited to get some ffmpeg integration -- hopefully some of the quality issues will be ironed out before the next ffmpeg release
Re: FFmpeg merges WebRTC support
#179Earlier quoted context omitted.
Don't try to do cutting edge stuff with a brain that doesn't know anything past a certian date.
Trying to do things off the beaten path with LLMs is rarely successful, especially if there's a related much more popular option. I'm convinced that programmers' bias towards LLMs is strongly correlated with the weirdness of their work. Very often my strange ideas pushed to LLMs look like solutions but are rather broken and hallucinated attempts which only vaguely represent what needs to be done.
> I'm convinced that programmers' bias towards LLMs is strongly correlated with the weirdness of their work.
This is an extremely astute observation; my work has always been somewhat weird and I've never found LLMs to be more then an interesting party-trick
Re: FFmpeg merges WebRTC support
#180Earlier quoted context omitted.
(long time FFmpeg dev here) You are being downvoted, but you are entirely correct. This is also explicitly not allowed in FFmpeg, but this was pushed after many months, with no heads up on the list, no final review sign off, and with some developers expressing (and continuing to express) reservations about its quality on the list and IRC.
That's really unfortunate to hear. I'm a huge fan of Webrtc and Pion, and was very excited to get some ffmpeg integration -- hopefully some of the quality issues will be ironed out before the next ffmpeg release
There's quite some time until the next release, I believe, so it should be.
The biggest thing missing right now is NACK support, and one of the authors has said they intend to do this (along with fixing old OpenSSL version support, and supporting other libraries). Until that is done, it isn't really "prod ready", so to speak.
For some context, there has been a history of half-supported things being pushed to FFmpeg by companies or people who just need some subset of $thing, in the past, and vendors using that to sell their products with "FFmpeg isn't good enough" marketing, while the feature is either brought up to standard, or in some cases, removed, as the original authors vanish, so it's perhaps a touchy subject for us :) (and why my post was perhaps unnecessarily grumpy).
As for the git / premature push stuff, I strongly believe it is a knock-on effect of mailing list based development - the team working on this support did it elsewhere, and had a designated person send it to the list, meaning every bit of communication is garbled. But that is a whole different can of worms :D.