How does this work when I show my back garden through the video stream?
It doesn't, and just falls back to using full bandwidth as usual.
Nvidia Uses AI to Slash Bandwidth on Video Calls
81–90 of 197 posts
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#82Isn’t this just like apples animated emoji (Animoji) where your face is mapped to a emoji character? Except instead of a cartoon it’s mapped to your actual face. https://blog.emojipedia.org/apples-new-animoji/ And how well does that work when you switch to screen sharing?
I’m actually somewhat surprised someone hasn’t already built video chat just based on this ^
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#83I would go to the origin announcement rather than this reproduction with ads https://developer.nvidia.com/maxine?ncid=so-yout-26905#cid=d...
What ads? I can't see any. That damn uBlock Origins, must be the culprit in my case :D
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#84Is AI cheaper than bandwidth?
Sometimes... Sometimes not...
spotty mobile connections with a powerful device (iPhone Android etc...) it might make sense.
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#85I would imagine Apple doing this with FaceTime soon.
Using their own NPU ( Neural processing unit ), you can now make FaceTime call with ridiculously low bandwidth. From the Nvidia example, 0.1165 KB/frame even at buttery smooth 60fps ( I could literally hear Apple market the crap out of this ), that is 7KBps or 56Kbps! Remember when the industry were trying to compress CD Audio quality ( aka 128Kbps MP3 ) down to 64Kbps? This FaceTime Video Call is using even less!
And since the NPU and FaceTime are all part of Apple's platform and not available anywhere else. They now have an even better excuse not to open it up and further lock customers into their ecosystem. ( Not such a good thing with how Apple are acting right now )
Not so sure where Nvidia is heading for this since Not everyone will have a CUDA GPU.
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#86A technology very similar to this plays a plot point in Vernor Vinge's 1992 novel A Fire Upon the Deep. In his universe, both the interstellar net and combat links between ships are low bandwidth. Hence, video is interpolated between sync frames or recreated from old footage. Vinge calls the resulting video "evocations".
I was thinking of exactly this when I read the article. The plot point being that when the bandwidth gets too low, the interpolation AI has to make lots of stuff up, you are not quite sure exactly what was said. I seem to remember the bandwidth in the book was very tiny, small number of bits per second (?) so the AI was taking the speech and compressing it into something more compressed than text then decompressing i…
Wow, that’s a fascinating concept. Effectively a bitmap index of possible terms said and synthesized speech back on the other end based on old footage of how that person talks.
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#87I would imagine Apple doing this with FaceTime soon. Using their own NPU ( Neural processing unit ), you can now make FaceTime call with ridiculously low bandwidth. From the Nvidia example, 0.1165 KB/frame even at buttery smooth 60fps ( I could literally hear Apple market the crap out of this ), that is 7KBps or 56Kbps! Remember when the industry were trying to compress CD Audio quality ( aka 128Kbps MP3 ) down to 64…
The latest NPU can't deliver this kind of "deep compression" at 25fps, not even 10fps. But in the future they can just send the Facemesh vertices and streaming text of the speech (and classically compress non-speech audio, if it's even desired as most people are happy just using it to talk) so it will be less data than 1Kbps.
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#88> they have managed to reduce the required bandwidth for a video call by an order of magnitude. In one example, the required data rate fell from 97.28 KB/frame to a measly 0.1165 KB/frame – a reduction to 0.1% of required bandwidth. A nitpick, perhaps, but isn't that three orders of magnitude? We've already seen people use outlandish backgrounds in calls, now it's going to be possible to design similar outlandish vie…
>isn't that three orders of magnitude?
sure is .. if you stream your face at >30-50Mbit/s. For contrast highest bitrate available on Twitch, used for streaming high motion full screen updating twitchy 1080@60 gaming, is ~6-8Mbit/s.
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#89I would imagine Apple doing this with FaceTime soon. Using their own NPU ( Neural processing unit ), you can now make FaceTime call with ridiculously low bandwidth. From the Nvidia example, 0.1165 KB/frame even at buttery smooth 60fps ( I could literally hear Apple market the crap out of this ), that is 7KBps or 56Kbps! Remember when the industry were trying to compress CD Audio quality ( aka 128Kbps MP3 ) down to 64…
Probably just turning an R&D project into a marketing feature.
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#90Using middle out compression?