Can't wait to see the bugs! GAN's are famous for some... interesting reconstructions. And better still, nvidia will have no way to debug it since the model is essentially a black box.
Nvidia Uses AI to Slash Bandwidth on Video Calls
121–130 of 197 posts
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#122What I don't like about AI processed images is that they are not real. I can't go past the fact that I am not looking at the picture as it looks like in reality but somehow smart approximation of the world that is not necessarily true.
Video is not real either. The perspective is different and the colors are wrong. It is even less real after it went through lossy compression. The whole point of a lossy compression is to remove details that you perceive as unimportant. For example, leaves on a tree may look like a greeny mess, but that's fine, from afar, you don't make a difference. Using neural networks for compression is far from being a new conce…
In some ways, of course, I agree, but as some other commenters have pointed out, this is essentially a "deepfake" of yourself...in principle there's no restriction on what one could use as their "avatar." As much was demonstrated in the "Different Output Styles" slide.
So I'd have a hard time saying this is not dramatically less "real" than some lossy compression technique. Is there some way to formalize "realness?" Maybe it would be inversely proportional to the hardness of manipulating the medium as a user..?
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#123If the "Free View" really works well, that sounds like possibly the most important part. The missing feeling of eye contact is a significant unsolved problem in video calls.
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#124Earlier quoted context omitted.
Modern machines aren't really designed for real computation like running tests or compiling stuff. They're made to render webpages and load up games. Run your computation tasks on a VM in the cloud...
Are remote development environments common? There’s a CI server doing the heavy lifting but I still run tests constantly on a docker box while developing locally.
I develop entirely remotely. Save the files on a network mounted filesystem, run shell commands over SSH/mosh, etc.
The server is in the same city as me to avoid excessive latency.
The benefits of being able to develop anywhere (including from a phone in a pinch), and being able to add extra CPU's and RAM at the click of a button outweigh the need for a network connection for my usecases.
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#125Nvidia Uses AI to Slash Bandwidth on Video Calls... But only if everybody in the call has a 600$ Nvidia GPU
$600 today, but in 3-5 years an equivalent component will be cheaper, perhaps significantly so.
This is a brilliant idea, even if the hardware is pricey today.
Disclaimer: no affiliation, but I use Zoom/Teams/Slack/FaceTime/YouTube.
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#126> they have managed to reduce the required bandwidth for a video call by an order of magnitude. In one example, the required data rate fell from 97.28 KB/frame to a measly 0.1165 KB/frame – a reduction to 0.1% of required bandwidth. A nitpick, perhaps, but isn't that three orders of magnitude? We've already seen people use outlandish backgrounds in calls, now it's going to be possible to design similar outlandish vie…
It's possible that the "order of magnitude" statement was the majority case, and the 0.1% statement was a best case scenario. So, 1 magnitude is to be expected, but 3 is possible.
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#127> they have managed to reduce the required bandwidth for a video call by an order of magnitude. In one example, the required data rate fell from 97.28 KB/frame to a measly 0.1165 KB/frame – a reduction to 0.1% of required bandwidth. A nitpick, perhaps, but isn't that three orders of magnitude? We've already seen people use outlandish backgrounds in calls, now it's going to be possible to design similar outlandish vie…
This reminds me of a sci-fi novel I read in the nineties. The premise had something to do with actors who took on roles in virtual reality where their bodies are fit with sense-points. They're cast in live-action role-plays with wealthy remote clients. They're basically deep-fakes in VR.
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#128> they have managed to reduce the required bandwidth for a video call by an order of magnitude. In one example, the required data rate fell from 97.28 KB/frame to a measly 0.1165 KB/frame – a reduction to 0.1% of required bandwidth. A nitpick, perhaps, but isn't that three orders of magnitude? We've already seen people use outlandish backgrounds in calls, now it's going to be possible to design similar outlandish vie…
I think you are right on the money with your thoughts on virtual avatars. I've already noticed this phenomenon cropping up in some niches. 1. the phenomenon of VTubers https://en.m.wikipedia.org/wiki/Virtual_YouTuber 2. in the virtual animal crossing late night show, Animal Talking, the presenter's (Gary Whitta) avatar doesn't really resemble how the presenter looks in real life https://en.m.wikipedia.org/wiki/Animal…
There's a Webtoon (it's okay but not great) that had a premise I think will turn out to be precient and reminds me of your last bit. The gist is that it's a future where everyone wears vr goggles. As a result, the teens in the comic all have personalized visuals mapped to their bodies that you can see if you also have your goggle's on when you look at them. The cooler kids even have full blown avatars that cover up their entire body and make them look like everything from aliens to elves.
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#129Earlier quoted context omitted.
If I had to guess, the issue around "black people" is that photos are 2D. We don't really understand just how little information is actually in a photo (we add huge amounts of info in our perception). My guess is that predictive systems are using contrast as a guide to essentially 3D structures which, simply, just cannot be reconstructed from 2D. And therefore, probably struggle more on dark faces which have differen…
While contrast is almost certainly part of it, I’d hazard a guess that the training set is also partly to blame. Now, I don’t know much about neural networks (AI), but my understanding is that if you provide a training set representative of the population makeup, (in America at least) it’ll be biased towards white people as it hasn’t “seen” enough black person images. My limited understanding would then make me think…
Sure, but were those datasets of 3D snapshots of white people, it wouldnt matter.
Black faces and white faces are "statistically equivalent" in 3D.
The issue is more, in my view, the hubris of calling this system "facial recognition". It isnt: it's pixel pattern color recognition which sometimes coincides with certain facial patterns.
Re: Nvidia Uses AI to Slash Bandwidth on Video Calls
#130For something a bit closer to traditional compression while using NNs / "AI", there's wave.one