Live data from Hacker News

Nvidia Uses AI to Slash Bandwidth on Video Calls

petapixel.com

91–100 of 197 posts

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#91

This includes a feature: > Called “Free View,” this would allow someone who has a separate camera off-screen to seemingly keep eye contact with those on a video call. Am I the only one who thinks eye contact on video calls feels creepy? I think I would prefer this feature to remove eye contact on video calls rather than add it.

If you have a laptop and an external screen this is useful to not look like someone looking away from people

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#92

> they have managed to reduce the required bandwidth for a video call by an order of magnitude. In one example, the required data rate fell from 97.28 KB/frame to a measly 0.1165 KB/frame – a reduction to 0.1% of required bandwidth. A nitpick, perhaps, but isn't that three orders of magnitude? We've already seen people use outlandish backgrounds in calls, now it's going to be possible to design similar outlandish vie…

I think the ability to, as someone mentioned it, have yourself look a bit tidier than you actually are (working from home) could be a huge benifit.

I mean taking away focus on things that doesn't matter in a virtual meeting such as: Where you are sitting - via Virtual Background Your daily hair style status or if you have a nose pimple - Via NVIDIAs AI showcased here. Would be great.

Though replacing yourself with a "digital" avatar I think takes away many of the benefits an actual live meeting provides.

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#93
post #85

I would imagine Apple doing this with FaceTime soon. Using their own NPU ( Neural processing unit ), you can now make FaceTime call with ridiculously low bandwidth. From the Nvidia example, 0.1165 KB/frame even at buttery smooth 60fps ( I could literally hear Apple market the crap out of this ), that is 7KBps or 56Kbps! Remember when the industry were trying to compress CD Audio quality ( aka 128Kbps MP3 ) down to 64…

> I would imagine Apple doing this with FaceTime soon.

What's this claim based on? Last I looked into FaceTime tech, they didn't do anything special - their quality comes from use of H.265 and the fact that iOS devices have good quality HW encoding blocks which provide good compression at low bandwidths.

FaceTime stream is usually also low motion / change so it's possible to achieve very good compression even with basic quality. Although they still don't quite match the quality of AV1 powered Google Duo on very poor connections.

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#94
post #85

I would imagine Apple doing this with FaceTime soon. Using their own NPU ( Neural processing unit ), you can now make FaceTime call with ridiculously low bandwidth. From the Nvidia example, 0.1165 KB/frame even at buttery smooth 60fps ( I could literally hear Apple market the crap out of this ), that is 7KBps or 56Kbps! Remember when the industry were trying to compress CD Audio quality ( aka 128Kbps MP3 ) down to 64…

I think Face ID could be used to create the point map, instead of generating it from an image. They could also use Face ID to prevent the tech from malicious deep fakes, e.g only allow people to use this feature I when Face ID confirms the user and the manipulated photo are the same person.

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#96

> they have managed to reduce the required bandwidth for a video call by an order of magnitude. In one example, the required data rate fell from 97.28 KB/frame to a measly 0.1165 KB/frame – a reduction to 0.1% of required bandwidth. A nitpick, perhaps, but isn't that three orders of magnitude? We've already seen people use outlandish backgrounds in calls, now it's going to be possible to design similar outlandish vie…

You just need to connect GPT3, and the dialogue is taken care of. Lyrebird’s API will take care of the speech synthesis. Viola! My deep fake can stand in at meetings now while I code.

GPT will also code for you.

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#97

What I don't like about AI processed images is that they are not real. I can't go past the fact that I am not looking at the picture as it looks like in reality but somehow smart approximation of the world that is not necessarily true.

your brain does lots of preprocessing to create a stable view of "reality", glossing over blind spots, retinal blood flow, etc. in a way we are always hallucinating, and it seems natural that computers should learn what we are actually paying attention to in video, and focus on that

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#98

What I don't like about AI processed images is that they are not real. I can't go past the fact that I am not looking at the picture as it looks like in reality but somehow smart approximation of the world that is not necessarily true.

It could be bad for telemedicine, since the NN may gloss over some "weird" pathological features not found in its training set. The current setup involving an initial image and keypoints should be fine though, as long as the initial image (keyframe?) is not messed with and is transmitted frequently enough.

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#100
People are extremely sensitive to subtleties in mouth articulation which facial landmark tracking tends to have trouble capturing. I question whether just a keyframe and facial landmarks are enough to generate convincing lip sync or gaze. I suspect that this is why the majority of the samples in the video are muted, which is a trick commonly used by facial performance capture researchers to hide bad lip sync results.
Post reply on HN