Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia Uses AI to Slash Bandwidth on Video Calls

petapixel.com

Post reply on HN
Print view View original

111–120 of 197 posts

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#111

Post by zaroth »

colejohnson66 wrote: post #101
loueed wrote: post #94

Earlier quoted context omitted.

I think Face ID could be used to create the point map, instead of generating it from an image. They could also use Face ID to prevent the tech from malicious deep fakes, e.g only allow people to use this feature I when Face ID confirms the user and the manipulated photo are the same person.

Isn’t the point map supposedly encrypted on the Secure Enclave?[a] If it is, you can’t access it from the main CPU; you can only ask it to compare a provided one with the stored one. [a]: I know the password is stored on it, but idk about the face mesh

The verifier point cloud used for unlocking the phone must be kept in the secure element.

That doesn’t mean you can’t read a point cloud for other purposes.

They currently expose ARPointCloud API but I don’t know what sensors it uses to produce it;

https://twitter.com/nobbis/status/1292262455490629633?lang=e...

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#112

Post by 3ch0 »

etcet wrote: post #21

A technology very similar to this plays a plot point in Vernor Vinge's 1992 novel A Fire Upon the Deep. In his universe, both the interstellar net and combat links between ships are low bandwidth. Hence, video is interpolated between sync frames or recreated from old footage. Vinge calls the resulting video "evocations".

There is also a similar technoligy in Rob Reids After On book. The AI has the ability in thet book to "refocus" the person so that they are looking into the camera.

I believe this is huge and would create higher engagement if everybody was acutally looking into the camera instead of to the side or up all the time. Creating a more human an emotional relation with the people you are talking to.

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#113

Post by sebular »

izacus wrote: post #93
ksec wrote: post #85

I would imagine Apple doing this with FaceTime soon. Using their own NPU ( Neural processing unit ), you can now make FaceTime call with ridiculously low bandwidth. From the Nvidia example, 0.1165 KB/frame even at buttery smooth 60fps ( I could literally hear Apple market the crap out of this ), that is 7KBps or 56Kbps! Remember when the industry were trying to compress CD Audio quality ( aka 128Kbps MP3 ) down to 64…

> I would imagine Apple doing this with FaceTime soon. What's this claim based on? Last I looked into FaceTime tech, they didn't do anything special - their quality comes from use of H.265 and the fact that iOS devices have good quality HW encoding blocks which provide good compression at low bandwidths. FaceTime stream is usually also low motion / change so it's possible to achieve very good compression even with ba…

I'm not the person you're replying to, but I think the main reason you'd expect to see this on iOS is due to the additional Face ID sensors available on most of their devices. Presumably Apple could offer a version of this tech that features higher quality tracking and thus a more realistic and natural-looking final output.

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#114

Post by no_one_ever »

OrderlyTiamat wrote: post #12

> they have managed to reduce the required bandwidth for a video call by an order of magnitude. In one example, the required data rate fell from 97.28 KB/frame to a measly 0.1165 KB/frame – a reduction to 0.1% of required bandwidth. A nitpick, perhaps, but isn't that three orders of magnitude? We've already seen people use outlandish backgrounds in calls, now it's going to be possible to design similar outlandish vie…

As these trends sort of become more and more prevalent, I am so shocked at how David Foster Wallace had nailed this prediction in his book Infinite Jest.

Humans becoming more and more dependent on virtual face-to-face meetings and also relying on embellishment of their supposed appearance through the screen. It reminds me of how SciFi authors predicted technology, but with a complimentary commentary on human psychology.

Sorry if it isn't directly related to the post, but it is so striking to me.

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#115

Post by agumonkey »

GuB-42 wrote: post #110
varispeed wrote: post #95

What I don't like about AI processed images is that they are not real. I can't go past the fact that I am not looking at the picture as it looks like in reality but somehow smart approximation of the world that is not necessarily true.

Video is not real either. The perspective is different and the colors are wrong. It is even less real after it went through lossy compression. The whole point of a lossy compression is to remove details that you perceive as unimportant. For example, leaves on a tree may look like a greeny mess, but that's fine, from afar, you don't make a difference. Using neural networks for compression is far from being a new conce…

I understand what parent means, usually processing was too linear to alter the meaning of what you see, it's distorded, altered, maybe filtered and enhanced but in general, it's the same. With ml/dl you can really be sent .. whatever.

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#117

Post by xioxox »

michaelt wrote: post #66
jstanley wrote: post #23

This includes a feature: > Called “Free View,” this would allow someone who has a separate camera off-screen to seemingly keep eye contact with those on a video call. Am I the only one who thinks eye contact on video calls feels creepy? I think I would prefer this feature to remove eye contact on video calls rather than add it.

Apparently researchers have measured this, and for people meeting in person it's normal to maintain eye contact 30%-60% of the time in a conversation [1] with each contact lasting ~3 seconds [2] So a system that maintained eye contact continuously would indeed risk looking creepy! [1] https://www.forbes.com/sites/carolkinseygoman/2014/08/21/fac... [2] https://www.businessinsider.com/heres-how-long-you-should-ho...

Indeed, have a look at what too much eye contact looks like: https://samharris.org/look-into-my-eyes/

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#119

Post by codesnik »

ksec wrote: post #85

I would imagine Apple doing this with FaceTime soon. Using their own NPU ( Neural processing unit ), you can now make FaceTime call with ridiculously low bandwidth. From the Nvidia example, 0.1165 KB/frame even at buttery smooth 60fps ( I could literally hear Apple market the crap out of this ), that is 7KBps or 56Kbps! Remember when the industry were trying to compress CD Audio quality ( aka 128Kbps MP3 ) down to 64…

> I would imagine Apple doing this with FaceTime soon.

And we'll notice it because somehow our and other people faces in facetime will start to subtly translate strangely aggravating emotions, as current memojis do.

Post reply on HN