Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia Uses AI to Slash Bandwidth on Video Calls

petapixel.com

Post reply on HN
Print view View original

31–40 of 197 posts

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#31

Post by berdario »

brink wrote: post #19

Watching the video it no longer feels like you're looking at a real person, but instead just another npc. It no longer feels as personal. The last thing remote relationships need is more impersonality. I hope this is used only when it's needed.

I'm really confused. I just watched the video again, and I cannot really see the effect that you're talking about.

In fact, I struggle to see any difference from the original. It totally feels genuine, I'm not sure why you perceive them to be npcs

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#33

Post by ACow_Adonis »

mehrdadn wrote: post #10
ACow_Adonis wrote: post #9

Earlier quoted context omitted.

fair, in that they aren't perfect photorealism, but if their comparisons with the regular codecs and techniques are correct, I'd take their modelled faces over the comparable- level of digital artifacts for the same bandwidth. After all, it doesn't need to be said that when you're on a regular videoconferencing call and bandwidth starts to suffer, the resulting images don't really look anything like a photorealistic…

The thing is you only want to make this trade-off when the bandwidth is actually starting to suffer. It'd be nice if there was a nice way to make this adaptive and use NNs only when throughput is low, but the nonlinearity of the distortions makes me think this would be really hard. [1] I know what I don't want is for a normal conversation in an uncongested network to look unnatural or for facial expressions to get di…

Perhaps it's just me, and my philosophical bent, but I can actually see coming at it from the exact opposite end.

I don't want bandwidth and things spent/wasted without it providing a significant benefit (I'm probably one of those 1080p/720p is good enough for most things type guys).

I definitely don't want work making large bandwidth or resource claims on my connection when I'm working at home. And if any of this remote working has taught me anything, it's that most of my colleagues don't have steady/reliable tech or connections, so i'd almost want it used pre-emptively as a default so we can spend the rest of those resources on robustness or other qualities. (I realise of course that at the moment none of them have high-grade Nvidia graphics cards, but I'm talking hypothetically in the far off future).

In short, I want a world where the cost to benefit ratio of things is orders of magnitude larger, because things like this let us spend network/resources on things which matter.

Yes, when I'm calling my parent/grandparent one on one I might want to upgrade the signal, but I don't need to see random colleague's face in all their HD glory, or remote people whom I have no idea who they look or sound like anyway (i believe that's also been one of the findings with deepfakes, that you don't notice the eerieness/falseness as much if it's a reference of a face that you don't have pre-determined knowledge of).

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#34

Post by colechristensen »

This may be projecting expectations but the example compressed video looks very slightly fake in a way that is just a little uncanny valley type unsettling.

Perhaps the nets they’re using are compressing out facial microexpressions and when we see it, it seems just a little unnatural. Compression artifacts might be preferable because the information they’re missing is more obvious and less artificial. In other words i’d rather be presented with something obviously flawed than something i can’t quite tell what is wrong.

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#35

Post by berdario »

mehrdadn wrote: post #5

Is it just me or do the videos no longer look natural? I feel like I see highly non-linear movements (parts of the video moving when they shouldn't, or vice-versa), and facial expressions don't really look quite the same.

They look perfectly natural to me

I tried to pay attention to what you and others flagged (parts not moving, facial expressions, shoulders, etc.) But I cannot spot anything out of place.

That said, looking at other comments here, you're definitely not the only one

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#36

Post by m463 »

qwerty456127 wrote: post #30

Now the person you are speaking to is going to be n% (partially) emulated. n is going to increase in future. One day there will be a paid feature letting you emulate 100% of yourself to respond to video calls when you are not available. And finally, they will replace yourself even without you knowing, and even after you die.

That's an interesting perspective.

The bandwidth and requirement to participate correlate.

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#37

Post by colechristensen »

berdario wrote: post #31
brink wrote: post #19

Watching the video it no longer feels like you're looking at a real person, but instead just another npc. It no longer feels as personal. The last thing remote relationships need is more impersonality. I hope this is used only when it's needed.

I'm really confused. I just watched the video again, and I cannot really see the effect that you're talking about. In fact, I struggle to see any difference from the original. It totally feels genuine, I'm not sure why you perceive them to be npcs

I think it would be easier to see if they had a comparison between a high bit rate traditionally compressed video and their neural net compressed video. To me the facial shapes and movement seem just a little bit wrong like you’re looking at a 3D animation of a person instead of a video.

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#38

Post by jamra »

jonplackett wrote: post #16

I wonder how weird it gets when you turn your head too much. This is very cool though - I was expecting to be able to tell a difference and maybe slip into uncanny valley territory but it looks good. Big question though - is this just substituting the problem of not having good internet with not having a really fast nVidia graphics card?

They have an example of head orientation on their site. It moves the head to look at the camera.

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#39

Post by jamra »

colechristensen wrote: post #37
berdario wrote: post #31

Earlier quoted context omitted.

I'm really confused. I just watched the video again, and I cannot really see the effect that you're talking about. In fact, I struggle to see any difference from the original. It totally feels genuine, I'm not sure why you perceive them to be npcs

I think it would be easier to see if they had a comparison between a high bit rate traditionally compressed video and their neural net compressed video. To me the facial shapes and movement seem just a little bit wrong like you’re looking at a 3D animation of a person instead of a video.

They do this using GANs so I don’t think it is compressed in any way. Instead, the points on the face are sent across the net and the face is actually generated. I’m sure it can get better with time. There is a site thispersondoesnotexist.com that uses the same tech but looks real.

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#40

Post by m463 »

zimpenfish wrote: post #15
OrderlyTiamat wrote: post #12

> they have managed to reduce the required bandwidth for a video call by an order of magnitude. In one example, the required data rate fell from 97.28 KB/frame to a measly 0.1165 KB/frame – a reduction to 0.1% of required bandwidth. A nitpick, perhaps, but isn't that three orders of magnitude? We've already seen people use outlandish backgrounds in calls, now it's going to be possible to design similar outlandish vie…

> A nitpick, perhaps, but isn't that three orders of magnitude? I dunno that I'd call it three orders - it's close at about 830x - but it's definitely not even close to being one order either.

I think it is three orders of magnitude.

"An order-of-magnitude difference between two values is a factor of 10. For example, the mass of the planet Saturn is 95 times that of Earth, so Saturn is two orders of magnitude more massive than Earth."

https://en.wikipedia.org/wiki/Order_of_magnitude

Post reply on HN