Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia Uses AI to Slash Bandwidth on Video Calls

petapixel.com

Post reply on HN
Print view View original

181–190 of 197 posts

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#181

Post by ghaff »

colejohnson66 wrote: post #106
mjburgess wrote: post #67

Earlier quoted context omitted.

If I had to guess, the issue around "black people" is that photos are 2D. We don't really understand just how little information is actually in a photo (we add huge amounts of info in our perception). My guess is that predictive systems are using contrast as a guide to essentially 3D structures which, simply, just cannot be reconstructed from 2D. And therefore, probably struggle more on dark faces which have differen…

While contrast is almost certainly part of it, I’d hazard a guess that the training set is also partly to blame. Now, I don’t know much about neural networks (AI), but my understanding is that if you provide a training set representative of the population makeup, (in America at least) it’ll be biased towards white people as it hasn’t “seen” enough black person images. My limited understanding would then make me think…

I imagine you're right on both counts. Contrast is certainly an issue. You definitely have to make adjustments to photograph dark skin (or animals with black fur).

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#182

Post by Animats »

This is a lot like Framefree.[1] That was developed around 2005 at Kerner Optical, which was a spinoff from Lucasfilm. The system finds a set of morph points in successive keyframes and morphs between them. This can do slow motion without jerkyness, and increase frame rate. Any modern GPU can do morphing in real time, so playback is cheap. There used to be a browser plug-in for playing Framefree-compressed video.

Compression was expensive, because finding good morph points is hard. But now hardware has caught up to doing it in real time on cheap hardware.

As a compression method, it's great for talking heads with a fixed camera. You're just sending morph point moves, and rarely need a new keyframe.

You can be too early. Kerner Optical went bust a decade ago.

[1] https://youtu.be/VBfss0AaNaU

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#184

Post by zimpenfish »

m463 wrote: post #40
zimpenfish wrote: post #15

Earlier quoted context omitted.

> A nitpick, perhaps, but isn't that three orders of magnitude? I dunno that I'd call it three orders - it's close at about 830x - but it's definitely not even close to being one order either.

I think it is three orders of magnitude. "An order-of-magnitude difference between two values is a factor of 10. For example, the mass of the planet Saturn is 95 times that of Earth, so Saturn is two orders of magnitude more massive than Earth." https://en.wikipedia.org/wiki/Order_of_magnitude

> "More precisely, the order of magnitude of a number can be defined in terms of the common logarithm, usually as the integer part of the logarithm, obtained by truncation."

    $ bc -l
    l(835)/l(10)
    2.92168647548360208478
That would make it 2 orders of magnitude by that method. Happy to accept that it's 3 orders of magnitude by the N=a*10^b method though. Either way, it's definitely not one.

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#185

Post by EricMausler »

OrderlyTiamat wrote: post #12

> they have managed to reduce the required bandwidth for a video call by an order of magnitude. In one example, the required data rate fell from 97.28 KB/frame to a measly 0.1165 KB/frame – a reduction to 0.1% of required bandwidth. A nitpick, perhaps, but isn't that three orders of magnitude? We've already seen people use outlandish backgrounds in calls, now it's going to be possible to design similar outlandish vie…

"Its just easier to apply the than it is to apply foundation. Its how I'd look anyway" - delusional early adopters probably

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#186

Post by EricMausler »

dweekly wrote: post #104
rhn_mk1 wrote: post #51

Earlier quoted context omitted.

The avatar thing isn't one-sided either: it'd be an awesome power to have to remake others! Real-time silly hats for people I talk to and I'm sold.

"Visualize your audience naked." they said. "Helps calm the nerves." They had no idea.

When there's a will there's a way

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#187

Post by noxToken »

Traster wrote: post #177
conradludgate wrote: post #175

I see a lot of people being alienated by the fact that people could take on different avatars during their meeting. I would honestly accept that with no question. In a work environment, I would expect the person I'm talking to to be presentable, ie their avatar would be presentable, so no goofy backgrounds or annoying accessories. But the key for me is, I'd actually have something to see. So often in my work in in me…

This is why I keep my camera on, so that whoever is talking sees me nodding away whilst they talk and when they go "any questions" you can pause and say "None from me". It's also why you always unmute yourself when someone joins the call - yes, we can hear you, yes your mic is fine, no you're not muted, it's just no one is replying becuase they're all muted and causedoing something else because we haven't started yet…

I assume it's because people are awaiting their return to physical work environments. If your employer or team hasn't done anything to facilitate working remotely other than moving meetings to Zoom/WebEx/Meet, then you're right: people haven't put thought into video conferences.

There are numerous articles on why this impromptu remote environment isn't the same as traditional remote environments. Are people turning on their camera during meetings? Are people actually responding when talked to during meetings? Is there some type of plan outlined for team members who have kids in virtual school or are unexpected caretakers? Have teams been given the proper collaborative tools to work together remotely? Are working hours being respected?

There are lots of things people didn't think about when places went remote. The problem is that they never went back to address them either.

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#188

Post by karthikb »

3ch0 wrote: post #112
etcet wrote: post #21

A technology very similar to this plays a plot point in Vernor Vinge's 1992 novel A Fire Upon the Deep. In his universe, both the interstellar net and combat links between ships are low bandwidth. Hence, video is interpolated between sync frames or recreated from old footage. Vinge calls the resulting video "evocations".

There is also a similar technoligy in Rob Reids After On book. The AI has the ability in thet book to "refocus" the person so that they are looking into the camera. I believe this is huge and would create higher engagement if everybody was acutally looking into the camera instead of to the side or up all the time. Creating a more human an emotional relation with the people you are talking to.

Kind of like this?

https://techxplore.com/news/2019-06-intel-eye-contact-video-...

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#189

Post by TuringNYC »

ipsum2 wrote: post #178
TuringNYC wrote: post #48

Earlier quoted context omitted.

You just need to connect GPT3, and the dialogue is taken care of. Lyrebird’s API will take care of the speech synthesis. Viola! My deep fake can stand in at meetings now while I code.

But who's playing the viola?

Sorry, misspelled Voila!

Re: Nvidia Uses AI to Slash Bandwidth on Video Calls

#190

Post by ssivark »

alkonaut wrote: post #151
varispeed wrote: post #95

What I don't like about AI processed images is that they are not real. I can't go past the fact that I am not looking at the picture as it looks like in reality but somehow smart approximation of the world that is not necessarily true.

If the sending machine (that films my face) does the same decoding I know will be on the other end, then diffs the raw video with the decoded video and finally sends both things, then the receiver should be able to always piece together a 100% reproduction of the actual video feed on my end. The transmitter can always predict exactly what the receiver will decode, so the correct amount of data to send is the amount o…

You’re thinking along the right lines, but the challenge is that a raw diff will have the same number of pixels as the raw image, so no compression in bandwidth. So, how do we represent the diff/residue also with fewer numbers? At that point it’s the same as choosing better parameters within some clever encoding (be it pre-designed like JPEG or H.264 or learned via ML).
Post reply on HN