Live data from Hacker News

Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

github.com

11–20 of 129 posts

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#11
From the paper: "we have to use all the GPUs in DGX1 (8 V100 GPUs, each with 16GB memory) for training. We distribute the generator computation task to 4 GPUs and the discriminator computation task to the other 4 GPUs. Training takes ∼10 days for 2K resolution."

As I don't have a DGX1 here, training the 2K resolution net for 10 days on a p3.16xlarge instance (also has 8 V100 GPUs) would cost USD 5875 on AWS. (USD24.48 per hour on-demand pricing * 24 hours/day * 10 days)

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#12
post #8

One of the example translates a full human pose to a video of a dancer. If the network would be trained on the facial pose(?) / features only, would that recreate something like the facial reenactment in http://niessnerlab.org/projects/thies2016face.html (source code for face2face is not public)?

Look into Deepfake, that's the tool 4chan is using for face swapping in their fake porn

Yes, but it just cuts out the face and pastes it on a different person/background. It does not do full reenactment where you keep the entire target video environment.

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#14

I'm probably being captain obvious here, but if this is what's being released for free, I wonder how much better a polished commercial version does, and when we reach the point where we can't trust anything we see anymore. It doesn't even have to be super perfect, even reaching the point where it takes experts about two weeks to determine if something's real or not might already be long enough to do great damage. Fro…

If you compare OSS or free software to commercial software generally, I don’t think there are that many massive gaps. It’s mostly polish and small incremental improvements, but the underlying tech is mostly the same. Why would that be different in this case?

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#15

I'm probably being captain obvious here, but if this is what's being released for free, I wonder how much better a polished commercial version does, and when we reach the point where we can't trust anything we see anymore. It doesn't even have to be super perfect, even reaching the point where it takes experts about two weeks to determine if something's real or not might already be long enough to do great damage. Fro…

If you compare OSS or free software to commercial software generally, I don’t think there are that many massive gaps. It’s mostly polish and small incremental improvements, but the underlying tech is mostly the same. Why would that be different in this case?

Adobe have definitely bested most oss competitors in their space. Although with the amount of man power at their disposal it would be hard to beat.

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#17
post #16

I'm curious what would happen if somebody tried to impersonate the US president using this. He would say it's fake, but who would believe him? How exactly can computer scientists explain deepfakes to laymen?

Well I’m this case the results are pretty good locally but have pretty obvious artefacts too. Especially the synthesised road videos, look at the trees or even more at the Lane change in the linked video.

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#18

I'm probably being captain obvious here, but if this is what's being released for free, I wonder how much better a polished commercial version does, and when we reach the point where we can't trust anything we see anymore. It doesn't even have to be super perfect, even reaching the point where it takes experts about two weeks to determine if something's real or not might already be long enough to do great damage. Fro…

It could also lead to crimes like Blackmail becoming extinct. It would be hard to hold incriminating recordings of anyone over them if near-perfect audio and video synthesis was common.

Especially for public figures with lots training data available.

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#19

I'm probably being captain obvious here, but if this is what's being released for free, I wonder how much better a polished commercial version does, and when we reach the point where we can't trust anything we see anymore. It doesn't even have to be super perfect, even reaching the point where it takes experts about two weeks to determine if something's real or not might already be long enough to do great damage. Fro…

If you compare OSS or free software to commercial software generally, I don’t think there are that many massive gaps. It’s mostly polish and small incremental improvements, but the underlying tech is mostly the same. Why would that be different in this case?

On average I agree they are on par. Sometimes OSS is better (ffmpeg), sometimes commercial stuff (adobe after effects).

In this specific case I think it might be beneficial if you can spend a lot of money on gathering training material and tweaking the network. But then again someone here mentioned the quality 4chans fake porn has reached, so maybe I'm wrong after all.

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#20
post #16

I'm curious what would happen if somebody tried to impersonate the US president using this. He would say it's fake, but who would believe him? How exactly can computer scientists explain deepfakes to laymen?

Well I’m this case the results are pretty good locally but have pretty obvious artefacts too. Especially the synthesised road videos, look at the trees or even more at the Lane change in the linked video.

Can't you easily obfuscate that by making the video intentionally grainy and low-res and passing it off as "caught by CCTV" or "found footage"?
Post reply on HN