Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

github.com

Post reply on HN
Print view View original

81–90 of 129 posts

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#81

Post by sp332 »

atrilumen wrote: post #49
pjeide wrote: post #23

What media would someone collect now to be used in the future to reproduce the likeness of loved ones? Video clips of them moving? Talking? Different poses of pictures? Reading the dictionary out loud to get vocal patterns? Heck with impersonating the POTUS. What about a lost friend, sibling or parent?

Wouldn't that be some kind of horrific emotional uncanny valley?

Yeah. https://www.youtube.com/watch?v=fkE6RBlfbXA This isn't the worst one, there was a time when PKD's daughter interviewed the head and he went off on a rant about how much he disliked his family. That was really rough.

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#82

Post by huling0 »

sp332 wrote: post #79
huling0 wrote: post #11

From the paper: "we have to use all the GPUs in DGX1 (8 V100 GPUs, each with 16GB memory) for training. We distribute the generator computation task to 4 GPUs and the discriminator computation task to the other 4 GPUs. Training takes ∼10 days for 2K resolution." As I don't have a DGX1 here, training the 2K resolution net for 10 days on a p3.16xlarge instance (also has 8 V100 GPUs) would cost USD 5875 on AWS. (USD24.4…

The DXG-1 costs $129,000 so AWS is cheaper unless you need to do it 22 times. And you can have multiple instances running at once and get all of your results in ten days, instead of waiting ten days again for each run.

yeah, but does not look like something many hobbyists would try for fun ;-)

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#83

Post by bsenftner »

davedx wrote: post #70
iforgotpassword wrote: post #7

I'm probably being captain obvious here, but if this is what's being released for free, I wonder how much better a polished commercial version does, and when we reach the point where we can't trust anything we see anymore. It doesn't even have to be super perfect, even reaching the point where it takes experts about two weeks to determine if something's real or not might already be long enough to do great damage. Fro…

Yeah, the Tweet I found this from had a similar sentiment: https://twitter.com/PiratePartyINT/status/104296466807811686... "Starting now, we cannot trust video or audio evidence. The ramifications for our legal & political systems will not be known for many years"

"Starting now"? How's about 5 to 10 years ago. This is being released free now, which indicates to me this is now disposable tech and the authors have much, much better in their labs.

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#84

Post by exit »

Findeton wrote: post #22

Some applications for this kind of tech: - Porn, yeah, first application you can think of, there are already some startups doing it. - Doubling actors, and applied to sound, maybe you could translate from one language to another but kind of keeping the accent and tone. - Propaganda and misinformation. Now you can get your enemy to say and do whatever you want, on video. - Photo-realistic games. Create a rough 3D mode…

regarding photo-realistic games, can anyone comment on how quickly this the model executes?

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#85

Post by TeMPOraL »

TomMarius wrote: post #41
pbhjpbhj wrote: post #33

Earlier quoted context omitted.

Are you sure they're amateurs, not professionals don't it for fun (/pro boner/!?!).

I meant something a little different. Any random dude with a random mid-level PC can download the software and produce amazing results without any special hardware or knowledge. I recommend you to try it out for yourself.

Was there a huge progress made since news about deepfakes broke early this year?

Around March/April this year, I actually did download the TensorFlow toolkit for face transfer that was used by /r/deepfakes people and tested it out (there were samples of photos of politicians included); the results were, at best, worse that I could do in 2 minutes in Gimp. Maybe they could get better if I had an expensive GPU farm at my disposal, but I'm pretty doubtful - given that the news died down pretty quickly, and no reasonable-quality faked pictures or videos were reported ever since.

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#86

Post by nickbecker »

yonkshi wrote: post #75
iforgotpassword wrote: post #7

I'm probably being captain obvious here, but if this is what's being released for free, I wonder how much better a polished commercial version does, and when we reach the point where we can't trust anything we see anymore. It doesn't even have to be super perfect, even reaching the point where it takes experts about two weeks to determine if something's real or not might already be long enough to do great damage. Fro…

AI can also be used to identify these fake videos, and they are probably better than human identification. There will be a rise of AI forensics. The type of neural network they use (GAN) works by having two networks battling each other, one tries to generate fake, and another (discriminator) tries to identify fake, it's a constant arms race. As the generator gets better, the discriminator also gets better. Which mean…

This is how their model was trained, but I think what you've said may not quite be the case.

Because the discriminator (D) and generator (G) usually compete in a minimax game, the equilibrium probability of D correctly classifying an image as fake tends to 1/2 (ignoring distributional factors). If the competing networks have enough capacity and can be stably trained, then in theory they will reach equilibrium as the data distribution from G converges to the actual data distribution. If this is the case, then the discriminator correctly identifies fake videos with a probability of 1/2.

They may not reach equilibrium (making D > 0.5), but it's not clear that the discriminator itself is a panacea for identifying fake videos/images.

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#87

Post by wongarsu »

questionasked wrote: post #69
bitL wrote: post #66

Earlier quoted context omitted.

DRM will be pushed hard, starting from video/audio acquisition, perhaps assisted by blockchain to keep footage verified at all processing steps.

Don't you think that a blockchain that works for anything other than a rather useless currency should be created before suggesting one for such a use? I see comments all the time about how we should use blockchain for this and that, and yet far simpler uses for blockchain haven't yet worked out.

A blockchain is actually a reasonable part of the solution here (just not nessesarily creating a new one). What a blockchain really does is decentralized timestamping. Usually you use it to combine it with a bit of cryptography to process flows of money, but in the same blockchain you can just write "I know hash XXXX", and if later you produce a work with that hash you can prove that it already existed at the point your message was written in the blockchain.

That's not all you would need for verification, but it is a big help.

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#88

Post by wongarsu »

thanatropism wrote: post #68
NullPrefix wrote: post #48

Earlier quoted context omitted.

MS Office is still miles ahead in the benchmark of opening it's own proprietary files.

There is no competitor, proprietary or open, that comes close to Excel. It's been relentlessly, extensively polished for years and years, and keeps gaining new features every year. And this sticking to the spreadsheet concept, which is very limiting. --- Contrast for example Tableau -- it's a great idea and generated a lot of enthusiasm for a while, but never quite took off as an office package one needs to have. The…

Tableau is great, but it has a much narrower use case: given one or more tables of data, generate graphs for presentation or for exploring the dataset.

In comparison, Excel can do that too (just worse), but it can also solve equations, do your company's bookkeeping, and pretty much every other task that relies mostly on numbers.

I would argue Open/LibreOffice Calc comes fairly close to Excel if you ignore the worse user interface (which is fair in the original assertion that it's "mostly polish and small incremental improvements")

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#89

Post by currymj »

bsenftner wrote: post #83
davedx wrote: post #70

Earlier quoted context omitted.

Yeah, the Tweet I found this from had a similar sentiment: https://twitter.com/PiratePartyINT/status/104296466807811686... "Starting now, we cannot trust video or audio evidence. The ramifications for our legal & political systems will not be known for many years"

"Starting now"? How's about 5 to 10 years ago. This is being released free now, which indicates to me this is now disposable tech and the authors have much, much better in their labs.

Probably not; the GAN technique wasn't even published until 2014, and this is based on other work that's been done even more recently. The field moves fast and is more open than you might expect (because it's still so academic).

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#90

Post by twtw »

huling0 wrote: post #11

From the paper: "we have to use all the GPUs in DGX1 (8 V100 GPUs, each with 16GB memory) for training. We distribute the generator computation task to 4 GPUs and the discriminator computation task to the other 4 GPUs. Training takes ∼10 days for 2K resolution." As I don't have a DGX1 here, training the 2K resolution net for 10 days on a p3.16xlarge instance (also has 8 V100 GPUs) would cost USD 5875 on AWS. (USD24.4…

Which is why it's nice that the pre-trained models are available.

If you want to train on your own dataset, that price does not seem unreasonable to me.

Post reply on HN