Are there any high resolution examples available? Am I just not finding them in the README?
Nvidia Vid2vid: High-resolution photorealistic video-to-video translation
91–100 of 129 posts
Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation
#92Earlier quoted context omitted.
Yeah, the Tweet I found this from had a similar sentiment: https://twitter.com/PiratePartyINT/status/104296466807811686... "Starting now, we cannot trust video or audio evidence. The ramifications for our legal & political systems will not be known for many years"
"Starting now"? How's about 5 to 10 years ago. This is being released free now, which indicates to me this is now disposable tech and the authors have much, much better in their labs.
Usually it's the other way around with ML researchers - what they show is better than what they have in the lab :)
Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation
#93I'm probably being captain obvious here, but if this is what's being released for free, I wonder how much better a polished commercial version does, and when we reach the point where we can't trust anything we see anymore. It doesn't even have to be super perfect, even reaching the point where it takes experts about two weeks to determine if something's real or not might already be long enough to do great damage. Fro…
Yeah but it was already the case with CGI. IT's true that it's going to be easier and easier to do fake porn, fake speeches, fake voice recording, fake vides....
Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation
#94From the paper: "we have to use all the GPUs in DGX1 (8 V100 GPUs, each with 16GB memory) for training. We distribute the generator computation task to 4 GPUs and the discriminator computation task to the other 4 GPUs. Training takes ∼10 days for 2K resolution." As I don't have a DGX1 here, training the 2K resolution net for 10 days on a p3.16xlarge instance (also has 8 V100 GPUs) would cost USD 5875 on AWS. (USD24.4…
The DXG-1 costs $129,000 so AWS is cheaper unless you need to do it 22 times. And you can have multiple instances running at once and get all of your results in ten days, instead of waiting ten days again for each run.
Or you can build your own 8x2080Ti rig which will have 80% of performance for 1/10th of the cost.
Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation
#95Some applications for this kind of tech: - Porn, yeah, first application you can think of, there are already some startups doing it. - Doubling actors, and applied to sound, maybe you could translate from one language to another but kind of keeping the accent and tone. - Propaganda and misinformation. Now you can get your enemy to say and do whatever you want, on video. - Photo-realistic games. Create a rough 3D mode…
> Photo-realistic games. Create a rough 3D model of an scenario and train the AI for it. Instead of photo-realistic rendering with math, render it with the AI based on a rough render, in real-time. According to last month's nvidia rtx presentation/launch event [1], they are going to do something similar quite soon. Games will ship with DNN pre-trained offline on extremely high quality renderings. Game itself renders…
I wonder, since the NN cores of the GPU are used for real-time raytracing, will they be able to run custom NNs possibly not related to visual stuff in parallel to the ray-tracing stuff?
Edit : found the answer on Internet, apparently the RT (raytracing) cores are different and separated from the Tensor (NN) cores on the RTX
Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation
#96I am a bit surprised how shallow the comments on this one are.
Look closely, while it does generate videos of a passing similarity, they aren't "photorealistic" in the slightest. They are good locally across time and space domains, but globally they are as far from realism as Doom 2 was.
The only explanation for the attitude I see in this submission is that most IT people trained themselves to spot CGI by looking at local artifacts, assuming that global artifacts won't happen because stuff on the scene is reasonable. There is no "stuff on the scene" with those videos, it's just mindless vector manipulation with no underlying world model. Cars wave around, trees grow a feet from each other and behave in a way incompatible with 3D perspective.
Relax, it'll require at least another AI/ML revolution (or even several) to achieve photorealism.
Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation
#97Earlier quoted context omitted.
The vast bulk of Adobe's advantage is in UX, not technical algorithms. Which makes perfect sense because that tends to be the case with most F/OSS software—technically brilliant but with an face only a programmer could love. Yes, Adobe do have some remarkable algorithms that would be difficult to replicate (e.g. heal brush and content aware fill) but these are a small minority of Adobe's software advantage. The one t…
for content aware fill and heal brush both GMIC ( https://patdavid.net/2014/02/getting-around-in-gimp-gmic-inp... ) and resynthetizer ( http://www.logarithmic.net/pfh/resynthesizer ) are working quite OK.
Gimp even had content aware fill first, It was based on a SIGGRAPH paper iirc.
Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation
#98Earlier quoted context omitted.
I meant something a little different. Any random dude with a random mid-level PC can download the software and produce amazing results without any special hardware or knowledge. I recommend you to try it out for yourself.
Was there a huge progress made since news about deepfakes broke early this year? Around March/April this year, I actually did download the TensorFlow toolkit for face transfer that was used by /r/deepfakes people and tested it out (there were samples of photos of politicians included); the results were, at best , worse that I could do in 2 minutes in Gimp. Maybe they could get better if I had an expensive GPU farm at…
It's more about experimenting with your training data and other configuration. What I've heard, getting great results takes time - but it's possible. Most of the focus of the community is on porn, so I can imagine not so many journalists are checking the newest results and reporting about them.
Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation
#99Earlier quoted context omitted.
The DXG-1 costs $129,000 so AWS is cheaper unless you need to do it 22 times. And you can have multiple instances running at once and get all of your results in ten days, instead of waiting ten days again for each run.
Or you can build your own 8x2080Ti rig which will have 80% of performance for 1/10th of the cost.
DGX 1V has $7500 worth of CPU alone to feed the gpus. Throw in 8 TB of nvme ssd for training data and you're looking at something more like 1/5th the cost.
V100 has ~50% higher memory bandwidth than 2080Ti, so you probably wouldn't get 80% of the performance. Also, only two 2080Ti can be connected via nvlink.
Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation
#100Earlier quoted context omitted.
The level of realism can be gauged from the examples they provide right there on the page. Of course your results may vary basing on the initial bulk of data of realistic source images you use. You have the code right there on Github, just install it on some PC with powerful GPUs (or rent one), tune some parameters, train the network and you can do the same things.
The hardware they used costs tons of money, and so does doing on a cloud provider. It’s not something to do in a weekend with your gaming card.
It is if you drop the resolution.