Live data from Hacker News

Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

github.com

101–110 of 129 posts

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#101
Does anybody know a) the performance (e.g. introduced latency) and processor requirements for the client/input (e.g. is real time canny edge detection good enough and how fast would that run)? and

b) what the latency impact on the NN side to build the images (e.g. how many ms are we talking about?)

Thank a lot!

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#103
post #102

Kind of surprised this stuff hasn't totally blown away traditional video compression techniques yet.

I suspect that a combination of the two is where it's at, ie, store a lossy classical compressed version, then remove the artifacts/dream up details with deep learning

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#104
post #95
post #40

Earlier quoted context omitted.

> Photo-realistic games. Create a rough 3D model of an scenario and train the AI for it. Instead of photo-realistic rendering with math, render it with the AI based on a rough render, in real-time. According to last month's nvidia rtx presentation/launch event [1], they are going to do something similar quite soon. Games will ship with DNN pre-trained offline on extremely high quality renderings. Game itself renders…

I wonder, since the NN cores of the GPU are used for real-time raytracing, will they be able to run custom NNs possibly not related to visual stuff in parallel to the ray-tracing stuff? Edit : found the answer on Internet, apparently the RT (raytracing) cores are different and separated from the Tensor (NN) cores on the RTX

I think this is about trading storage for computation - you replace terabytes of model/texture data with a compute heavy NN.

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#105

I'm probably being captain obvious here, but if this is what's being released for free, I wonder how much better a polished commercial version does, and when we reach the point where we can't trust anything we see anymore. It doesn't even have to be super perfect, even reaching the point where it takes experts about two weeks to determine if something's real or not might already be long enough to do great damage. Fro…

If you compare OSS or free software to commercial software generally, I don’t think there are that many massive gaps. It’s mostly polish and small incremental improvements, but the underlying tech is mostly the same. Why would that be different in this case?

The gaps are massive in several electrical/silicon CAD verticals and simulation. Due to IP secrecy, no OSS Verilog/VHDL synthesis alternative exists for Vivado/ISE/Quartus and the Intel/Xilinx line of FPGAs. I don't think any practical alternatives exist for Mentor Graphics' line of silicon design or simulation software, nor have I seen any OS software capable of complex mixed domain simulations like COMSOL or Ansys - many of the pieces exist, but it takes a lot of work to verify that algorithmic physical models actually work together.

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#106
post #70

Earlier quoted context omitted.

Yeah, the Tweet I found this from had a similar sentiment: https://twitter.com/PiratePartyINT/status/104296466807811686... "Starting now, we cannot trust video or audio evidence. The ramifications for our legal & political systems will not be known for many years"

"Starting now"? How's about 5 to 10 years ago. This is being released free now, which indicates to me this is now disposable tech and the authors have much, much better in their labs.

Not sure it works this way except in extremely specific areas of research (and I'm not even sure about that).

Publicly available research is usually the cutting edge in most areas of knowledge.

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#107

Earlier quoted context omitted.

There is no competitor, proprietary or open, that comes close to Excel. It's been relentlessly, extensively polished for years and years, and keeps gaining new features every year. And this sticking to the spreadsheet concept, which is very limiting. --- Contrast for example Tableau -- it's a great idea and generated a lot of enthusiasm for a while, but never quite took off as an office package one needs to have. The…

Tableau is great, but it has a much narrower use case: given one or more tables of data, generate graphs for presentation or for exploring the dataset. In comparison, Excel can do that too (just worse), but it can also solve equations, do your company's bookkeeping, and pretty much every other task that relies mostly on numbers. I would argue Open/LibreOffice Calc comes fairly close to Excel if you ignore the worse u…

> if you ignore the worse user interface

considering that's a major part of "better" that's big ask!

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#108
post #2

Am i reading that right? Its making the videos that look real, from the simplistic input? If so, that is amazing. And if so, how do I turn a video I have into a simple/line version, to be able to then put a different 'skin' on it?

Yes, also you might want to look up DLSS - they use pretrained upscaling network on GPU to generate 4K from native 1440p picture, instant ~20-50% performance bump with free AA.

https://www.youtube.com/watch?v=MMbgvXde-YA

of course this being Nvidia they didnt implement it universally, you need to sign up for API access to black box gameworx like scam programs in order to implement it in your game.

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#109
post #103
post #102

Kind of surprised this stuff hasn't totally blown away traditional video compression techniques yet.

I suspect that a combination of the two is where it's at, ie, store a lossy classical compressed version, then remove the artifacts/dream up details with deep learning

or pretrain the network on the movie you want to compress, then ship cartoon compressed version + trained network.

Re: Nvidia Vid2vid: High-resolution photorealistic video-to-video translation

#110
As someone not into AI / Machine Learning.

Can we expect CUDA to be the x86 on PC and Servers? Literally all works are defaulting to CUDA and Nvidia's library. I don't even see a contender trying compete. I don't even see AMD's ROCm being used or even mentioned anywhere.

Post reply on HN