Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

VideoGigaGAN: Towards detail-rich video super-resolution

videogigagan.github.io

Post reply on HN
Print view View original

61–70 of 243 posts

Re: VideoGigaGAN: Towards detail-rich video super-resolution

#61

Post by incorrecthorse »

kfarr wrote: post #46

This is amazing and all but at what point do we reach the point of there is no more “real” data to infer from low resolution? In other words there are all sorts of information theory research on the amount of unique entropy on a given medium and even with compression there is a limit. How does that limit relate to work like this? Is there a point at which it can say we know it’s inventing things beyond x scaling cons…

I'm not sure information theory deals with this question.

Since this isn't lossless decompression, the point of having no "real" data is already reached. It _is_ inventing things, and the only relevant question is how plausible are the things being invented; in other words, if the video also existed in higher resolution, how close would it actually look like the inferred version. Seems obvious that this metric increases as a function of the amount of information from the source, but I would guess the exact relationship is a very open question.

Re: VideoGigaGAN: Towards detail-rich video super-resolution

#62

Post by lupusreal »

j45 wrote: post #57
herculity275 wrote: post #43

Wonder how long until Hollywood CGI shops have these types of models running as part of their post-production pipeline. Big blockbusters often release with ridiculously broken CGI due to crunch (Black Panther's third act was notorious for looking like a retro video-game), adding some extra generative polish in those cases is a no-brainer.

A few years if not less. They will have huge budgets for compute and the makers of compute will be happy to absorb those budgets. Cloud production was already growing but this will continue to accelerate it imho

Wasn't Hollywood an early adopter of advanced AI video stuff, w.r.t. de-aging old famous actors?

Re: VideoGigaGAN: Towards detail-rich video super-resolution

#63

Post by constantcrying »

sharpshadow wrote: post #56
turnsout wrote: post #4

Earlier quoted context omitted.

I agree that it's not perfect, though it does appear to be SoTA. Eventually something like this will just be part of every video codec. You stream a 480p version and let the TV create the 4K detail.

That’s a good idea it would save a lot of bandwidth and could be used to buffer drops while keeping the quality.

A better idea would obviously be to enhance the compressed data.

Re: VideoGigaGAN: Towards detail-rich video super-resolution

#64

Post by sharpshadow »

Jabrov wrote: post #20
constantcrying wrote: post #6

Earlier quoted context omitted.

Why would you ever do that? If you have the high res data you can actually compress the details which are there and then recreate them. No need to have those be recreated, when you actually have them. Downscaling the images and then upscaling them is pure insanity when the high res images are available.

There's lots of videos where there isn't high res data available

Satellite data would benefit from that.

Re: VideoGigaGAN: Towards detail-rich video super-resolution

#66

Post by itishappy »

It's impressive, but still looks kinda bad?

I think the video of the camera operator on the ladder shows the artifacts the best. The main camera equipment is no longer grounded in reality, with the fiddly bits disconnected from the whole and moving around. The smaller camera is barely recognizable. The plant in the background looks blurry and weird, the mountains have extra detail. Finally, the lens flare shifts!

Check out the spider too, the way the details on the leg shift is distinctly artificial.

I think the 4x/8x expansion (16x/64x the pixels!) is pushing the tech too far. I bet it would look great at <2x.

Re: VideoGigaGAN: Towards detail-rich video super-resolution

#69

Post by lupusreal »

anvuong wrote: post #58
metalrain wrote: post #7

Video quality seems really good, but limitations are quite restrictive "Our model encounters challenges when processing extremely long videos (e.g. 200 frames or more)". I'd say most videos in practice are longer than 200 frames, so lot more research is still needed.

At 24fps that's not even 10 seconds. Calling it extremely long is kinda defensive.

10 seconds is what, about a dozen cuts in a modern movie? Much longer has people pulling out their phones.
Post reply on HN