Earlier quoted context omitted.
You mean _invents_ details.
You mean _infers_ details.
What's the distinction?
VideoGigaGAN: Towards detail-rich video super-resolution
91–100 of 243 posts
Re: VideoGigaGAN: Towards detail-rich video super-resolution
#92Is anyone else concerned at the societal effects of technology like this? In one of the examples they show a young girl. In the upscale example it's quite clearly hallucinating makeup and lipstick. I'm quite worried about tools like this perpetuating social norms even further.
Re: VideoGigaGAN: Towards detail-rich video super-resolution
#93This is great for entertainment (and hopefully the main application), but we need clear marking of such type of videos before hallucinated details are used as "proofs" of any kind by people not knowing how this works. Software video/photography on smartphones is already using proprietary algorithms that "infer" non-existent or fake details, and this would be at an even bigger scale.
Funny to think of all those scenes in TV and movies when someone would magically "enhance" a low-resolution image to be crystal clear. At the time, nerds scoffed, but now we know they were simply using an AI to super-scale it. In retrospect, how many fictional villains were condemned on the basis of hallucinated evidence? :-D
Enemy of the State (1998) was prescient, that had a ridiculous example of "zoom and enhance" where they move the camera, but they hand-waved it as the computer "hypothesizing" what the missing information might have been. Which is more or less what gaussian splat 3D reconstructions are doing today.
Re: VideoGigaGAN: Towards detail-rich video super-resolution
#94Earlier quoted context omitted.
Yes. Down sampling makes only sense if you store per pixel data, which is obviously a dumb idea. You get a stream for 480p which contains frames which were compressed from the source files, or the 4k version. At some point there might have been down sampling involved, but you never actually get any of that data, you get the compressed version of those.
Not sure if I’m being dumb, or if it’s you not explaining it clearly: if Neflix produced low resolution frames from high resolution (4k to 480p), and if these 480p frames are what my TV is receiving - are you saying it’s not downsampling, and my TV would not benefit from this new upsampling method?
Your TV never receives per pixel data. Why would you use a NN to enhance the data which your TV has constructed instead of enhancing the data it actually receives?
Re: VideoGigaGAN: Towards detail-rich video super-resolution
#95Is anyone else concerned at the societal effects of technology like this? In one of the examples they show a young girl. In the upscale example it's quite clearly hallucinating makeup and lipstick. I'm quite worried about tools like this perpetuating social norms even further.
Yes, but if you mention that here, you’ll get accused of wokeism.
More seriously, though, yes, the thing you’re describing is exactly what the AI safety field is attempting to address.
Re: VideoGigaGAN: Towards detail-rich video super-resolution
#96Is anyone else concerned at the societal effects of technology like this? In one of the examples they show a young girl. In the upscale example it's quite clearly hallucinating makeup and lipstick. I'm quite worried about tools like this perpetuating social norms even further.
I don't know, it's a mirror, right? It's up to us to change really. Besides, failures like the one you point out make subtle stereotypes and biases more conspicuous, which could be a good thing.
Re: VideoGigaGAN: Towards detail-rich video super-resolution
#97Earlier quoted context omitted.
Not sure if I’m being dumb, or if it’s you not explaining it clearly: if Neflix produced low resolution frames from high resolution (4k to 480p), and if these 480p frames are what my TV is receiving - are you saying it’s not downsampling, and my TV would not benefit from this new upsampling method?
Your TV never receives per pixel data. Why would you use a NN to enhance the data which your TV has constructed instead of enhancing the data it actually receives?
OK, I admit I don’t know much about video compression. So what does my TV receives from Netflix if it’s not pixels? And when my TV does “upsampling” (according to the marketing) what does it do exactly?
Re: VideoGigaGAN: Towards detail-rich video super-resolution
#98Curious how this compares to Topaz which is the current industry leader in the field.
Re: VideoGigaGAN: Towards detail-rich video super-resolution
#99This is amazing and all but at what point do we reach the point of there is no more “real” data to infer from low resolution? In other words there are all sorts of information theory research on the amount of unique entropy on a given medium and even with compression there is a limit. How does that limit relate to work like this? Is there a point at which it can say we know it’s inventing things beyond x scaling cons…
That point is the starting point.
There is plenty of real information: that's what the model is trained on. That information ceases to be real the moment it is used by a model to fill in the gaps of other real information. The result of this model is a facade, not real data.
Re: VideoGigaGAN: Towards detail-rich video super-resolution
#100Earlier quoted context omitted.
At 24fps that's not even 10 seconds. Calling it extremely long is kinda defensive.
The average shot length in a modern movie is around 2.5 seconds (down from 12 seconds in 1930's). For animations it's around 15 seconds.
The textures of objects need to maintain consistency across much larger time frames, especially at 4k where you can see the pores on someone's face in a closeup.