Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Extreme video compression with prediction using pre-trainded diffusion models

github.com

Post reply on HN
Print view View original

81–89 of 89 posts

Re: Extreme video compression with prediction using pre-trainded diffusion models

#81

Post by polemic »

snovv_crash wrote: post #66
polemic wrote: post #64

Earlier quoted context omitted.

Meh, AI doesn't break information theory. The relationship between the prompt size to the "similarity" of the result will be such that it doesn't beat traditional compression techniques. At best we might consider it a new type of lossy (or... replacey?) compression. Of course if storage / RAM / bandwidth keeps increasing, this is quite likely the least energy efficient technique available.

If the compression can take into account the sub-manifold of potential outputs that people would actually be interested in watching a movie about it can achieve enormously higher compression than if it doesn't know about this.

Unproven - but yes like I say - a new type of compression

Re: Extreme video compression with prediction using pre-trainded diffusion models

#82

Post by nuancebydefault »

david-gpu wrote: post #77
leoff wrote: post #70

Earlier quoted context omitted.

AI is always deterministic. We add noise to the models to get "non-deterministic" results, but if the noise and input is the same, the output is also the same.

It's a bit more nuanced than that. Floating point arithmetic is not associative: "(A+B)+C" is not always equal to "A+(B+C)". Because of that, certain mathematical operations used in neural networks, such as parallel reductions, will yield slightly different results if you run them multiple times with the same arguments. There are some people working hard to provide the means to perform deterministic AI computations l…

Is AI in general relying on floating point calculations?

Re: Extreme video compression with prediction using pre-trainded diffusion models

#83

Post by nuancebydefault »

orblivion wrote: post #40
polemic wrote: post #29

Earlier quoted context omitted.

How big is the SORA model itself?

I can show you an algorithm that compresses an entire 2 hour movie to a single bit, but it only works on one movie.

I think the most impressive movie compression technique is youtube. It compresses movies into a short sequence of characters denoted as URLs.

Re: Extreme video compression with prediction using pre-trainded diffusion models

#84

Post by astrange »

ThisIsMyAltAcct wrote: post #17
astrange wrote: post #8

Earlier quoted context omitted.

Graphs (especially PSNR) aren't a good way to judge video compression. It's better to just watch the video. Many older/commercial video codecs optimized for PSNR, which results in the output being blurry and textureless because that's the best way to minimize rate for the same PSNR.

Someone should train a model to evaluate video compression quality

That wouldn't work forever due to Goodhart's law.

Re: Extreme video compression with prediction using pre-trainded diffusion models

#85

Post by david-gpu »

nuancebydefault wrote: post #82
david-gpu wrote: post #77

Earlier quoted context omitted.

It's a bit more nuanced than that. Floating point arithmetic is not associative: "(A+B)+C" is not always equal to "A+(B+C)". Because of that, certain mathematical operations used in neural networks, such as parallel reductions, will yield slightly different results if you run them multiple times with the same arguments. There are some people working hard to provide the means to perform deterministic AI computations l…

Is AI in general relying on floating point calculations?

One hundred percent. It's mostly fancy floating point matrix multiplications.

Re: Extreme video compression with prediction using pre-trainded diffusion models

#86

Post by Rygian »

leoff wrote: post #70
Rygian wrote: post #68

Earlier quoted context omitted.

How does that make sense? Compression is deterministic (for same prompt, same output is algorithmically guaranteed). AI is only deterministic in corner cases.

AI is always deterministic. We add noise to the models to get "non-deterministic" results, but if the noise and input is the same, the output is also the same.

That assumes an AI that is trained exactly once ever.

Re: Extreme video compression with prediction using pre-trainded diffusion models

#87

Post by snovv_crash »

polemic wrote: post #81
snovv_crash wrote: post #66

Earlier quoted context omitted.

If the compression can take into account the sub-manifold of potential outputs that people would actually be interested in watching a movie about it can achieve enormously higher compression than if it doesn't know about this.

Unproven - but yes like I say - a new type of compression

Not new. If you have a preshared dictionary even zstd can give you better compression.

Re: Extreme video compression with prediction using pre-trainded diffusion models

#88

Post by GuB-42 »

daveguy wrote: post #33
Vecr wrote: post #32

Earlier quoted context omitted.

> A lossless compression contest to encourage research in AI. It's lossless, I think just to standardize scoring, but I always thought a lossy version would be better for AI -- our memories are definitely lossy! Gwern posts about this when people say something like that on here, but I'll do it instead. Lossless encoding is just lossy encoding + error correction of some sort.

Hah. "error correction of some sort" is doing a lot of heavy lifting there. Bit level correction is what I generally consider under the "error correction" umbrella. Which we have no problem with -- curious the bit error rate that would make text unreadable. In the context of compressing a large part of the English language wiki -- I think lossy also includes loss so significant that you wouldn't be able to reproduce…

The way we learn exact sentences is usually by getting an intuitive sense and applying corrections.

For example, to memorize "Doggo woofs at kity", we first get the concept of "dog barks at cat", it compresses well because intuitively, we know that dogs bark and cats are common targets. That's our lossy compression and we could stop there but it is only part of the story. It is not a "dog" but a "doggo", and it goes well with the familiar tone, a good compression algorithm will take only a few bits for that. Then there is the typo "kity" vs "kitty", it will take a bit of extra space, but again, a good algorithm will recognize the common typos and compress even that. So it means the entire process to lossless matters, lossy is just stopping halfway.

And if there is pure random noise remaining, there is nothing you can do, but all algorithms are on an equal footing here. But the key is to make what the algorithm consider as uncompressible noise as small as possible.

Re: Extreme video compression with prediction using pre-trainded diffusion models

#89

Post by cstejerean »

adgjlsfhk1 wrote: post #48
cstejerean wrote: post #44

Earlier quoted context omitted.

Citation needed.

https://github.com/cloudinary/ssimulacra2?tab=readme-ov-file... shows a higher correlation with human responses across 4 different datasets and correlation metrics for one. also see https://jon-cld.s3.amazonaws.com/test/ahall_of_fshame_SSIMUL... which is an ab comparison of a lot of images where it gives 2 versions, one preferred by ssimulacra, the other preferred by vmaf

The authors of the metric found some cases where it works better is not the same thing as it being widely considered to be better. When it comes to typical video compression and scaling artifacts VMAF does really well. To prove something is better than VMAF on video compression it should be compared on datasets like MCL-V, BVI-HD, CC-HD, CC-HDDO, SHVC, IVP, VQEGHD3 and so on (and of course Netflix Public).

TID2013 for example is an image dataset with many artifacts completely unrelated to compression and scaling.

- Additive Gaussian noise - Additive noise in color components is more intensive than additive noise in the luminance component - Spatially correlated noise - Masked noise - High frequency noise - Impulse noise - Quantization noise - Gaussian blur - Image denoising - JPEG compression - JPEG2000 compression - JPEG transmission errors - JPEG2000 transmission errors - Non eccentricity pattern noise - Local block-wise distortions of different intensity - Mean shift (intensity shift) - Contrast change - Change of color saturation - Multiplicative Gaussian noise - Comfort noise - Lossy compression of noisy images - Image color quantization with dither - Chromatic aberrations - Sparse sampling and reconstruction

Doing better on TID2013 is not really an indication of doing better on a video compression and scaling dataset (or being more useful for making decisions for video compression and streaming).

Post reply on HN