Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Extreme video compression with prediction using pre-trainded diffusion models

github.com

Post reply on HN
Print view View original

41–50 of 89 posts

Re: Extreme video compression with prediction using pre-trainded diffusion models

#41

Post by daemonologist »

resolutebat wrote: post #37

Here's the research behind this: https://arxiv.org/html/2402.08934v1 As a casual non-scholar, non-AI person trying to parse this though, it's infuriatingly convoluted. I was expecting a table of "given source file X, we got file size Y with quality loss Z", but while quality (SSIM/LPIPS) is compared to standard codecs like H.264, for the life of me I can't find any measure of how efficient the compression is here. Ap…

I believe they're using "bpp" (bits per pixel) to indicate compression efficiency, and in the section about quality they're holding it constant at 0.06 bpp. The charts a bit further down give quality metrics as a function of compression level (however, they seem to indicate that h.264 is outperforming h.265 in their tests which would be surprising to me).

Re: Extreme video compression with prediction using pre-trainded diffusion models

#42

Post by adgjlsfhk1 »

cedricd wrote: post #22
ThisIsMyAltAcct wrote: post #17

Earlier quoted context omitted.

Someone should train a model to evaluate video compression quality

Netflix did VMAF for this: https://github.com/Netflix/vmaf It checks a reference video against an encoded video and returns a score representing how close the encoded video appears to the original from a human perspective.

that said, iiuc, SSIMULACRA 2.1 is generally considered a strictly better quality measurement.

Re: Extreme video compression with prediction using pre-trainded diffusion models

#43

Post by Leo_Germond »

orblivion wrote: post #40
polemic wrote: post #29

Earlier quoted context omitted.

How big is the SORA model itself?

I can show you an algorithm that compresses an entire 2 hour movie to a single bit, but it only works on one movie.

Ah I got it to work for two movies :)

Re: Extreme video compression with prediction using pre-trainded diffusion models

#44

Post by cstejerean »

adgjlsfhk1 wrote: post #42
cedricd wrote: post #22

Earlier quoted context omitted.

Netflix did VMAF for this: https://github.com/Netflix/vmaf It checks a reference video against an encoded video and returns a score representing how close the encoded video appears to the original from a human perspective.

that said, iiuc, SSIMULACRA 2.1 is generally considered a strictly better quality measurement.

Citation needed.

Re: Extreme video compression with prediction using pre-trainded diffusion models

#46

Post by ec109685 »

LeoPanthera wrote: post #45

It's important to remember that any compression gains must include the size of the decompressor which, I assume, will include an enormous diffusion model.

Can’t that be amortized across all videos (e.g. if YouTube had a decompressor they downloaded once)?

Re: Extreme video compression with prediction using pre-trainded diffusion models

#47

Post by LeoPanthera »

ec109685 wrote: post #46
LeoPanthera wrote: post #45

It's important to remember that any compression gains must include the size of the decompressor which, I assume, will include an enormous diffusion model.

Can’t that be amortized across all videos (e.g. if YouTube had a decompressor they downloaded once)?

Yes, absolutely, it's just important to keep in mind when thinking of these decompressors as "magic". If every laptop shipped with a copy of Wikipedia, then you could compress Wikipedia, and any text that looks similar to Wikipedia, really well.

Re: Extreme video compression with prediction using pre-trainded diffusion models

#48

Post by adgjlsfhk1 »

cstejerean wrote: post #44
adgjlsfhk1 wrote: post #42

Earlier quoted context omitted.

that said, iiuc, SSIMULACRA 2.1 is generally considered a strictly better quality measurement.

Citation needed.

https://github.com/cloudinary/ssimulacra2?tab=readme-ov-file... shows a higher correlation with human responses across 4 different datasets and correlation metrics for one.

also see https://jon-cld.s3.amazonaws.com/test/ahall_of_fshame_SSIMUL... which is an ab comparison of a lot of images where it gives 2 versions, one preferred by ssimulacra, the other preferred by vmaf

Re: Extreme video compression with prediction using pre-trainded diffusion models

#49

Post by sandkoan »

aqme28 wrote: post #21
Animats wrote: post #6

Extreme compression will be when you put in a movie and get a SORA prompt back that regenerates something close enough to the movie.

Where’s that quote? Something like “AI is just compression, and compression is indistinguishable from AI”

Ilya says this here: https://www.youtube.com/watch?v=AKMuA_TVz3A

Re: Extreme video compression with prediction using pre-trainded diffusion models

#50

Post by gcanyon »

Animats wrote: post #6

Extreme compression will be when you put in a movie and get a SORA prompt back that regenerates something close enough to the movie.

Ha, I've commented almost exactly this twice now on HN. We'll see how long before it's a reality -- probably better measured in months rather than years.
Post reply on HN