Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Extreme video compression with prediction using pre-trainded diffusion models

github.com

Post reply on HN
Print view View original

31–40 of 89 posts

Re: Extreme video compression with prediction using pre-trainded diffusion models

#31

Post by solardev »

gedy wrote: post #28
IgorPartola wrote: post #13

Earlier quoted context omitted.

“Alexa show me Star Wars but with Dustin Hoffman as Luke”.

I actually would really like this flexibility. "Star Wars, but in Korean with k-pop stars cast", etc.

Or as a VR game. "Star Wars, but with the Empire and Rebel Alliance teaming up to defeat the latest threat to the galaxy: me as Jar-Jar Binks, Jedi Jester. My abilities include Force Juggle, Failed Comedic Relief, and Turn Into Merchandise. Oh, and Darth Vader is Morgan Freeman, and everyone else is Natalie Portman."

Re: Extreme video compression with prediction using pre-trainded diffusion models

#32

Post by Vecr »

daveguy wrote: post #23
aqme28 wrote: post #21

Earlier quoted context omitted.

Where’s that quote? Something like “AI is just compression, and compression is indistinguishable from AI”

I'm not sure the quote, but you're probably thinking something related to the Hutter Prize: https://en.m.wikipedia.org/wiki/Hutter_Prize A lossless compression contest to encourage research in AI. It's lossless, I think just to standardize scoring, but I always thought a lossy version would be better for AI -- our memories are definitely lossy!

> A lossless compression contest to encourage research in AI. It's lossless, I think just to standardize scoring, but I always thought a lossy version would be better for AI -- our memories are definitely lossy!

Gwern posts about this when people say something like that on here, but I'll do it instead. Lossless encoding is just lossy encoding + error correction of some sort.

Re: Extreme video compression with prediction using pre-trainded diffusion models

#33

Post by daveguy »

Vecr wrote: post #32
daveguy wrote: post #23

Earlier quoted context omitted.

I'm not sure the quote, but you're probably thinking something related to the Hutter Prize: https://en.m.wikipedia.org/wiki/Hutter_Prize A lossless compression contest to encourage research in AI. It's lossless, I think just to standardize scoring, but I always thought a lossy version would be better for AI -- our memories are definitely lossy!

> A lossless compression contest to encourage research in AI. It's lossless, I think just to standardize scoring, but I always thought a lossy version would be better for AI -- our memories are definitely lossy! Gwern posts about this when people say something like that on here, but I'll do it instead. Lossless encoding is just lossy encoding + error correction of some sort.

Hah. "error correction of some sort" is doing a lot of heavy lifting there. Bit level correction is what I generally consider under the "error correction" umbrella. Which we have no problem with -- curious the bit error rate that would make text unreadable. In the context of compressing a large part of the English language wiki -- I think lossy also includes loss so significant that you wouldn't be able to reproduce the exact text. So, well beyond what we would generally consider "error correcting". But intuitively understandable as equivalent by humans. Impossible to quantify that objectively, hence lossless only for the competition.

Re: Extreme video compression with prediction using pre-trainded diffusion models

#34

Post by resolutebat »

newaccount7g wrote: post #18
userbinator wrote: post #16

How fast is this and how big is the decoder/encoder? The model weights are not accessible. From the description, it looks like it's only being tested with 128x128 frames, which implies that the speed is very low.

Why would you expect those kind of details in a paid commercial?

It's a link to a Github repo, not a "paid commercial".

Re: Extreme video compression with prediction using pre-trainded diffusion models

#35

Post by resolutebat »

yonixw wrote: post #3
zaptrem wrote: post #2

Can you share example videos?

Googling gave me the article: https://www.arxiv.org/abs/2402.08934 Which have examples in it.

Direct link to HTML article: https://arxiv.org/html/2402.08934v1

Unfortunately it only contains still images with teeny thumbnails: https://arxiv.org/html/2402.08934v1/x2.png

Re: Extreme video compression with prediction using pre-trainded diffusion models

#36

Post by est »

polemic wrote: post #29
Animats wrote: post #6

Extreme compression will be when you put in a movie and get a SORA prompt back that regenerates something close enough to the movie.

How big is the SORA model itself?

it depends which resolution you want to see.

Re: Extreme video compression with prediction using pre-trainded diffusion models

#37

Post by resolutebat »

Here's the research behind this: https://arxiv.org/html/2402.08934v1

As a casual non-scholar, non-AI person trying to parse this though, it's infuriatingly convoluted. I was expecting a table of "given source file X, we got file size Y with quality loss Z", but while quality (SSIM/LPIPS) is compared to standard codecs like H.264, for the life of me I can't find any measure of how efficient the compression is here.

Applying AI to image compression has been tried before though, with distinctly mediocre results: some may recall the Xerox debacle about 10 years, when it turned out copiers were helpfully "optimizing" images by replacing digits with others in invoices, architectural drawings, etc.

https://www.theverge.com/2013/8/6/4594482/xerox-copiers-rand...

Re: Extreme video compression with prediction using pre-trainded diffusion models

#38

Post by jasonjmcghee »

polemic wrote: post #29
Animats wrote: post #6

Extreme compression will be when you put in a movie and get a SORA prompt back that regenerates something close enough to the movie.

How big is the SORA model itself?

You only need one copy of it - even if it is 100GB. If it's baked into every OS... and storage / RAM keeps getting cheaper... just might work.

Re: Extreme video compression with prediction using pre-trainded diffusion models

#39

Post by lifthrasiir »

resolutebat wrote: post #37

Here's the research behind this: https://arxiv.org/html/2402.08934v1 As a casual non-scholar, non-AI person trying to parse this though, it's infuriatingly convoluted. I was expecting a table of "given source file X, we got file size Y with quality loss Z", but while quality (SSIM/LPIPS) is compared to standard codecs like H.264, for the life of me I can't find any measure of how efficient the compression is here. Ap…

> [S]ome may recall the Xerox debacle about 10 years, when it turned out copiers were helpfully "optimizing" images by replacing digits with others in invoices, architectural drawings, etc.

This is not even AI. JBIG2 allows a reuse of once-decoded image patches because it's quite reasonable for bi-level images like fax documents. It is true that similar glyphs may be incorrectly groupped into the same patch, but such error is not specific to patch-based compression methods (quantization can often lead to the same result). The actual culprit was Xerox's bad implementation of JBIG2 that incorrectly merged too many glyphs into the same patch.

Re: Extreme video compression with prediction using pre-trainded diffusion models

#40

Post by orblivion »

polemic wrote: post #29
Animats wrote: post #6

Extreme compression will be when you put in a movie and get a SORA prompt back that regenerates something close enough to the movie.

How big is the SORA model itself?

I can show you an algorithm that compresses an entire 2 hour movie to a single bit, but it only works on one movie.
Post reply on HN