Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Generative Adversarial Networks for Extreme Learned Image Compression

data.vision.ee.ethz.ch

Post reply on HN
Print view View original

11–20 of 33 posts

Re: Generative Adversarial Networks for Extreme Learned Image Compression

#11

Post by bcheung »

I'd be curious to see how different levels of quantization affect the image. From the paper it looks like the quantization is applied at the latent feature space. I wonder if it has similar effects like the celebrity GAN's we have seen where interpolating in the latent space results in morphing from one face to another. Could be funny when compression doesn't result in something blocky or distorted, but replacing objects with other objects that look similar to them.

This seems to be for static images, but this gets me wondering if an RNN can be used and have better motion prediction that other current "hard coded" solutions.

Also, the more specific the domain, the better the compression, since it can specialize. I'm wondering about the practical applications of this. Do we have different baselines that can be used for different use cases?

Re: Generative Adversarial Networks for Extreme Learned Image Compression

#12

Post by bcheung »

tmpmov wrote: post #7

Take the following as coming from a dilettante... I'm still trying to understand the remainder of the paper but felt like writing on the basics of the encoder/decoder/quantizer setup they mention. I found this particularly interesting "To compress an image x ∈ X , we follow the formulation of [20, 8] where one learns an encoder E, a decoder G, and a finite quantizer q." I feel like this is related to some of the stan…

An encoder / decoder architecture learns a more "efficient" representation. It tries to find features it can use that are useful for describing the variations in the input data (images) that it has seen.

For example, if trained on faces, it will learn features for things like eyes and mouths. So the image can be encoded as put a mouth of this type with this width at this location rather than operating at the level of pixels.

If trained on text, it might learn features related to letters and typography (boldness, italics, size, spacing). So it might encode things as Helevetica, 16pt, italics.

This is a gross oversimplification, and things rarely map exactly to concepts humans would use, but hopefully it communicates the concept.

Re: Generative Adversarial Networks for Extreme Learned Image Compression

#13

Post by bcheung »

ttoinou wrote: post #3

Was thinking about this use case of neural networks for months... Glad to read a paper about that. Wonder how to adapt that to video

It's very interesting. I've heard it said in online lectures that it does a sort of compression but nobody really uses it for that because existing algorithms perform much better. Guess this is no longer true.

Re: Generative Adversarial Networks for Extreme Learned Image Compression

#14

Post by rasz »

Veedrac wrote: post #10
fludlight wrote: post #8

Earlier quoted context omitted.

Also, the car behind the bus disappears. It looks like it's been photoshopped out. This is unexpected behavior from a compression algorithm. Users are conditioned to expect the quality to degrade uniformly across the whole image.

In fairness that detail isn't preserved in the other formats either. The new issue is merely the illusion of accuracy.

car is visible in WebP.

Re: Generative Adversarial Networks for Extreme Learned Image Compression

#15

Post by rasz »

Picture is not compressed, its hallucinated from vague memory of the real thing, a mere dream. Cars vanish, building change wall structure, even the license plate receives fake text absent from source materia.

Its a giant guesswork of what was there originally. Reminds me of Xerox scanners lying about scanned in numbers http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...

Re: Generative Adversarial Networks for Extreme Learned Image Compression

#16

Post by make3 »

rasz wrote: post #15

Picture is not compressed, its hallucinated from vague memory of the real thing, a mere dream. Cars vanish, building change wall structure, even the license plate receives fake text absent from source materia. Its a giant guesswork of what was there originally. Reminds me of Xerox scanners lying about scanned in numbers http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...

It's not as bad as you say. Sure, the positions of the individual leaves and of the grain of the concrete and of little puffs of the clouds are hallucinated, but most salient semantic features like the presence of a car or of a person are left untouched.

In other words, mostly only unimportant details are hallucinated, which is what we want.

Re: Generative Adversarial Networks for Extreme Learned Image Compression

#17

Post by yorwba »

rasz wrote: post #15

Picture is not compressed, its hallucinated from vague memory of the real thing, a mere dream. Cars vanish, building change wall structure, even the license plate receives fake text absent from source materia. Its a giant guesswork of what was there originally. Reminds me of Xerox scanners lying about scanned in numbers http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...

All lossy compression algorithms hallucinate. That's the whole point: reducing image size by dropping some information and then hallucinating a plausible replacement to decompress.

The only difference is that this compression is better at hallucinating, so you don't get ringing artifacts or blocks, but some internally consistent alternate reality.

If you don't want to lose data you should not use lossy compression at all. JPEG can erase the distinction between digits as well.

Re: Generative Adversarial Networks for Extreme Learned Image Compression

#18

Post by skybrian »

yorwba wrote: post #17
rasz wrote: post #15

Picture is not compressed, its hallucinated from vague memory of the real thing, a mere dream. Cars vanish, building change wall structure, even the license plate receives fake text absent from source materia. Its a giant guesswork of what was there originally. Reminds me of Xerox scanners lying about scanned in numbers http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...

All lossy compression algorithms hallucinate. That's the whole point: reducing image size by dropping some information and then hallucinating a plausible replacement to decompress. The only difference is that this compression is better at hallucinating, so you don't get ringing artifacts or blocks, but some internally consistent alternate reality. If you don't want to lose data you should not use lossy compression at…

Okay, but still, some kinds of changes are better than others. They should probably start testing for this in the visual perception tests: is lost information greyed out in a visible way? Are words and digits always fuzzed, or replaced?

Because it turns out that fuzziness and compression artifacts have a higher-level meaning: when you see them, you know something has been lost. That's an important (if inadvertent) signal. We need to make sure the artifacts don't go away.

Re: Generative Adversarial Networks for Extreme Learned Image Compression

#19

Post by make3 »

bcheung wrote: post #11

I'd be curious to see how different levels of quantization affect the image. From the paper it looks like the quantization is applied at the latent feature space. I wonder if it has similar effects like the celebrity GAN's we have seen where interpolating in the latent space results in morphing from one face to another. Could be funny when compression doesn't result in something blocky or distorted, but replacing obj…

it also means that you could get a meaningful average of two images by averaging their compressed form (aka latent state z), and decoding, just like with the celebrities :)
Post reply on HN