Generative Adversarial Networks for Extreme Learned Image Compression
data.vision.ee.ethz.ch
Generative Adversarial Networks for Extreme Learned Image Compression
1–10 of 33 posts
Re: Generative Adversarial Networks for Extreme Learned Image Compression
#2Just waiting for this to show up in a video compression standard. With the right network it could be just as fast to decompress, though probably insanely slow to compress.
Re: Generative Adversarial Networks for Extreme Learned Image Compression
#3Was thinking about this use case of neural networks for months... Glad to read a paper about that. Wonder how to adapt that to video
Re: Generative Adversarial Networks for Extreme Learned Image Compression
#4The trick is not so much about compression, but rather about image generation.
The trade off is completely different from usual lossy algorithms. A highly compressed image might still retain high visual quality but with completely different details, textures.
Kinda what would happen if you use a perfect painter with a blurry memory.
Re: Generative Adversarial Networks for Extreme Learned Image Compression
#5The trick is not so much about compression, but rather about image generation. The trade off is completely different from usual lossy algorithms. A highly compressed image might still retain high visual quality but with completely different details, textures. Kinda what would happen if you use a perfect painter with a blurry memory.
This is quite visible in their demonstration slider-image; the building in the back-right changes from a brick building with glass windows to a stucco-ish building with a bunch of exterior duct-work.
Re: Generative Adversarial Networks for Extreme Learned Image Compression
#6The trick is not so much about compression, but rather about image generation. The trade off is completely different from usual lossy algorithms. A highly compressed image might still retain high visual quality but with completely different details, textures. Kinda what would happen if you use a perfect painter with a blurry memory.
This would have interesting implications if we accidentally trained biased networks (provocative example: improperly changing people's skin-colors in images when decoding them)
Re: Generative Adversarial Networks for Extreme Learned Image Compression
#7Take the following as coming from a dilettante... I'm still trying to understand the remainder of the paper but felt like writing on the basics of the encoder/decoder/quantizer setup they mention.
I found this particularly interesting "To compress an image x ∈ X , we follow the formulation of [20, 8] where one learns an encoder E, a decoder G, and a finite quantizer q."
I feel like this is related to some of the standard human memorization/learning techniques. Example: I'm learning the guitar fretboard note placement in e standard. It's difficult for me to visualize the first 4 frets on a 6 string guitar with notes on each fret.
To help me memorize the note placement I develop various mnemonic devices (both lossy and lossless). I know I've memorized the fretboard sufficiently when I can visualize it.
Attempting to translate my reading of the paper I believe the following analogy is apt. My "encoder" operates on a short term image when I close my eyes after looking at a fret diagram. It produces semantic objects, i.e. an ordered sequence of "letters" or pairs of letters (letters that are horizontally, vertically or diagonally aligned). The quantizer takes these objects and looks at the order/distribution. The quantizer places more importance on some of the semantic objects than others (the fourth fret has 4 natural notes before an accidental). My decoder is interpreting the stored/compressed note information to try to produce the image. It may be off substantially, so I correct and repeat the process.
The process of optimizing what the semantic objects are, the weight each gets, and how I use them to derive the original image seems like a fairly good representation of what I do (though at least some of that appears to be fixed in the learning algorithm typically). Of course, analogies are just that and mine doesn't take into account the discriminator or the remaining "heart" of the paper.
I think the heart of the paper is that they're trying to determine through GANs a good way to both store the image and recover it while reducing bits per pixel and increasing the quality of reproduction. Using some classical terms, the GAN algorithm thus tweaks the compressor, the data storage format and the decompressor to optimize what should be "hard-coded" in the compressing/decompressing process or program vs what will be stored as a result of the compression program.
Very handwavey but I think the general idea is right?
Re: Generative Adversarial Networks for Extreme Learned Image Compression
#8The trick is not so much about compression, but rather about image generation. The trade off is completely different from usual lossy algorithms. A highly compressed image might still retain high visual quality but with completely different details, textures. Kinda what would happen if you use a perfect painter with a blurry memory.
This is quite visible in their demonstration slider-image; the building in the back-right changes from a brick building with glass windows to a stucco-ish building with a bunch of exterior duct-work.
Also, the car behind the bus disappears. It looks like it's been photoshopped out. This is unexpected behavior from a compression algorithm. Users are conditioned to expect the quality to degrade uniformly across the whole image.
Re: Generative Adversarial Networks for Extreme Learned Image Compression
#9Earlier quoted context omitted.
This is quite visible in their demonstration slider-image; the building in the back-right changes from a brick building with glass windows to a stucco-ish building with a bunch of exterior duct-work.
Also, the car behind the bus disappears. It looks like it's been photoshopped out. This is unexpected behavior from a compression algorithm. Users are conditioned to expect the quality to degrade uniformly across the whole image.
You would have to be very very careful indeed if you wanted to use this for anything with safety implications.
Re: Generative Adversarial Networks for Extreme Learned Image Compression
#10Earlier quoted context omitted.
This is quite visible in their demonstration slider-image; the building in the back-right changes from a brick building with glass windows to a stucco-ish building with a bunch of exterior duct-work.
Also, the car behind the bus disappears. It looks like it's been photoshopped out. This is unexpected behavior from a compression algorithm. Users are conditioned to expect the quality to degrade uniformly across the whole image.
In fairness that detail isn't preserved in the other formats either. The new issue is merely the illusion of accuracy.