Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Decoding AVIF: Deep dive with cats and imgproxy

evilmartians.com

Post reply on HN
Print view View original

21–29 of 29 posts

Re: Decoding AVIF: Deep dive with cats and imgproxy

#21

Post by CharlesW »

newpavlov wrote: post #20

I wonder what are modern use-cases for 4:2:2 sampling? Is it simply a historic relic, which gets ported from codec to codec?

> I wonder what are modern use-cases for 4:2:2 sampling? Is it simply a historic relic, which gets ported from codec to codec?

https://en.wikipedia.org/wiki/Chroma_subsampling nicely explains this clever hack's continuing relevance.

TLDR: Chroma subsampling (as done in the 4:2:2 Y'CbCr color space) is used to improve video encoding efficiency by taking advantage of the "human visual system's lower acuity for color differences than for luminance".

Re: Decoding AVIF: Deep dive with cats and imgproxy

#22

Post by newpavlov »

CharlesW wrote: post #21
newpavlov wrote: post #20

I wonder what are modern use-cases for 4:2:2 sampling? Is it simply a historic relic, which gets ported from codec to codec?

> I wonder what are modern use-cases for 4:2:2 sampling? Is it simply a historic relic, which gets ported from codec to codec? https://en.wikipedia.org/wiki/Chroma_subsampling nicely explains this clever hack's continuing relevance. TLDR: Chroma subsampling (as done in the 4:2:2 Y'CbCr color space) is used to improve video encoding efficiency by taking advantage of the "human visual system's lower acuity for color di…

I do understand [1] why 4:2:0 sampling is used, my question was strictly about 4:2:2.

[1]: https://news.ycombinator.com/item?id=28204865

Re: Decoding AVIF: Deep dive with cats and imgproxy

#23

Post by leguminous »

HALtheWise wrote: post #11
dylan604 wrote: post #2

"Commonly, three numbers are used to specify downsampling: the first is always 4, don’t ask me why" Wow. Someone going to this much detail on explaining how a video/image codec works, and cannot bother learning what the numbers of chroma subsampling mean? The first number represents the luminance.[0] Even if they know the first number represents luminance, the "don't ask me why" is just horrible on its own. The detai…

One thing I never understood is why _downsampling_ is the most efficient way to compress the data about chroma into fewer bits while maximizing perceptual accuracy. It really seems like for any given target bitrate for the chroma data, there should always be a more efficient compression scheme available than simply throwing out 3/4 of the pixels and running compression algorithms on the rest. Surely modern compressio…

I have had the same thought. Why not do away with chroma subsampling and just compress the chroma planes more heavily than the luma plane? Does heavy compression perform worse than just throwing away 3/4 of the data?

Re: Decoding AVIF: Deep dive with cats and imgproxy

#24

Post by brigade »

newpavlov wrote: post #20

I wonder what are modern use-cases for 4:2:2 sampling? Is it simply a historic relic, which gets ported from codec to codec?

Unlike 4:2:0 it works well as a packed format (e.g. Y0 Cb Y1 Cr), and unlike 4:4:4 it's a simple 2 bytes per pixel (packed 4:4:4 either uses an uneven 3 bytes per pixel or wastes 1 byte per pixel...)

Which led to a lot of simple and professional HW/SW being designed for packed 4:2:2, so codecs support 4:2:2 to fit into professional pipelines.

Re: Decoding AVIF: Deep dive with cats and imgproxy

#25

Post by newpavlov »

brigade wrote: post #24
newpavlov wrote: post #20

I wonder what are modern use-cases for 4:2:2 sampling? Is it simply a historic relic, which gets ported from codec to codec?

Unlike 4:2:0 it works well as a packed format (e.g. Y0 Cb Y1 Cr), and unlike 4:4:4 it's a simple 2 bytes per pixel (packed 4:4:4 either uses an uneven 3 bytes per pixel or wastes 1 byte per pixel...) Which led to a lot of simple and professional HW/SW being designed for packed 4:2:2, so codecs support 4:2:2 to fit into professional pipelines.

Thank you! Yeah, the ability to handle pixel data in chunks of 4 bytes can be useful in certain context.

Re: Decoding AVIF: Deep dive with cats and imgproxy

#26

Post by a_e_k »

SilverRed wrote: post #14

I had a really good go at reading this and trying to understand it but I feel I don't understand a whole lot more about image decoding/encoding than before I started. Like I get the core concepts of key frames, motion vectors and such on a high level but if you asked me to actually create a decoder I wouldn't have a clue where to start. I feel like I would need a full hour long video on each paragraph of this post to…

Last year, I did a deep dive into Fabrice Bellard's obfuscated image decoder (http://www.ioccc.org/2018/bellard/hint.html) entry for the 2018 IOCCC. His code implements of a lot of these decoding techniques in just 4KB of source, including a stringified 128x128 test image.

You can find my deobfuscation and detailed explanation of his program here: http://eastfarthing.com/blog/2020-09-14-decoder/

Re: Decoding AVIF: Deep dive with cats and imgproxy

#27

Post by magicalhippo »

magicalhippo wrote: post #18
SilverRed wrote: post #14

I had a really good go at reading this and trying to understand it but I feel I don't understand a whole lot more about image decoding/encoding than before I started. Like I get the core concepts of key frames, motion vectors and such on a high level but if you asked me to actually create a decoder I wouldn't have a clue where to start. I feel like I would need a full hour long video on each paragraph of this post to…

We made a mpeg-ish encoder and decoder at uni, and it was surprisingly simple. Keyframes were just jpegs. Then for intraframes, you first found the motion vectors for each 8x8 block, then generated the predicted intraframe from the keyframe and motion vector. Then you subtracted the prediction off the keyframe. The resulting "prediction error image" was then simply jpeg encoded as the intraframe, and appended to the…

> simply jpeg encoded as the intraframe

In case it wasn't obvious, the error image can of course have negative values which jpeg can't handle. So you add a bias of +128 and clamp the biased error to [0, 255].

During decoding you simply subtract the bias when adding back the error image.

Re: Decoding AVIF: Deep dive with cats and imgproxy

#28

Post by pornel »

I wish AVIF had a simpler web-oriented profile. It has inherited bloat from HEIF, which is built on a tower of specs, which were designed to be a catch-all of all features for everyone, including cameras, photo management, and editing. Because it recycles a video codec and old MP4 specs, there's also a ton of compatibility with legacy video tools — in a brand new image format.

AVIF/HEIF has hundreds of features that decoders are theoretically supposed to implement. Many of ISOBMFF "boxes" also have multiple versions (typically 16/32/64-bit versions or "oops we forgot to add a field" version). Does every decoder really need to support all of them? It's such a waste of effort, and file bloat. Browsers only care about getting pixels on screen. They don't have UIs nor APIs to browse through photo bursts, bracketed exposures, or an infrared channel (or is every AVIF viewer supposed to implement these now?)

And some features are even dangerous: lossless cropping. Great for your camera roll app, but for publishing on the Web it's a huge footgun. You could think you've cropped private info out of the picture you've shared, but randos on the internet can uncrop your HEIF pictures.

Post reply on HN