Live data from Hacker News

QOI: Lossless Image Compression in O(n) Time

phoboslab.org

11–20 of 303 posts

Re: QOI: Lossless Image Compression in O(n) Time

#11

This is as cool as it gets. I have a honest question about the source code, a lot of time since I coded anything meaningful in C, but I remember the header files did not have that much code on it, while here I see most of the code is in the header file. Why is this the case? Inline compilation? What are the advantages?

There is a strong tradition of single-file C/C++ libraries [1] because it has been difficult to integrate larger C/C++ libraries into the codebase.

[1] https://github.com/p-ranav/awesome-hpp

Re: QOI: Lossless Image Compression in O(n) Time

#12

tl;dr: 00xxxxxx - copy (x+1)-th last EXPLICITLY ENCODED pixel (i.e. ignoring repeats) 010xxxxx - repeat the last pixel x+1 times 011xxxxx xxxxxxxx - repeat the last pixel x+33 times 10rrggbb - copy the last pixel and adjust RGB by (r-1, g-1, b-1) 110rrrrr ggggbbbb - copy the last pixel and adjust RGB by (r-15, g-7, b-7) 1110rrrr rgggggbb bbbaaaaa - copy the last pixel and adjust RGBA by (r-15, g-15, b-15, a-15) 1111R…

You might be able to do better compression by traversing the pixels in Hilbert [1] order or Z [2] order, to take into account vertical neighbours as well as horizontal. It might cost time or space though, as you couldn't stream the pixels through it anymore - you'd need them all in memory.

[1] https://en.wikipedia.org/wiki/Hilbert_curve

[2] https://en.wikipedia.org/wiki/Z-order_curve

Re: QOI: Lossless Image Compression in O(n) Time

#13
Very cool. Typo: "I barely understand how Huffman Coding and DTC works" I guess you mean DCT? (discrete cosine transform)

One other thought: To make it so that compression and decompression can be multithreaded, you might want to 'reset' the stream every N rows. (i.e. break any RLE runs, start from a colour literal). This would allow a bunch of threads to start at different places in the image, in parallel. There would be some small cost to compression ratio but might be worth it.

Re: QOI: Lossless Image Compression in O(n) Time

#14

This is as cool as it gets. I have a honest question about the source code, a lot of time since I coded anything meaningful in C, but I remember the header files did not have that much code on it, while here I see most of the code is in the header file. Why is this the case? Inline compilation? What are the advantages?

Packing the entire library in a header file is a somewhat recent and much welcome trend.

See for example:

https://github.com/nothings/stb

https://github.com/nothings/single_file_libs

Re: QOI: Lossless Image Compression in O(n) Time

#15
post #4

This is impressive, I was curious if it was just better on simpler images with many duplicate pixels but it's nice to see it working well with photography too e.g. https://phoboslab.org/files/qoibench/images/wallpaper/EwZDbL... decode ms encode ms decode mpps encode mpps size kb libpng: 148.4 3995.5 55.88 2.08 12223 stbi: 161.0 1858.3 51.50 4.46 19199 qoi: 60.8 95.6 136.49 86.78 12868 I'm interested if there's a stan…

When I download that PNG file it is 10785 KB. If I re-encode it using FLIF 0.4 (to my knowledge still the best off-the-shelf lossless image compression) it is 7368 KB. That's 57% of the size of qoi, quite a lot smaller. You pay for it in encode/decode time though.

Re: QOI: Lossless Image Compression in O(n) Time

#16

Very cool. Typo: "I barely understand how Huffman Coding and DTC works" I guess you mean DCT? (discrete cosine transform) One other thought: To make it so that compression and decompression can be multithreaded, you might want to 'reset' the stream every N rows. (i.e. break any RLE runs, start from a colour literal). This would allow a bunch of threads to start at different places in the image, in parallel. There wou…

Thanks, fixed!

I'll probably investigate "resetting" the stream to allow for multithreaded en-/decode when I try to roll this into a video codec.

Re: QOI: Lossless Image Compression in O(n) Time

#17
post #12

tl;dr: 00xxxxxx - copy (x+1)-th last EXPLICITLY ENCODED pixel (i.e. ignoring repeats) 010xxxxx - repeat the last pixel x+1 times 011xxxxx xxxxxxxx - repeat the last pixel x+33 times 10rrggbb - copy the last pixel and adjust RGB by (r-1, g-1, b-1) 110rrrrr ggggbbbb - copy the last pixel and adjust RGB by (r-15, g-7, b-7) 1110rrrr rgggggbb bbbaaaaa - copy the last pixel and adjust RGBA by (r-15, g-15, b-15, a-15) 1111R…

You might be able to do better compression by traversing the pixels in Hilbert [1] order or Z [2] order, to take into account vertical neighbours as well as horizontal. It might cost time or space though, as you couldn't stream the pixels through it anymore - you'd need them all in memory. [1] https://en.wikipedia.org/wiki/Hilbert_curve [2] https://en.wikipedia.org/wiki/Z-order_curve

You could still stream the pixels, you'd just have to stream them in Z order. This may seem pedantic, but this applies to any mismatch between the order in which you store pixels and the order in which the format stores pixels.

E.g. some file formats may be row-major, others column-major, some block-based (like JPEG) or a fancy interleaving scheme (like Adam7). Some might store the channels interleaved, others separate the channels. If any of these choices doesn't match the desired output format, it breaks streaming.

Re: QOI: Lossless Image Compression in O(n) Time

#18
post #12

tl;dr: 00xxxxxx - copy (x+1)-th last EXPLICITLY ENCODED pixel (i.e. ignoring repeats) 010xxxxx - repeat the last pixel x+1 times 011xxxxx xxxxxxxx - repeat the last pixel x+33 times 10rrggbb - copy the last pixel and adjust RGB by (r-1, g-1, b-1) 110rrrrr ggggbbbb - copy the last pixel and adjust RGB by (r-15, g-7, b-7) 1110rrrr rgggggbb bbbaaaaa - copy the last pixel and adjust RGBA by (r-15, g-15, b-15, a-15) 1111R…

You might be able to do better compression by traversing the pixels in Hilbert [1] order or Z [2] order, to take into account vertical neighbours as well as horizontal. It might cost time or space though, as you couldn't stream the pixels through it anymore - you'd need them all in memory. [1] https://en.wikipedia.org/wiki/Hilbert_curve [2] https://en.wikipedia.org/wiki/Z-order_curve

Iteration order through arrays has a big impact on performance due to cache effects. I'd guess that you'd lose a lot of performance by a complicated iteration scheme.

Re: QOI: Lossless Image Compression in O(n) Time

#19
I am in awe with how the post neatly interleaves the why and the how. Having recently implemented L4 decompressor (itself a rather reasonable and "lightweight" standard), I like the QOI even better. Hoping to see the video decoder author mentions as possibility!

Re: QOI: Lossless Image Compression in O(n) Time

#20
post #3

To answer the author's question: No, SIMD would not help with this. This is neat. I wonder if the author would be willing to write a Kaitai Struct definition for it. Something else interesting: QOI is 1,2,2 letters off from PNG. I'm quite certain this is an accident but it's interesting nonetheless.

I suspect SIMD would help with the encoding. The lookup table is small enough to fit into 8 AVX2 registers, so instead of hashing, you could use direct lookup, which would improve compression ratio further (a little bit).
Post reply on HN