Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

QOI: Lossless Image Compression in O(n) Time

phoboslab.org

Post reply on HN
Print view View original

191–200 of 303 posts

Re: QOI: Lossless Image Compression in O(n) Time

#191

Post by pixel_fcker »

formerly_proven wrote: post #182
cornstalks wrote: post #178

Earlier quoted context omitted.

> When I say (255, 0, 0), I expect the display to show me its best approximation of red. Which red? There isn't a single, true "red". And different materials produce different reds (my work's Sony BVM-X300's reddest red is going to look way different than your monitor's red). Not all displays use only RGB color primaries, too. For example, at Dolby's office in San Francisco they have a projector in their theater that…

> An absolute color space exists. It's called XYZ. We could use it. Some image formats support it. XYZ is really annoying to work in though. ACES is a pragmatic solution here, quite literally: https://blog.frame.io/wp-content/uploads/2019/09/ACES-APO.jp... Okay, fair enough two out of three primaries are imaginary colors, but nobody said you have to use the whole color space when your processing pipeline is meant to…

ACES (AP0) is an archive format for when you want to be absolutely sure you’re not clipping any colours. As a working space it’s terrible and AP1 should be preferred, either as ACEScg or ACEScc

Re: QOI: Lossless Image Compression in O(n) Time

#192

Post by adgjlsfhk1 »

makapuf wrote: post #181
kaetemi wrote: post #166

Earlier quoted context omitted.

The sRGB color #ffffff should never mean "the brightest white on this monitor", unless you're using an average CRT. Just imagine you're using an HDR display, where the brightest white is as bright as the sun.

OK but then when will you use that "bright as sun" color? If not, why provide them? If so, what color will you use ?

The color you will use isn't in SRGB, but is in other color spaces such as Rec2020.

Re: QOI: Lossless Image Compression in O(n) Time

#193

Post by dv_dt »

pornel wrote: post #77
cyber_kinetist wrote: post #47

Earlier quoted context omitted.

I think the author's intending to use this only as a data packing format (ex. for app/game development) and not a general-purpose exchange format. If all you want to do is pack image/texture data for your game then I totally get why they assume RGBA8 storage and leave out metadata such as color space information.

I know games tend to assume color spaces from context, and do other hacky things like reusing alpha for height maps or RGB for vector coordinates. But this approach is a source of endless conflict with image processing tooling, because nobody except the game author knows what the channels mean. Game developers keep complaining about image processing tools doing anything other than bytes-in-bytes-out on that data (e.g…

Yup embed enough metadata so you can actually nicely automate/integrate doing the right thing with tooling.

Re: QOI: Lossless Image Compression in O(n) Time

#194

Post by Trixter »

st_goliath wrote: post #110

Out of curiosity, I modified the benchmark program to simply toss the pixel data into lz4/lz4hc for comparison (patch file: https://paste.debian.net/1220663/ ). I'm actually quite impressed how the resulting size is a little bit smaller than lz4hc (actually not even that far away from libpng), while the encoding speed is quite close to lz4, despite seemingly not having as many hacks and tweaks under the hood to get t…

> liblz4 probably benefits from decades of tweaking & tuning to get there

Decade, singular. LZ4 was proper around 2011. But the real reason it's faster is because its not doing nearly as much work as QOI. LZ4 is just about as asymmetric as you can get.

Re: QOI: Lossless Image Compression in O(n) Time

#195

Post by spc476 »

cornstalks wrote: post #178
sam0x17 wrote: post #159

Earlier quoted context omitted.

> We're stuck with images stored in the display profile of old CRTs by default, because that was the most practical option at the time. The analogue in photography is photographs lose color clarity and fade as they age. Why should I care if this is the case in images as display technology evolves when this is already a problem with every physical medium? > For correct editing operations, and correct display. Sadly, i…

> When I say (255, 0, 0), I expect the display to show me its best approximation of red. Which red? There isn't a single, true "red". And different materials produce different reds (my work's Sony BVM-X300's reddest red is going to look way different than your monitor's red). Not all displays use only RGB color primaries, too. For example, at Dolby's office in San Francisco they have a projector in their theater that…

And the red in my left eye isn't the same as the red in my right eye. Yes, when the lighting conditions are just right, I see different hues out of each eye [1]. I have to wonder how one would (or should?) correct for that in an image file format.

[1] I think from a stupid thing I did as a kid.

Re: QOI: Lossless Image Compression in O(n) Time

#196

Post by jijji »

superjan wrote: post #173
BlueTemplar wrote: post #156

Earlier quoted context omitted.

The most important is probably the difference between lossy and lossless compression... (The GP-mentioned Bink is lossy ! Then he also for some reason uses a picture tending towards photo realistic which does NOT play well with lossless !)

Yeah there’s a difference in techniques used for lossy vs lossless. But my point was that in compression research, runtime performance was much less important than compression performance, that was no different for lossy compression.

speaking of runtime performance if you look at lzma for instance it's horrible on runtime performance but it produces some of the best compression ratio of all of them

Re: QOI: Lossless Image Compression in O(n) Time

#197

Post by haxiomic »

gfody wrote: post #167
haxiomic wrote: post #4

This is impressive, I was curious if it was just better on simpler images with many duplicate pixels but it's nice to see it working well with photography too e.g. https://phoboslab.org/files/qoibench/images/wallpaper/EwZDbL... decode ms encode ms decode mpps encode mpps size kb libpng: 148.4 3995.5 55.88 2.08 12223 stbi: 161.0 1858.3 51.50 4.46 19199 qoi: 60.8 95.6 136.49 86.78 12868 I'm interested if there's a stan…

pngcrush -brute 10355kb cjxl -q 100 -e 9 8413kb

Can be made smaller for sure, but if I'm say working with 10,000 images in my app and decoding all the time, a 3-4x speedup in the decode time can be a major saving for ~ 10-40% increase in file size with a lib I can just drop in my project without effort

Re: QOI: Lossless Image Compression in O(n) Time

#198

Post by zosoworld »

jameshart wrote: post #160
zosoworld wrote: post #25

I'm wondering how scalable would this be for hyperspectral images, with 25 channels for example. I'm guessing it's just repeating the processing for RGB channels but for 25?

Very much the kind of expert group design-by-committee type of requirement that this simple compression scheme was designed not to handle. Since this scheme works primarily on exact cross-channel pixel matches, no. No it won’t work well for 25 channel images. Which is a good thing. Most people are not trying to encode 25 channel hyperspectral data and if they are they can use their own format designed well for that p…

I did not try to imply that this should contemplate the other use case, but I was wondering whether the compression percentage would scale up with more channels that might not be related in the same way RGB are (as in luminance, for example).

This effort really makes me want to try to do something similar for the other use case, and see how well it fares compared to the convoluted standards that are used for this topic.

Thank you for your insight!

Re: QOI: Lossless Image Compression in O(n) Time

#199

Post by kortex »

simcop2387 wrote: post #136
kortex wrote: post #130

Super cool work! I was also mucking around with codecs last week. I got some surprisingly good results from delta encoding each channel separately and then zstandard compression. Dirt simple, crazy fast. One trap to be aware of when profiling image codecs is not knowing the provenance of the test images. Many images will have gone through one or more quantization steps. This can lead to impressive compression ratios,…

> Super cool work! I was also mucking around with codecs last week. I got some surprisingly good results from delta encoding each channel separately and then zstandard compression. Dirt simple, crazy fast. My understanding is that this is one of the techniques that png uses (though not zstd) for doing it's xompression too. Along with I think some different strides down the image and some other stuff but I don't fully…

Yeah, my technique was equivalent to filter mode 0 (the only mode), filter type 1 (sub pixel to the left). literally array.ravel().diff(prepend=0) in numpy.

I think part of the reason png is so slow is that it empirically tests to find the best one.

> Compression is further improved by choosing filter types adaptively on a line-by-line basis.

I'd be curious to unpack a png and see the statistics of how much each kind is used.

Re: QOI: Lossless Image Compression in O(n) Time

#200

Post by Trixter »

SavantIdiot wrote: post #177

What I like about this approach is that it requires zero differential calculus. People forgot that Radix2 DCTs come from a century's worth of advanced math in the S domain. This approach is applied-computer-science based, I can get my head around that a lot better than a Laplace/Z transform. But what shocks me is how did absolutely no-one apply such a naive RLE in the 30 years of internet image formats? Is this a cas…

RLE on runs of deltas is in very many compression schemes; it's a common technique (probably so common that many authors don't deem it worth mentioning). Lossless video codecs in particular (huffyuv, lagarith, utcodec, etc.) have had this going back 20 years at least.
Post reply on HN