Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

QOI – The “Quite OK Image Format” for fast, lossless image compression

github.com

Post reply on HN
Print view View original

101–105 of 105 posts

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#101

Post by sushibowl »

midjji wrote: post #45
FullyFunctional wrote: post #30

Since we are rehashing this for the 3rd (4th?) time, I'll repeat mine (and apparently many others) key critique: there is no thought at all to enabling parallel decoding, be it, thread-parallel or SIMD (or both). That makes it very much a past millennium style format that will age very poorly. At the very least, break it into chunks and add an offset directory header. I'm sure one could do something much better, but…

A surprisingly good image format is to use a per line, or block, ar encoder, then compress the result with gzip on a low setting. It parallelizes very nicely, and beats png for encode, decode, and is trivial to implement.

isn't that very similar to what PNG does? I'm not sure what you mean by "ar encoder", but PNG uses a per-line filter and then adds DEFLATE on top of that.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#102

Post by seoaeu »

meltedcapacitor wrote: post #85
seoaeu wrote: post #71

Earlier quoted context omitted.

No it can't. The encoder doesn't insert any "cut-off points". In fact, nearly every chunk encodes the current pixel value in terms of one of the previous pixel values, so without knowing those it is impossible for a second core to start up and initialize its decoder state enough to produce correct output.

Read again the proposal. A top level thread scans the opcodes only to solve this, with no decoding and no writing, thus progressing faster in the stream than the child chunked decoding threads it progressively spawns. Not as quick as a format with a chunk table, but faster than naive single core.

I did read your comment. You don’t have any explanation of how the top level scan can maintain the color index array for QOI_OP_INDEX chunks without doing any decoding.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#103

Post by midjji »

ErikCorry wrote: post #83
midjji wrote: post #60

Earlier quoted context omitted.

I'm really not a fan of using the generally unsupported aspects of the file formats. Like sure, png supports alternate compression schemes which in combination with the fully generic datablocks system which I think technically accidentally makes it Turing complete. But its not like any reader ever supports it fully, hell most readers dont support showing the other images in a png file. Its also quite common that pale…

I think I don't understand what you mean by an AR encoder

Its auto regression. Think of a row of pixels as a signal y[n]. Assume y[n] = a y[n-1] + b y[n-2]. estimate a,b. Then store y[0], y[1], a, b, and the residual, then encode the four values and the residual which will mostly be zeros. You can vary the length and number of coefficients very cheaply, but a fixed two beat png in my tests. There are fast standard algorithms to find a,b and which along with some simple tricks to correct for the precision ensure lossless encoding. Its the mathematicians version of the ad hoc thing png tries to do.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#104

Post by midjji »

spider-mario wrote: post #80
midjji wrote: post #60

Earlier quoted context omitted.

I'm really not a fan of using the generally unsupported aspects of the file formats. Like sure, png supports alternate compression schemes which in combination with the fully generic datablocks system which I think technically accidentally makes it Turing complete. But its not like any reader ever supports it fully, hell most readers dont support showing the other images in a png file. Its also quite common that pale…

> But its not like any reader ever supports it fully, hell most readers dont support showing the other images in a png file. Do you mean APNG? In all fairness, that is not even in the specification although there is discussion to add it. https://github.com/w3c/PNG-spec/issues/26 The one that was specified was MNG but it was a different format and practically no one used it since it was not parsable as a PNG.

No not animated, I mean that the .png format specifies how to specify an arbitrary number of datablocks, without specifying how they should be used aside from amounts to an recommendation of which one should be shown.

It used by some videogames as assets, for instance storing what amounts as a thumbnail as a shown image, but leaving meshes and textures in the other internal datablocks.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#105

Post by aspyct »

rocqua wrote: post #82
aspyct wrote: post #75

I would love to have something to compress the raw files from my camera. They're huge, I have to keep a ton of them, and I also need to transmit them over internet for my backup. I tried a few standard compression format, with very little luck. Canon has devised a very smart (slightly lossy) compression format for newer cameras, but there's no converter that I know of for my old camera files. So, unless I shell out l…

There is the option of converting to DNG files. Which allow for really good lossless compression. This does come at the cost of changing the file format, and risks losing metadata. That's why I personally decided to just buy more storage instead. Come to think of it, have you tried running a modern compression algorithm on the data? I don't think I did. Could be cool if combined with ZFS or similar to get the compres…

Converting the CR2 (Canon) to DNG tends to double or even triple it's size, but I haven't tried compressing it afterwards.

I should, as you suggest, test a more exhaustive list of formats, who knows...

Post reply on HN