Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

QOI – The “Quite OK Image Format” for fast, lossless image compression

github.com

Post reply on HN
Print view View original

71–80 of 105 posts

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#71

Post by seoaeu »

meltedcapacitor wrote: post #42
FullyFunctional wrote: post #30

Since we are rehashing this for the 3rd (4th?) time, I'll repeat mine (and apparently many others) key critique: there is no thought at all to enabling parallel decoding, be it, thread-parallel or SIMD (or both). That makes it very much a past millennium style format that will age very poorly. At the very least, break it into chunks and add an offset directory header. I'm sure one could do something much better, but…

A thread can scan the opcodes only to find cut-off points and distribute actual decoding to other cores. Surely you can do that with some simd magic, as well as the decoding threads, without needing to encode properties of today's simd in the encoding.

No it can't. The encoder doesn't insert any "cut-off points". In fact, nearly every chunk encodes the current pixel value in terms of one of the previous pixel values, so without knowing those it is impossible for a second core to start up and initialize its decoder state enough to produce correct output.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#72

Post by ajdude »

chrismorgan wrote: post #59
jws wrote: post #21

Earlier quoted context omitted.

I find it interesting that QOI avoids any kind of Huffman style coding. Huffman encoding lets you store frequently used values in fewer bits than rarely occurring values, but the cost of a naïve implementation is a branch on every encoded bit. You can mitigate this by making a state machine keyed by "accumulated prefix bits" and as many bits as you want to process in a whack, these tables will blow out your L1 data c…

Meta: ␄ ( https://en.wikipedia.org/wiki/End-of-Transmission_character ) isn’t the right control character when footnotes follow; ␃ ( https://en.wikipedia.org/wiki/End-of-Text_character ) is a better fit, and ␌ ( https://en.wikipedia.org/wiki/Form_feed ) would be a decent choice too. (I write comments with footnotes in the same style as you, but use “—⁂—” as the separator, via Compose+h+r (name from the HTML tag horiz…

Entirely unrelated to the OP, but comments like this are why I love HN.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#73

Post by Dylan16807 »

FullyFunctional wrote: post #69
Dylan16807 wrote: post #67

Earlier quoted context omitted.

Strawman? I wasn't even trying to characterize anyone else's point, I was just trying to list some significant improvements over x86. > The simplest possible mitigation would have been to disallow an instruction from spanning a 64-byte boundary. Sure, that sounds good. But before this you hadn't even mentioned any problems with split instructions that need to be mitigated. (You did mention decoding without a known en…

I did forget to mention the requirement that you can't branch into the middle of an instruction. If you have both of these constraints then you can unambiguously determine the location of all instructions in any aligned 64-byte block, including at I$ fill time. Implementing this would require instruction fetch to take an exception on line-crossing instructions (which must be illegal) and a change to the assembler to…

You'd also need to guarantee that there are no constants or other non-instruction data in the same cache line as instructions. If that's a reasonable constraint then sure, that sounds like it would be helpful.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#74

Post by versteegen »

throwamon wrote: post #44

Didn't HN disallow recent reposts? This (or its spec) was already posted 3 days ago (twice) and then 2 days ago... https://news.ycombinator.com/item?id=29625084 https://news.ycombinator.com/item?id=29631717 https://news.ycombinator.com/item?id=29643370

It's because this link (https://github.com/phoboslab/qoi) has been approved by mods for reposting: it appears on the "pool" list [1] [2]. Which is a bit odd because as you point out, a different link [3] for the same project already received lots of attention.

[1] https://news.ycombinator.com/pool

[2] https://news.ycombinator.com/item?id=26998308

[3] https://news.ycombinator.com/item?id=29625084

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#75

Post by aspyct »

I would love to have something to compress the raw files from my camera. They're huge, I have to keep a ton of them, and I also need to transmit them over internet for my backup.

I tried a few standard compression format, with very little luck.

Canon has devised a very smart (slightly lossy) compression format for newer cameras, but there's no converter that I know of for my old camera files.

So, unless I shell out large amounts of money for a new camera, I'm stuck sending twice the data over the internet. Talk about pollution...

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#76

Post by cycomanic »

adgjlsfhk1 wrote: post #39
makapuf wrote: post #29

Earlier quoted context omitted.

I tried passing stereo wavs in 2 x 16bits (4bytes) as rgba for qoi but I haven't been very successful.

That's not surprising. QOI is heavily optimized for images which tend to be relatively continuous, while audio tends to oscillate a ton.

It might work to fourier transform first (although likely will kill performance)

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#77

Post by zigzag312 »

TylerE wrote: post #65
zigzag312 wrote: post #23

Earlier quoted context omitted.

WavPack seems a bit too slow already. 3x slower decode compared to FLAC in this test https://stsaz.github.io/fmedia/audio-formats/

Wavpack on a modern CPU, from your own link, decodes at approx. 250x realtime. How fast is 'fast enough' if that isn't?

Projects with 100+ tracks are not uncommon. Sampler/rompler of a single virtual instrument can play 10+ sounds simultaneously. Playback of an orchestral score with virtual instruments can easily go over 250 simultaneous sounds, so just a real-time playback (without any additional processing) would already be a challenge.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#78

Post by adgjlsfhk1 »

cycomanic wrote: post #76
adgjlsfhk1 wrote: post #39

Earlier quoted context omitted.

That's not surprising. QOI is heavily optimized for images which tend to be relatively continuous, while audio tends to oscillate a ton.

It might work to fourier transform first (although likely will kill performance)

fixed size FFT (eg length 64) can be made scary fast.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#79

Post by hulitu »

phoboslab wrote: post #47

Don't expect too much of QOI. I wanted a simple format that allows you to load/save images quickly, without dealing with the complexity of JPEG or PNG. Even BMP, TIFF and other "legacy" formats are way more complicated to handle when you start looking into it. So that's what QOI aims to replace. There's a lot of research for a successor format ongoing. Block based encoding, conversion to YUV, more OP-types etc. have…

> I wanted a simple format that allows you to load/save images quickly, without dealing with the complexity of JPEG or PNG. Even BMP, TIFF and other "legacy" formats are way more complicated to handle when you start looking into it. So that's what QOI aims to replace.

XPM ? compressed with gzip ?

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#80

Post by spider-mario »

midjji wrote: post #60
ErikCorry wrote: post #53

Earlier quoted context omitted.

You should be able to do the same technique, but in a PNG-compatible format.

I'm really not a fan of using the generally unsupported aspects of the file formats. Like sure, png supports alternate compression schemes which in combination with the fully generic datablocks system which I think technically accidentally makes it Turing complete. But its not like any reader ever supports it fully, hell most readers dont support showing the other images in a png file. Its also quite common that pale…

> But its not like any reader ever supports it fully, hell most readers dont support showing the other images in a png file.

Do you mean APNG? In all fairness, that is not even in the specification although there is discussion to add it. https://github.com/w3c/PNG-spec/issues/26

The one that was specified was MNG but it was a different format and practically no one used it since it was not parsable as a PNG.

Post reply on HN