Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

QOI – The “Quite OK Image Format” for fast, lossless image compression

github.com

Post reply on HN
Print view View original

81–90 of 105 posts

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#81

Post by abainbridge »

hulitu wrote: post #79
phoboslab wrote: post #47

Don't expect too much of QOI. I wanted a simple format that allows you to load/save images quickly, without dealing with the complexity of JPEG or PNG. Even BMP, TIFF and other "legacy" formats are way more complicated to handle when you start looking into it. So that's what QOI aims to replace. There's a lot of research for a successor format ongoing. Block based encoding, conversion to YUV, more OP-types etc. have…

> I wanted a simple format that allows you to load/save images quickly, without dealing with the complexity of JPEG or PNG. Even BMP, TIFF and other "legacy" formats are way more complicated to handle when you start looking into it. So that's what QOI aims to replace. XPM ? compressed with gzip ?

XPM and gzip are still not that simple. QOI is much simpler.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#82

Post by rocqua »

aspyct wrote: post #75

I would love to have something to compress the raw files from my camera. They're huge, I have to keep a ton of them, and I also need to transmit them over internet for my backup. I tried a few standard compression format, with very little luck. Canon has devised a very smart (slightly lossy) compression format for newer cameras, but there's no converter that I know of for my old camera files. So, unless I shell out l…

There is the option of converting to DNG files. Which allow for really good lossless compression. This does come at the cost of changing the file format, and risks losing metadata. That's why I personally decided to just buy more storage instead.

Come to think of it, have you tried running a modern compression algorithm on the data? I don't think I did. Could be cool if combined with ZFS or similar to get the compression done transparently.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#83

Post by ErikCorry »

midjji wrote: post #60
ErikCorry wrote: post #53

Earlier quoted context omitted.

You should be able to do the same technique, but in a PNG-compatible format.

I'm really not a fan of using the generally unsupported aspects of the file formats. Like sure, png supports alternate compression schemes which in combination with the fully generic datablocks system which I think technically accidentally makes it Turing complete. But its not like any reader ever supports it fully, hell most readers dont support showing the other images in a png file. Its also quite common that pale…

I think I don't understand what you mean by an AR encoder

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#84

Post by TylerE »

zigzag312 wrote: post #77
TylerE wrote: post #65

Earlier quoted context omitted.

Wavpack on a modern CPU, from your own link, decodes at approx. 250x realtime. How fast is 'fast enough' if that isn't?

Projects with 100+ tracks are not uncommon. Sampler/rompler of a single virtual instrument can play 10+ sounds simultaneously. Playback of an orchestral score with virtual instruments can easily go over 250 simultaneous sounds, so just a real-time playback (without any additional processing) would already be a challenge.

In those contexts samples are loaded once and then kept in RAM.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#85

Post by meltedcapacitor »

seoaeu wrote: post #71
meltedcapacitor wrote: post #42

Earlier quoted context omitted.

A thread can scan the opcodes only to find cut-off points and distribute actual decoding to other cores. Surely you can do that with some simd magic, as well as the decoding threads, without needing to encode properties of today's simd in the encoding.

No it can't. The encoder doesn't insert any "cut-off points". In fact, nearly every chunk encodes the current pixel value in terms of one of the previous pixel values, so without knowing those it is impossible for a second core to start up and initialize its decoder state enough to produce correct output.

Read again the proposal.

A top level thread scans the opcodes only to solve this, with no decoding and no writing, thus progressing faster in the stream than the child chunked decoding threads it progressively spawns.

Not as quick as a format with a chunk table, but faster than naive single core.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#86

Post by meltedcapacitor »

adgjlsfhk1 wrote: post #48
meltedcapacitor wrote: post #43

It is nice but pity it does not have a "turn right" opcode: start going left, on the turn opcode, continue decoding pixels after turning 90 degrees to the right, until you hit a previously decoded pixel or the wall of the bounding box defined after the first two turns, in which case you turn automatically. The file ends when there's nowhere to turn. This would eliminate the need for a header (bloat!) as the end of fi…

Writing an encoder for that sounds like a nightmare though. Also the speed would suck since you would have unpredictable memory accesses.

An encoder just walking a regular spiral (no uniform bands detection) is not hard. The band thing is an accidental artefact of the idea but plain run length encoding probably already captures most of the effect so no imperative to actually implement it.

Speed, yes, it is a fair objection, until hardware adopts spiral encoding :-)

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#87

Post by causality0 »

PostThisTooFast wrote: post #49
phoboslab wrote: post #47

Don't expect too much of QOI. I wanted a simple format that allows you to load/save images quickly, without dealing with the complexity of JPEG or PNG. Even BMP, TIFF and other "legacy" formats are way more complicated to handle when you start looking into it. So that's what QOI aims to replace. There's a lot of research for a successor format ongoing. Block based encoding, conversion to YUV, more OP-types etc. have…

What are we to make of that warning? What shortcomings do you think people will find?

Mostly the fact people have found and will find shortcomings that won't be fixed because the project is done, like everything being big-endian.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#88

Post by flohofwoe »

PostThisTooFast wrote: post #49
phoboslab wrote: post #47

Don't expect too much of QOI. I wanted a simple format that allows you to load/save images quickly, without dealing with the complexity of JPEG or PNG. Even BMP, TIFF and other "legacy" formats are way more complicated to handle when you start looking into it. So that's what QOI aims to replace. There's a lot of research for a successor format ongoing. Block based encoding, conversion to YUV, more OP-types etc. have…

What are we to make of that warning? What shortcomings do you think people will find?

It's simply better suited for some types of images than others (e.g. the resulting size is sometimes bigger than expected). The main advantage is the very simple encoder and decoder with a specification that fits on a single page (and which still yields surprisingly good results for many image types):

https://qoiformat.org/qoi-specification.pdf

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#89

Post by flohofwoe »

causality0 wrote: post #87
PostThisTooFast wrote: post #49

Earlier quoted context omitted.

What are we to make of that warning? What shortcomings do you think people will find?

Mostly the fact people have found and will find shortcomings that won't be fixed because the project is done, like everything being big-endian.

"Everything" means two 32-bit integer values (width and height) in the header, that's hardly much of a downside ;)

Usually it's a good idea anyway to read file headers byte by byte instead of mapping a struct over it to avoid alignment, padding and endianness issues.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#90

Post by zigzag312 »

TylerE wrote: post #84
zigzag312 wrote: post #77

Earlier quoted context omitted.

Projects with 100+ tracks are not uncommon. Sampler/rompler of a single virtual instrument can play 10+ sounds simultaneously. Playback of an orchestral score with virtual instruments can easily go over 250 simultaneous sounds, so just a real-time playback (without any additional processing) would already be a challenge.

In those contexts samples are loaded once and then kept in RAM.

No, actually only first few dozen KB of each sample are usually preloaded into RAM. The rest is streamed from a SSD. One library of an orchestral section can have 100+ GB of samples. You wouldn't fit all sections in 128GB of RAM.
Post reply on HN