Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

QOI – The “Quite OK Image Format” for fast, lossless image compression

github.com

Post reply on HN
Print view View original

31–40 of 105 posts

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#31

Post by wombatmobile »

zigzag312 wrote: post #3

Is there any open source audio compression format like that? Lossless and very fast. I haven't found any yet. EDIT: I'm thinking about a format that would be suitable as a replacement for uncompressed WAV files in DAWs. Rendered tracks often have large sections of silence and uncompressed WAVs have always seemed wasteful to me.

I'd also like to know what's the best (or any) lossless audio compression process/tools.

My application is to send audio (podcast recordings) to a remote audio engineer friend who will do the post processing, then round trip it to me to complete the editing.

Wav is so big it makes a 1 hr podcast a difficult proposition.

MP3 is unsuitable because compression introduces too many artefacts the quality suffers unacceptably.

What do other people do in this circumstances?

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#32

Post by stathibus »

FullyFunctional wrote: post #30

Since we are rehashing this for the 3rd (4th?) time, I'll repeat mine (and apparently many others) key critique: there is no thought at all to enabling parallel decoding, be it, thread-parallel or SIMD (or both). That makes it very much a past millennium style format that will age very poorly. At the very least, break it into chunks and add an offset directory header. I'm sure one could do something much better, but…

Who cares that it's not set up for simd?

Seriously, who?

This project is interesting because of how well it does compared to other systems of much higher complexity and without optimizing the implementation to high heaven. We can all learn something from that.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#33

Post by BlueSwordM »

323 wrote: post #26
zigzag312 wrote: post #24

Earlier quoted context omitted.

Reaper does. Unfortunately, WavPack has a bit too much performance overhead.

It's a 20 year old format. ZStandard is a very good compressor, with an especially fast decompressor. Maybe someone should try using this instead of zlib in an audio format (FLAC, WavPack, ...)

I mean, is there really a need for utilizing ZStd for audio compression?

FLAC is extremely good at compression audio, has very fast encode and uber fast decode. It also doesn't use zlib...

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#34

Post by selectodude »

wombatmobile wrote: post #31
zigzag312 wrote: post #3

Is there any open source audio compression format like that? Lossless and very fast. I haven't found any yet. EDIT: I'm thinking about a format that would be suitable as a replacement for uncompressed WAV files in DAWs. Rendered tracks often have large sections of silence and uncompressed WAVs have always seemed wasteful to me.

I'd also like to know what's the best (or any) lossless audio compression process/tools. My application is to send audio (podcast recordings) to a remote audio engineer friend who will do the post processing, then round trip it to me to complete the editing. Wav is so big it makes a 1 hr podcast a difficult proposition. MP3 is unsuitable because compression introduces too many artefacts the quality suffers unacceptab…

FLAC and ALAC can be losslessly converted to back to WAV and cuts the file size in half.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#35

Post by FullyFunctional »

stathibus wrote: post #32
FullyFunctional wrote: post #30

Since we are rehashing this for the 3rd (4th?) time, I'll repeat mine (and apparently many others) key critique: there is no thought at all to enabling parallel decoding, be it, thread-parallel or SIMD (or both). That makes it very much a past millennium style format that will age very poorly. At the very least, break it into chunks and add an offset directory header. I'm sure one could do something much better, but…

Who cares that it's not set up for simd? Seriously, who? This project is interesting because of how well it does compared to other systems of much higher complexity and without optimizing the implementation to high heaven. We can all learn something from that.

Good question. The answer is all the poor souls that N years later find themselves stuck with a data in a legacy format that they have to struggle to decode faster.

Of all the artifacts in our industry, few things live longer than formats. Eg. we are still unpacking tar files (Tape ARchieve), transmitted over IPv4, decoded by machines running x86 processors (and others, sure). All of these formats couldn't possible anticipate the evolution that follow nor predicted the explosive popularity they would have. And all of these (the latter two notably) have overheads that have real material costs. IPv6 fixed all the misaligned fields, but IPv4 is still dominant. Ironically, RISC-V didn't learn from x86 but added variable length instructions making decoding harder to scale than necessary.

I'm not sure what positive lessons you think we should learn from QOI. It's not hard to come up with simple formats. It's much harder coming up with a format that learns from past failures and avoids future pitfalls.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#36

Post by ricardobeat »

FullyFunctional wrote: post #35
stathibus wrote: post #32

Earlier quoted context omitted.

Who cares that it's not set up for simd? Seriously, who? This project is interesting because of how well it does compared to other systems of much higher complexity and without optimizing the implementation to high heaven. We can all learn something from that.

Good question. The answer is all the poor souls that N years later find themselves stuck with a data in a legacy format that they have to struggle to decode faster. Of all the artifacts in our industry, few things live longer than formats. Eg. we are still unpacking tar files (Tape ARchieve), transmitted over IPv4, decoded by machines running x86 processors (and others, sure). All of these formats couldn't possible a…

QOI is designed with a very specific purpose in mind, which is fast decoding for games. This kind of image will be very unlikely be large enough to benefit from multi threading, and if you have a lot of them you can simply decode in parallel. It’s not meant to the the “best” image format.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#37

Post by nynx »

FullyFunctional wrote: post #35
stathibus wrote: post #32

Earlier quoted context omitted.

Who cares that it's not set up for simd? Seriously, who? This project is interesting because of how well it does compared to other systems of much higher complexity and without optimizing the implementation to high heaven. We can all learn something from that.

Good question. The answer is all the poor souls that N years later find themselves stuck with a data in a legacy format that they have to struggle to decode faster. Of all the artifacts in our industry, few things live longer than formats. Eg. we are still unpacking tar files (Tape ARchieve), transmitted over IPv4, decoded by machines running x86 processors (and others, sure). All of these formats couldn't possible a…

Unrelated to the rest of your comment, but risc-v does not have variable-length instructions. It has compressed instructions, but they're designed in such a way to be easily and efficiently integrated into the decoder for normal instructions, which are all 32 bits.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#38

Post by adgjlsfhk1 »

jws wrote: post #21
sreekotay wrote: post #10

If QOI is interesting because of speed, you might take a look at fpng, a recent/actively developed png reader/writer that is achieving comparable speed/compression to QOI, while staying png compliant. https://github.com/richgel999/fpng Disclaimer: have not actively tried either.

I find it interesting that QOI avoids any kind of Huffman style coding. Huffman encoding lets you store frequently used values in fewer bits than rarely occurring values, but the cost of a naïve implementation is a branch on every encoded bit. You can mitigate this by making a state machine keyed by "accumulated prefix bits" and as many bits as you want to process in a whack, these tables will blow out your L1 data c…

One thing to note is that QOI composes really nicely with high quality entropy encoders like LZ4 and ZSTD. LZ4 gives a roughly 5% size reduction with negligible speed impact, and ZSTD gives a 20% size reduction with moderate speed impact (https://github.com/nigeltao/qoi2-bikeshed/issues/25).

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#39

Post by adgjlsfhk1 »

makapuf wrote: post #29
LeoPanthera wrote: post #7

Earlier quoted context omitted.

FLAC is always lossless, but has a variable compression ratio so you can trade compression for speed. Using the command line "flac" tool, "flac -0" is the fastest, "flac -8" is the slowest, but produces the smallest files. In my experience, 0-2 all produce roughly equivalent sized files, as do 4-8.

I tried passing stereo wavs in 2 x 16bits (4bytes) as rgba for qoi but I haven't been very successful.

That's not surprising. QOI is heavily optimized for images which tend to be relatively continuous, while audio tends to oscillate a ton.

Re: QOI – The “Quite OK Image Format” for fast, lossless image compression

#40

Post by StreamBright »

zigzag312 wrote: post #8
StreamBright wrote: post #6

Earlier quoted context omitted.

ALAC? FLAC? What is the problem with these?

FLAC is limited to 24 bit depth. I was thinking of intermediate format suitable for use in DAWs and samplers that also supports floating point to avoid clipping.

Nice, I was not aware.
Post reply on HN