How does it work?
LZ4 – Extremely fast compression
21–30 of 115 posts
Re: LZ4 – Extremely fast compression
#22Here’s a fork of the Windows compression tool 7-Zip which has LZ4 support baked in along with some other useful algorithms – the repo has a good comparison of them: https://github.com/mcmilk/7-Zip-zstd/ (Linking to this more for the overview than the Windows tool in itself.)
The frustrating thing about 7zip is that whilst it's opensource, a single author just does code drops.
So there's a bunch of forks with useful features that'll never be adopted because there's no collaboration.
At least that's what I could tell when I looked into it
Re: LZ4 – Extremely fast compression
#23If I remember correctly, it is very popular in video games because it is faster to load compressed assets from disk and decompress them in memory than loading the uncompressed assets from disk, even on an SSD.
I did a search and found this patch note for a game which has a comparison graph comparing cold load, hot load and size of some algorithms:
https://store.steampowered.com/news/app/991270/view/18064466...
Re: LZ4 – Extremely fast compression
#24How does it work?
It compresses stuff. Amazing.
Re: LZ4 – Extremely fast compression
#25LZ4 rocks. Used it in the past with great results (much less CPU intensive then the GZIP we were using, still getting good compression).
Re: LZ4 – Extremely fast compression
#26The wikipedia article about LZ4 says that compression is slightly worse, compression speed is comparable, and decompression can be significantly slower compared to LZO. Can anyone enlighten me why I should not used LZO? I do not know anything about either algorithms, but if I switch from deflate I want to mak the right decision.
Re: LZ4 – Extremely fast compression
#27How does it work?
Most interesting question on this thread and gets downvoted. :)
Re: LZ4 – Extremely fast compression
#28How does it work?
I think it’s middle-out compression
Re: LZ4 – Extremely fast compression
#29Earlier quoted context omitted.
Zfs is so fast it should be the default for everything where a slight compression may benefit the system: disk io, network transfers, ...
I'm curious. I use btrfs daily. Although I have been interested in using zfs, I haven't yet gotten the time. In your experience, is zfs faster than btrfs?
Yes. Much faster. Especially for HDDs. But at a cost of a lot of RAM.
Also lz4 compression can speed up your HDDs up to 10x (!) to read and 3x to write. [1, see "To compress, or not to compress, that is the question" section.] But it's going to have a considerably higher CPU usage as well.
Re: LZ4 – Extremely fast compression
#30Earlier quoted context omitted.
LZ4 has branchless decompression, and lower cache footprint, thus it can work on low end, and non-desktop CPUs equally well. zstd, brotly, snappy were seemingly all made with high end x86 capabilities in mind.
I also appreciate LZ4's simplicity and tiny code footprint. zstd is brilliant as well, but in terms of code base it's a whole other beast.
Yes decompression on baremetal cortex m4 is a mere hundreds of bytes, you can decompress it from flash directly to its output buffer.