The answer is good, but is missing a key section: Salty form: They're all quite slow compared to modern competitors.
What are some of those modern competitors?
How are zlib, gzip and zip related?
51–60 of 81 posts
Re: How are zlib, gzip and zip related?
#52The answer is good, but is missing a key section: Salty form: They're all quite slow compared to modern competitors.
What are some of those modern competitors?
zstd is over 4x faster than zlib, while having a better compression ratio.
Re: How are zlib, gzip and zip related?
#53The answer is good, but is missing a key section: Salty form: They're all quite slow compared to modern competitors.
What are some of those modern competitors?
For zlib compatible workloads, there are cloudflare patches and chromium forks, intel forks, and zlib-ng which are compatible but >50% faster. (I think the cloudflare patches eventually made it into upstream zlib, but you may not see that in your distro for a decade).
lz4 and zstd have both been very popular since their release, they're similar and by the same author, though zstd has had more thorough testing and fuzzing, and is more featureful. lz4 maintains an extremely fast decompression speed.
Snappy also performs very well, with zstd and snappy having very close performance with tuning to achieve comparable compression levels.
In recent years Zstd has started to make heavy inroads in broader usage in OSS with a number of distro package managers moving to it and observing substantial benefits. There are HTTP extensions to make it available which Chrome originally resisted but I believe it's now finally coming there too (https://chromestatus.com/feature/6186023867908096).
In gaming circles there's also Oodle and friends from RAD tools which are now available in Unreal engine as builtin compression offerings (since 4.27+). You could see the effects of this in for example Ark Survival Evolved (250GB) -> Ark Survival Ascended (75GB, with richer models & textures), and associated improved load times.
Re: How are zlib, gzip and zip related?
#54What a great historical summary. Compression has moved on now but having grown up marveling at PKZip and maximizing usable space on very early computers, as well as compression in modems (v42bis ftw!), this field has always seemed magical. These days it generally is better to prefer Zstandard to zlib/gzip for many reasons. And if you need seekable format, consider squashfs as a reasonable choice. These stand on the s…
> These days it generally is better to prefer Zstandard to zlib/gzip for many reasons.
I'd agree for new applications, but just like MP3, .gz files (and by extension .tar.gz/.tgz) and zlib streams will probably be around for a long time for compatibility reasons.
Re: How are zlib, gzip and zip related?
#55Fun fact: in a sense. gzip can have multiple files, but not in a specially useful way ... $ echo meow >cat $ echo woof > dog $ gzip cat $ gzip dog $ cat cat.gz dog.gz >animals.gz $ gunzip animals.gz $ cat animals meow woof
Wow, that's surprising (at least to me)!
Is there a limit in the default gunzip implementation? I'm aware of the concept of ZIP/tar bombs, but I wouldn't have expected gunzip to ever produce more than one output file, at least when invoked without options.
Re: How are zlib, gzip and zip related?
#56There's also pzip/punzip (https://github.com/ybirader) for those wanting more performant (concurrent) zip/unzip.
Disclaimer: I'm the author.
Re: How are zlib, gzip and zip related?
#57What a great historical summary. Compression has moved on now but having grown up marveling at PKZip and maximizing usable space on very early computers, as well as compression in modems (v42bis ftw!), this field has always seemed magical. These days it generally is better to prefer Zstandard to zlib/gzip for many reasons. And if you need seekable format, consider squashfs as a reasonable choice. These stand on the s…
I think zlib/gzip still has its place these days. It's still a decent choice for most use cases. If you don't know what usage patterns your program will see, zlib still might be a good choice. Plus, it's supported virtually everywhere, which makes it interesting for long-term storage. Often, using one of the modern alternatives is not worth the hassle.
Re: How are zlib, gzip and zip related?
#58Fun fact: in a sense. gzip can have multiple files, but not in a specially useful way ... $ echo meow >cat $ echo woof > dog $ gzip cat $ gzip dog $ cat cat.gz dog.gz >animals.gz $ gunzip animals.gz $ cat animals meow woof
Wow, that's surprising (at least to me)! Is there a limit in the default gunzip implementation? I'm aware of the concept of ZIP/tar bombs, but I wouldn't have expected gunzip to ever produce more than one output file, at least when invoked without options.
It only produces one output. It's just a stream of data.
Re: How are zlib, gzip and zip related?
#59Earlier quoted context omitted.
Wow, that's surprising (at least to me)! Is there a limit in the default gunzip implementation? I'm aware of the concept of ZIP/tar bombs, but I wouldn't have expected gunzip to ever produce more than one output file, at least when invoked without options.
It only produces one output. It's just a stream of data.
Ah, I somehow imagined a second `cat` in there. That makes more sense, thank you!
Re: How are zlib, gzip and zip related?
#60Earlier quoted context omitted.
You could publish a little webpage called "An historical note about the Docker prototype" under your own name, which you could then cite on Wikipedia. I think it makes perfect sense as a general and strict policy for an encyclopedia. It would simply be too hard to audit every case to check if it's someone like you, or a crank.
Maybe I should write the story as a comment on hacker news, and link to it ;) Joke aside, I should probably take up on your advice.
Yes. Do this (make sure it's not a top-level submission) and cite the HN comment specifically. Stupid rules deserve stupid compliance.