21–30 of 145 posts
Not only is this a great read, but the follow up for citations is replied with "I am the reference". If this were reddit I'd post the hot fire gif. Eh, here it's anyway: http://i.imgur.com/VQLGJOL.gif
Then I read the name: Adler: as in Adler-32 (the checksum function that zlib uses).
Then I knew it was real. The darn author of zlib answered the question. That's as close as you're gonna get to "primary source" folks!
It seems like it wouldn't be that hard to create an indexed tar.gz format that's backwards compatible. One way would be to use the last file in the tar as the index, and as files are added, you can remove the index, append the new file, append some basic file metadata and the compressed offset (maybe of the deflate chunk) into the index, update the index size in bytes in a small footer at the end of the index, and ap…
If I recall correctly and it's been a while: The initial header is x bytes from the beginning of the file, or that you have to search for a known key string PKsomething. Then you have to go back and forth between that and the compressed data as you decompress. When compressing files you have to go back to the beginning of the zip file ( or disk1 in a multi disk archive) and update that table with the CRC info. Just w…
Not only is this a great read, but the follow up for citations is replied with "I am the reference". If this were reddit I'd post the hot fire gif. Eh, here it's anyway: http://i.imgur.com/VQLGJOL.gif
I thought he was joking. Then I read the name: Adler: as in Adler-32 (the checksum function that zlib uses). Then I knew it was real. The darn author of zlib answered the question. That's as close as you're gonna get to "primary source" folks!
FYI: Mark Adler is an American software engineer, and has been heavily involved in space exploration. He is best known for his work in the field of data compression as the author of the Adler-32 checksum function, and a co-author of the zlib compression library and gzip. He has contributed to Info-ZIP, and has participated in developing the Portable Network Graphics (PNG) image format. Adler was also the Spirit Cruise Mission Manager for the Mars Exploration Rover mission. (wikipedia)
That's quite a résumé.
One important difference in practice is that zip files needs to be saved to disk to be extracted. gzip files on the other hand can be stream unzipped i.e curl http://example.com/foo.tar.gz | tar zxvf - is possible but not with zip files. I am not sure if this is a limitation of the unzip tool. I would love to know if there is a work around to this.
You can't do that with a .zip file because the file header information is actually at the end of the file. You could work around that by sending the header first, though.
It seems like it wouldn't be that hard to create an indexed tar.gz format that's backwards compatible. One way would be to use the last file in the tar as the index, and as files are added, you can remove the index, append the new file, append some basic file metadata and the compressed offset (maybe of the deflate chunk) into the index, update the index size in bytes in a small footer at the end of the index, and ap…
If I recall correctly and it's been a while: The initial header is x bytes from the beginning of the file, or that you have to search for a known key string PKsomething. Then you have to go back and forth between that and the compressed data as you decompress. When compressing files you have to go back to the beginning of the zip file ( or disk1 in a multi disk archive) and update that table with the CRC info. Just w…
Earlier quoted context omitted.
The accepted answer is often wrong (or obsolete) and grants a small number of points. The obsession with it on SO is a bit odd.
Yeah I tend to think if another answer has double the points of an accepted answer, that answer should come first.
Where do the other popular compression utilities (e.g. bzip2, xzip, lzma, 7zip) fit in to this?
Run Length Encoding
Huffman Encoding
Lempel Ziv (LZ77)
Burrows Wheeler Transform
The interesting part of all these specific implementations is their own specialized way of extending or combining these algorithms for their own niche (or tradeoff point).
Bzip2 is for example, compresses really well because BWT is expensive, but clever - rzip extends the LZ part of bzip2 to look further across the file (instead of a few hundred kb).
Zlib itself has enough of these flags exposed out, so all Zlib isn't really the same - Z_FILTERED, Z_HUFFMAN_ONLY, Z_FIXED, Z_RLE etc. Look at something like Zopfli to see how they can be remixed, provided the tradeoffs of CPU change from the historic positions of Zlib.
One important difference in practice is that zip files needs to be saved to disk to be extracted. gzip files on the other hand can be stream unzipped i.e curl http://example.com/foo.tar.gz | tar zxvf - is possible but not with zip files. I am not sure if this is a limitation of the unzip tool. I would love to know if there is a work around to this.
You can't do that with a .zip file because the file header information is actually at the end of the file. You could work around that by sending the header first, though.
Not only is this a great read, but the follow up for citations is replied with "I am the reference". If this were reddit I'd post the hot fire gif. Eh, here it's anyway: http://i.imgur.com/VQLGJOL.gif
I thought he was joking. Then I read the name: Adler: as in Adler-32 (the checksum function that zlib uses). Then I knew it was real. The darn author of zlib answered the question. That's as close as you're gonna get to "primary source" folks!
http://stackoverflow.com/questions/6493270/why-is-tar-gz-sti...