131–140 of 181 posts
Earlier quoted context omitted.
zstd is a great setup, for sure, but the build system they use is patently awful. Can someone make an autoconf or hell, even CMake build system for them pleasee???
Quick glance of zstd github repo shows they do provide CMake build scripts?
Earlier quoted context omitted.
Can BWT be combined with zstd, which uses asymmetric numeral systems?
Yes, it would actually be interesting to just have a bwt pass which does no compression, so we can then try lots of post compression options.
Earlier quoted context omitted.
Hi, tool author here! Thank you for your benchmark! As you may be aware, different compression tools fill in different data type niches. In particular, less specialised statistical methods (bzip2, bzip3, PPMd) generally perform poorly on vaguely defined binary data due to unnatural distribution of the underlying data that at least in bzip3's case does not lend well to suffix sorting. Conversely, Lempel-Ziv methods us…
Shouldn't a modern compression tool, targeting a high compression rate, try to switch its compression method on the fly depending on the input data? I have no idea about compression, just a naive thought.
Earlier quoted context omitted.
I suppose the point of open source software is to edit the source code and make your own version. If you can't build it, you can't do that.
Zstd has lots of contributors who have all figured out how to edit and build it.
Earlier quoted context omitted.
Zstd has lots of contributors who have all figured out how to edit and build it.
Yeah, not sure what the original commenter is complaining about. I did "git clone https://github.com/facebook/zstd ", "cd zstd", "make" and have a working binary. Doesn't get easier than that.
Earlier quoted context omitted.
Shouldn’t knowing how big it’s supposed to be make it easier to stop a zip bomb? Just stop decompressing once you hit the size from the header.
That only works if the standard actually describes what you're supposed to do with extra data at the end, and everyone agrees. In practice, there have been antivirus bypasses that made use of AV scanners treating the additional data differently from common extraction software (I believe it was winrar?). One could argue that a text document with a corrupt file size byte should still be decodeable. One could also argue…
Earlier quoted context omitted.
The bash 4 thing is due to the GPL 3, not some inherent slowness in updating software. It has nothing to do with zstd, which is permissively licensed.
It doesn't matter why. All that matters is that "current" is not a valid word. Old things exist in "current" systems. And current systems may also be old systems.
I don't even know why we're talking about bash here. But for the record, this is a unique circumstance that affects only software which made the transition from GPLv2 -> GPLv3. Changing your license can cause you to fork away from some users. News at 11.
I've studied the Burrows-Wheeler Transform, I understand the transformation, I've re-implemented it countless times for kicks, I see how it improves compressability, but for the life of me the intuition of _why_ it works has never really clicked. It's a fantastic bit of algorithmic magic that will always impress me to see it.
The Burroughs-Wheeler transform has been described as a unique algorithm idea in that there are no non-trivial variations or related algorithms, unlike more conventional compression algorithms, which can be tweaked and improved in so many ways. There is no general compression theory in which BWT could be described as a special case. It looks to me that the above still holds: Bzip2 and Bzip3 are simply combining more…
BWT sorts rotated data and what is achieved is that same suffixes group together:
...
"Bzip2 and Bzip3 are simply combining more"
"Bzip3 are simply combining moreBzip2 and "
BWT also breaks near dependencies in data, this is why move-to-front with Huffman or arithmetic encoding works well there.
(2022)
what do you mean?? usually this signifies the year some article was finalized and published. and this is a GitHub repo with recent commits so this is not correct here.
> Bzip3: A better and stronger spiritual successor to BZip2 Please, no. What's next ? BZip4 and BZip5, each one incompatible with each other ?