bzip3
github.com
bzip3
1–10 of 129 posts
Re: bzip3
#2Impressive compression benchmark. Four times smaller than z standard.
Re: bzip3
#3Isn't that XZ?
Re: bzip3
#4Previously:
“Hi, tool author here.” A useful explanation of Burrows-Wheelers transform as used by bzip3: https://news.ycombinator.com/item?id=42902407
“bzip3 is not yet listed on the large text compression benchmark” It is now: https://mattmahoney.net/dc/text.html
(2 years ago, 176 comments) https://news.ycombinator.com/item?id=42899713
(4 years ago, 104 comments) https://news.ycombinator.com/item?id=31324439
Re: bzip3
#5Impressive compression benchmark. Four times smaller than z standard.
with zstd at level 16 with default params (dict size, ...). Serious compression starts at level 19 and with much higher dict sizes.
how is this an honest benchmark:
bzip3 ... 12178M memory
zstd ... 687M memoryRe: bzip3
#6Impressive compression benchmark. Four times smaller than z standard.
I have not experimented with bzip3 recently, but more than a year ago I have done many tests with it.
Initially I was extremely impressed with it, because in a lot of tests it succeeded to compress hard-to-compress files, like movies, and in many cases it demonstrated a much better compromise between speed and compression ratio than zstd, i.e. depending on the command parameters I could make it either compress better than zstd at similar compression/decompression speed, or compress/decompress faster at a similar compression ratio.
Alas, the initial extremely favorable conclusion was short-lived, because trying later bzip3 on other data files gave worse results than zstd.
So the final conclusion was that the performance of bzip3 was somewhat unpredictable, being highly data dependent. For some files it provided outstanding compression ratio or speed, but for others it was inferior.
The problem was that without doing a compression there was no way to guess whether a file would be among those preferred by bzip3 or by zstd or by xz.
So now I would use it only for a file for which I want maximum compression and which I would compress once and decompress many times, so I can afford a very long compression time, during which I would test multiple compression algorithms, including bzip3 and zstd, with multiple parameter choices, and I would eventually choose the one that offers the best compromise between compression ratio and decompression time, for that particular file.
It certainly is a competitive compression algorithm, but unless it has changed since I last tested it, you cannot guess for which files it would win the compression competition.
Re: bzip3
#7The latest release is a year ago, the last commit is two months ago, and the build is failing.
The claim “stronger than bzip2” is strange. What does it even mean?
Also, comparing parallel decompression benchmarks with bzip2 instead of pbzip2 seems unfair.
Re: bzip3
#8Isn't that XZ?
No. It is an unrelated algorithm.
Re: bzip3
#9Impressive compression benchmark. Four times smaller than z standard.
The benchmark is very rudimentary. It does not test different levels/settings apart from its own -b 256/512 (does it affect decompression?), it doesn't measure compression time and memory usage. It does not specify parallel vs single-threaded (it mentions parallel on the one decoding number but what about the others?).
The lrzip test is interesting but it omits for example zstd and doesn't even have (de-)compression timings.
A lot more numbers are needed to present a fair and informative comparison.
I don't want this to be a swipe against bzip3, I only want to point out the presented benchmarks could be a lot better.
Re: bzip3
#10Impressive compression benchmark. Four times smaller than z standard.
with zstd at level 16 with default params (dict size, ...). Serious compression starts at level 19 and with much higher dict sizes. how is this an honest benchmark: bzip3 ... 12178M memory zstd ... 687M memory
There is a comparison with "zstd -19" on the Silesia corpus, showing better compression ratio for bzip3 (47.2 vs 53MB) while being ~5 times faster (and using only half the memory).
Even if the examples are highly cherry-picked, it is quite suprising to me that such pareto-dominance is possible at all.
edit: Tested it myself and found that it often also does slightly worse than zstd -19 in compression ratio but faster (it was slower in one case on "uncompressible" input).
Compression performance vs "zstd -19" seems to depends a lot on actual input data in a very unpredictable way. I'd assume the benchmarks that they show are definitely somewhat cherry-picked.