Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bzip3 – A better and stronger spiritual successor to bzip2

github.com

Post reply on HN
Print view View original

1–10 of 107 posts

Re: Bzip3 – A better and stronger spiritual successor to bzip2

#3

Post by roelschroeven »

asicsp wrote: post #2

Good work! I was also confused with faster speed claims than bzip2, and then saw the discussion in the issue: https://github.com/kspalaiologos/bzip3/issues/2

That discussion doesn't really clear up my confusion though.

I don't understand how bzip3 gets to claim "A better, faster and stronger spiritual successor to BZip2." when even all its own benchmarks show it's slower than bzip2?

Re: Bzip3 – A better and stronger spiritual successor to bzip2

#4

Post by easytiger »

roelschroeven wrote: post #3
asicsp wrote: post #2

Good work! I was also confused with faster speed claims than bzip2, and then saw the discussion in the issue: https://github.com/kspalaiologos/bzip3/issues/2

That discussion doesn't really clear up my confusion though. I don't understand how bzip3 gets to claim "A better, faster and stronger spiritual successor to BZip2." when even all its own benchmarks show it's slower than bzip2?

And what does stronger mean? It's not cryptography.

Unless it is

Re: Bzip3 – A better and stronger spiritual successor to bzip2

#5

Post by verytrivial »

easytiger wrote: post #4
roelschroeven wrote: post #3

Earlier quoted context omitted.

That discussion doesn't really clear up my confusion though. I don't understand how bzip3 gets to claim "A better, faster and stronger spiritual successor to BZip2." when even all its own benchmarks show it's slower than bzip2?

And what does stronger mean? It's not cryptography. Unless it is

I read it as three quarters of a Daft Punk reference (only).

There is no shortage of compression tools that are better in this or that. At the very least this is a fun engineering exercise, but there is always a massive inertia propblem regarding install base with compression, esp. for data-at-rest. gzip is still used by default in so many contexts, and that's not such a bad thing IMHO.

Re: Bzip3 – A better and stronger spiritual successor to bzip2

#6

Post by palaiologos »

roelschroeven wrote: post #3
asicsp wrote: post #2

Good work! I was also confused with faster speed claims than bzip2, and then saw the discussion in the issue: https://github.com/kspalaiologos/bzip3/issues/2

That discussion doesn't really clear up my confusion though. I don't understand how bzip3 gets to claim "A better, faster and stronger spiritual successor to BZip2." when even all its own benchmarks show it's slower than bzip2?

bzip3 usually operates on bigger block sizes, up to 16 times bigger than bzip2. additionally, bzip3 supports parallel compression/decompression out of the box. for fairness, the benchmarks have been performed using single thread mode, but they aren't quite as fair towards bzip3 itself, as it uses a way bigger block size.

what bzip3 aims to be is a replacement for bzip2 on modern hardware. what used to not be viable decades ago (arithmetic coding, context mixing, SAIS algorithms for BWT construction) became viable nowadays, as CPU Frequencies don't tend to change, while cache and RAM keep getting bigger and faster.

it should be noted that while using 16 times larger block sizes than bzip2 while providing compression ratios up to 10%-50% better at a cost of, as empirically shown, 17 seconds per 1.3GB of data, is a pretty good trade-off and if bzip2 wanted to get anywhere close to that (e.g. using the C API to tweak the block size), it'd have to sacrifice a lot of its performance.

Re: Bzip3 – A better and stronger spiritual successor to bzip2

#8

Post by palaiologos »

easytiger wrote: post #4
roelschroeven wrote: post #3

Earlier quoted context omitted.

That discussion doesn't really clear up my confusion though. I don't understand how bzip3 gets to claim "A better, faster and stronger spiritual successor to BZip2." when even all its own benchmarks show it's slower than bzip2?

And what does stronger mean? It's not cryptography. Unless it is

it's fairly common, at least in the circles i usually dwell in, to call compression ratio "compression _strength_". bzip3 is _better_ than bzip2 since it uses a better technological model as outlined in one of my replies.

Re: Bzip3 – A better and stronger spiritual successor to bzip2

#10

Post by once_inc »

denzquix wrote: post #9

From their own benchmarks it seems more like bzip3 is geared towards a different compression/speed trade-off than bzip2, rather than an unambiguous all-around improvement. Am I misreading it?

That's what I took out of it too. Sacrificed a bit of speed and a lot of memory for a smaller output size.

edit: ah, bzip3 is parallelizable, while bzip2 isn't. That alone is enough for me to be able to claim 'faster'.

Post reply on HN