Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

bzip3

github.com

Post reply on HN
Print view View original

131–137 of 137 posts

Re: bzip3

#131

Post by palaiologos »

IvanK_net wrote: post #107
palaiologos wrote: post #91

Earlier quoted context omitted.

Tool author here. bzip2 also has this clause, in fact it has been lifted from its dist tarball README verbatim. So does lzma, xz, or in practice any open source program that you use.

Hey, I am the creator of https://www.photopea.com and if a user decides to publish their work, it compresses PSD files (inside a browser) and sends them to the server for storing. Right now, we have about 2 TB of files. I use the DEFLATE compression which I fully implemented myself (compressor and decompressor, inside a library called UZIP.js). I wish I could use something better :D but it must be implemented in Java…

Hi Ivan,

Please send me an e-mail! Perhaps we can figure something out.

Re: bzip3

#132

Post by Leftium »

whatever1 wrote: post #93

I think compression algorithms are ripe for significant improvement with LLMs. It’s an ideal candidate problem you can have in a closed loop evaluation, and you can just let agent try things.

> Compression is prediction

(According to recent article on HN[1]. So LLMs _are_ compression!)

[1]: https://hn.leftium.com/i/49263497

Re: bzip3

#133

Post by LtWorf »

account42 wrote: post #128
LtWorf wrote: post #53

Earlier quoted context omitted.

Please stop questioning people's choice of license. If you don't like it, you're entitled to not using it.

You're also entitled to not comment on discussions you're not interested in. License discussion is very much relevant for anything that might end up being used for data interchange.

Having been annoyed by people who asked me to change license several times, I have no sympathy for these requests.

Re: bzip3

#134

Post by ahknight »

idoubtit wrote: post #99
andruby wrote: post #72

Earlier quoted context omitted.

> “bzip3 is not yet listed on the large text compression benchmark” It is now And it comes in relatively well, in my opinion. I'm a compression amateur, but bzip3 is the first entry I recognize as a general purpose compression program.

Yes, bzip3 compresses that 1GB text file into 170 MB while zstd needs 213 MB (SI units, from my own tests with the "enwik9" file of the GP's page, and the latest releases of both programs, same commands). But the decompression (memory and speed) is a different story. On my desktop, decompressing with zstd requires 128 MB and 1 s, while bzip3 uses 3.2 GB and 90 s. Without `-b 511`, bzip3's ratio decreases and decompre…

Feels like it's the opposite of what it should be. It's fine to impose the world on the compression side, but decompression should be optimized for low-end devices. This is where deflate and zstd get it.

Re: bzip3

#135

Post by algorithm314 »

palaiologos wrote: post #123
algorithm314 wrote: post #122

Earlier quoted context omitted.

Source of agiannis_text. He probably decompiled it.

Tool author here. Sure, out of all bait in this thread I will bite this one. Curiously, it seems like Konstantinos has opened this ticket, i.e. https://github.com/iczelia/bzip3/issues/177 , and open-sourced his algorithm here -- https://codeberg.org/kagiannis/gdcc-2021 . As the allegation is very serious, I will also copy the edited version of this response to my website. I have not seen his tool before, or even was…

sorry palaiologos. I just looked into the code and it is different enough from my compressor. I just looked the description and it looked similar sorry. Just tested it with enwik9

agiannis_text enc 0m16,319s dec 0m15,594s size 270365918

bzip3 enc 1m36,544s dec 1m43,699s size 169990721

Re: bzip3

#136

Post by alerque »

JdeBP wrote: post #24

An interesting unintentional benchmark is to go to https://github.com/iczelia/bzip3/releases and see to what degree bzip3 compresses its own release archives; and go to https://github.com/iczelia/bzip3/blob/master/.github/workflo... to see what options have been chosen for the other compressors here.

As the one who contributed the autotools build system and CI job for posting source tarballs in various formats, I can assure you it was not "unintentional". Distributing that many different compression formats for the same source archive and even including an uncompressed version of the same archive has little practical purpose except benchmarking / demonstration.

Re: bzip3

#137

Post by alerque »

sedatk wrote: post #7

The latest release is a year ago, the last commit is two months ago, and the build is failing. The claim “stronger than bzip2” is strange. What does it even mean? Also, comparing parallel decompression benchmarks with bzip2 instead of pbzip2 seems unfair.

If your basic single purposes tools are getting released every other day that might be a bad sign. The latest release being a year old for a tool like this is not something to worry about.

As for your claim that "the build is failing", that is false, and it was false when you wrote this. The CI job for cross compiling to armv6 was failing, but all the other builds were green. You can still check CI status for older releases to confirm my assertion. Now armv6 is fixed too.

Maybe dial back the FUD.

Post reply on HN