Dissecting the gzip format (2011) • Hacker News Board

Re: Dissecting the gzip format (2011)

#61

Post by zie » Wed, Sep 18, 2024, 2:06 PM UTC

qhwudbebd wrote: post #54

zie wrote: post #33

Earlier quoted context omitted.

That's cool! Is that a GNU tar only thing? Based on it being a longopt, I'm guessing a GNU tar only thing. That's the problem with these things, it takes a while to get pushed to all the installed copies of tar running around. Perhaps it's time to check: * MacOS Sonoma(14.6) has tar --auto-compress and --zstd * OpenBSD tar does not appear to have it: https://man.openbsd.org/tar * FreeBSD does: https://man.freebsd.org…

Both libarchive ("bsdtar") and GNU tar have -a, which I guess are the only two upstream tar implementations that are still relevant? You're right, it can take a while for these things to propagate downstream though.

Some of us use OpenBSD: https://man.openbsd.org/tar

Re: Dissecting the gzip format (2011)

#62

Post by PROMISE_237 » Wed, Sep 18, 2024, 3:28 PM UTC

Spooky23 wrote: post #28

oneshtein wrote: post #16

Earlier quoted context omitted.

gzip is fast (pigz is even faster), supported everywhere (even in DOS), uses low amounts of memory, and compresses well enough for practical needs. bzip2 is too slow. xz is too complex (see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1068024 ), designed to compress .exe files. lzip is good, but less popular. zstd is good and fast, but less popular.

Another factor is that gzip is over 30 years old and ubiquitous in many contexts. Zstd is awesome, but has only been around for a decade, but seems to be growing.

[dead]

Re: Dissecting the gzip format (2011)

#63

Post by PROMISE_237 » Thu, Sep 19, 2024, 1:30 PM UTC

eapriv wrote: post #15

082349872349872 wrote: post #11

Has anyone taken the coding as compression (when you create repeated behaviour, stuff it in the dictionary via creating a function; switching frameworks is changing initial dicts; etc.) metaphor seriously?

Yes. https://caseymuratori.com/blog_0015

[dead]

Re: Dissecting the gzip format (2011)

#64

Post by PROMISE_237 » Thu, Sep 19, 2024, 2:43 PM UTC

commandlinefan wrote: post #35

082349872349872 wrote: post #11

Has anyone taken the coding as compression (when you create repeated behaviour, stuff it in the dictionary via creating a function; switching frameworks is changing initial dicts; etc.) metaphor seriously?

Sounds like LZW compression to me - is what you're thinking of different than that?

[dead]

Re: Dissecting the gzip format (2011)

#65

Post by PROMISE_237 » Fri, Sep 20, 2024, 11:06 AM UTC

zie wrote: post #33

gloflo wrote: post #14

Earlier quoted context omitted.

... --auto-compress ... foo.tar.zstd

That's cool! Is that a GNU tar only thing? Based on it being a longopt, I'm guessing a GNU tar only thing. That's the problem with these things, it takes a while to get pushed to all the installed copies of tar running around. Perhaps it's time to check: * MacOS Sonoma(14.6) has tar --auto-compress and --zstd * OpenBSD tar does not appear to have it: https://man.openbsd.org/tar * FreeBSD does: https://man.freebsd.org…

[dead]

Re: Dissecting the gzip format (2011)

#66

Post by PROMISE_237 » Fri, Sep 20, 2024, 11:33 AM UTC

082349872349872 wrote: post #58

commandlinefan wrote: post #35

Earlier quoted context omitted.

Sounds like LZW compression to me - is what you're thinking of different than that?

Different in terms of applying the same principle to produce "semantically compressed" code — instead of having each new dictionary entry be a reference to an older one along with the new symbol, each new function will refer to older ones along with some new literal data. (If that still doesn't make sense, see the sibling comment to yours.)

[dead]

Re: Dissecting the gzip format (2011)

#67

Post by qhwudbebd » Sat, Sep 21, 2024, 12:33 PM UTC

zie wrote: post #61

qhwudbebd wrote: post #54

Earlier quoted context omitted.

Both libarchive ("bsdtar") and GNU tar have -a, which I guess are the only two upstream tar implementations that are still relevant? You're right, it can take a while for these things to propagate downstream though.

Some of us use OpenBSD: https://man.openbsd.org/tar

Ooh interesting. I'd assumed (incorrectly) that OpenBSD tar was just libarchive like FreeBSD and NetBSD. I prefer BSD libarchive tar to GNU tar as /bin/tar on my Linux machines too for what it's worth.

Re: Dissecting the gzip format (2011)

#68

Post by zie » Sat, Sep 21, 2024, 5:03 PM UTC

qhwudbebd wrote: post #67

zie wrote: post #61

Earlier quoted context omitted.

Some of us use OpenBSD: https://man.openbsd.org/tar

Ooh interesting. I'd assumed (incorrectly) that OpenBSD tar was just libarchive like FreeBSD and NetBSD. I prefer BSD libarchive tar to GNU tar as /bin/tar on my Linux machines too for what it's worth.

OpenBSD is fairly brutal on allowing third party dependencies into base. I'm sure there is a package for both libarchive and gnu tar(Haven't actually checked, but I'd be surprised if there is not).