101–110 of 155 posts
I used zstd for on-the-fly compression of game data for p2p multiplayer synchronization, and got 2-5x as much data (depends on the payload type) in each TCP packet. Sad that it still doesn't get much adoption in the industry.
Zstd knows how to use a user-supplied dictionary at each end. I hope you are doing that. But if latency matters you might better use lz4.
> Recompressing all packages to zstd with our options yields a total ~0.8% increase in package size on all of our packages combined, but the decompression time for all packages saw a ~1300% speedup. Impressive. As a AUR package maintainer I am also wondering how the compression speed is though.
While the speedup is nice pacman still seems to operate sequentially, i.e. download, then decompress one by one. Decompressing while downloading or decompressing in parallel seems like a low-hanging fruit that hasn't been plucked yet that wouldn't have needed any changes to the compressor.
FWIW I really applaud Arch here. Even if it's just a small step. Commercial operating systems should take notice. OS updates should really not take as long as they (mostly) do.
Earlier quoted context omitted.
While the speedup is nice pacman still seems to operate sequentially, i.e. download, then decompress one by one. Decompressing while downloading or decompressing in parallel seems like a low-hanging fruit that hasn't been plucked yet that wouldn't have needed any changes to the compressor.
I might be wrong, but wouldn't it be prudent to first verify the checksum/signature of the downloaded archive before unpacking it? Even when just decompressing there's at least the danger of being zip-bombed (assuming a zip bomb can be constructed for any dictionary-based compression algorithm.) FWIW I really applaud Arch here. Even if it's just a small step. Commercial operating systems should take notice. OS update…
Earlier quoted context omitted.
This isn't a function of an AUR helper but rather makepkg itself. In makepkg.conf, ommit compression by specifying: PKGEXT='.pkg.tar' More information can be found at https://wiki.archlinux.org/index.php/Makepkg#Tips_and_tricks
Then it happens for packages I build myself (and want compressed) and those that I just want to install.
PKGEXT=.pkg.tar makepkg
Meta: This post is yet another victim of the HN verbatim title rule despite the verbatim title making little sense as one of many headlines on a news page. How is "Now using Zstandard instead of xz for package compression" followed by the minuscule low-contrast grey "(archlinux.org)" better than "Arch Linux now using Zstandard instead of xz for package compression" like it was when I originally read this a few hours…
Saying it's "yet another victim" seems slightly too emotive to me. If people can't read the source site's domain after the headline then I agree there wouldn't be much context, but equally, if they can't read that, surely their best solution is to adjust the zoom level in the browser. It's clear you won't get complete context from the headline list plus domain, but a hint of it is provided and if you want more you cl…
I'd love to see Zstandard accepted in other places where the current option is only the venerable zlib. E.g., git packing, ssh -C. It's got more breadth and is better (ratio / cpu) than zlib at every point in the curve where zlib even participates.
It would be great to see better compression supported by browsers.
https://github.com/facebook/zstd#benchmarks
AFAIK (I haven't looked much into it since 2018) it's not widely supported by CDNs, but at least Cloudflare seems to serve it by default (EDIT: must be enabled per-site https://support.cloudflare.com/hc/en-us/articles/200168396-W...)
Earlier quoted context omitted.
Arch is actually surprisingly stable and even with infrequent updates on the order of months still upgrades cleanly most of the time. The caveats to this were the great period of instability when switching to systemd, changing the /usr/lib layout, etc but those changes are now pretty far in the past.
Sure, and I've done partial upgrades and it was mostly fine:) It just surprised me to see the devs going out of their way to support it on volunteer time. On the other hand, maybe that's exactly the reason; maybe someone said "hey look, I can make static packages that are immune to library changes! I guess I'll publish these in case they're useful". Open source is fun like that:)
Meta: This post is yet another victim of the HN verbatim title rule despite the verbatim title making little sense as one of many headlines on a news page. How is "Now using Zstandard instead of xz for package compression" followed by the minuscule low-contrast grey "(archlinux.org)" better than "Arch Linux now using Zstandard instead of xz for package compression" like it was when I originally read this a few hours…
I just woke up to this and was surprised the title was eddied as well. I looked up the guidelines and it looks like I violated the "If the title includes the name of the site, please take it out, because the site name will be displayed after the link." guideline.