Live data from Hacker News

Now using Zstandard instead of xz for package compression

archlinux.org

81–90 of 155 posts

Re: Now using Zstandard instead of xz for package compression

#81
post #78

Earlier quoted context omitted.

mkinitcpio is being replaced with dracut so zstd won't probably happen.

Man page says zstd is an option on dracut http://man7.org/linux/man-pages/man5/dracut.conf.5.html

The kernel doesn't support booting zstd compressed initramfs' yet, but you can very well use zstd compression with dracut and mkinitcpio

Re: Now using Zstandard instead of xz for package compression

#82
post #43

Earlier quoted context omitted.

I'm talking about the difference in size and contrast between the actual headline and the trailing HN sitebit "(archlinux.org)". The final size they end up on my screen is irrelevant to my point, because my point is about the size and contrast difference _between_ the two, whatever final sizes those might happen to be. The default HN stylesheet calls for 10pt and 8pt for those, respectively, so it's not like I'm just…

So even if the url got bumped to 12pt, you'd complain if the rest was 15? I think that's weird. As long as it's on the same line as the title and easily legible, I really don't see a problem. And it's not a spot of secondary importance. If it was still in the title, making it longer, the spot where you see the url would have title in it.

FWIW 12 vs 15pt text used like this is called visual hierarchy (or typographic hierarchy if the differences are limited to typography, they're not here, there's also positional difference between the titles and domains).

Desaturating and down-sizing suggests the information is not important, but in this case is critical to understanding and of equal weight and so should probably have the same visual hierarchy, that's easily achieved by including "Arch Linux" in the title.

Re: Now using Zstandard instead of xz for package compression

#83
Zstandard is awesome!

Earlier last year I was doing some research that involved repeatedly grepping through over a terabyte of data, most of which were tiny text files that I had to un-zip/7zip/rar/tar and it was painful (maybe I needed a better laptop).

With Zstd I was able to re-compress the whole thing down to a few hundred gigs and use ripgrep which solved the problem beautifully.

Out of curiosity I tested compression with (single-threaded) lz4 and found that multi-threaded zstd was pretty close. It was an unscientific and maybe unfair test but I found it amazing that I could get lz4-ish compression speeds at the cost of more CPU but with much better compression ratios.

EDIT: Btw, I use arch :) - yes, on servers too.

Re: Now using Zstandard instead of xz for package compression

#85
post #52

I'd love to see Zstandard accepted in other places where the current option is only the venerable zlib. E.g., git packing, ssh -C. It's got more breadth and is better (ratio / cpu) than zlib at every point in the curve where zlib even participates.

It would be great to see better compression supported by browsers.

Chrome apparently tested zstd out, and it's an improvement over Chrome's forked and optimized zlib on x64, but slower on ARM/Android. https://bugs.chromium.org/p/chromium/issues/detail?id=912902...

Few (or none?) of Chrome's fairly dramatic improvements to zlib have been upstreamed. https://github.com/madler/zlib/issues/346

Edit: Also, if browsers do adopt zstd, it's likely you'll end up with the same situation where they fork their own implementation of zstd. Upstreaming requires signing Facebook's CLA, which has patent clauses that don't work for most.

Re: Now using Zstandard instead of xz for package compression

#86

Apparently this is how to use Zstd with tar if anyone else was wondering: tar -I zstd -xvf archive.tar.zst https://stackoverflow.com/questions/45355277/how-can-i-decom... Hopefully there's another option added to tar that simplifies this if this compression becomes mainstream.

It's not too bad to specify compression separately, as in:

  zstd -cd archive.tar.xst | tar xvf -
it's needed anyway as soon as you step outside of what somebody made an option for, for example encryption.

Re: Now using Zstandard instead of xz for package compression

#87

Earlier quoted context omitted.

I love the AUR, but every single time I have to wait for it to compress Firefox nightly, and then wait for it to immediately decompress it again because the only reason I was building the package in the first place was to install it I about lose my mind. Hopefully this helps, but I really wish AUR helpers would just disable compression and call it a day so I don't have to go mess with config files that would also cha…

> EDIT: nevermind, this doesn't seem to have made this the default for building packages locally, just for ones you download from the official repos. Guess I'll go change that by hand and then still be sad that I can't have it easily disabled entirely for AUR helpers but build my packages with compression. I believe you can supply it via an environment variable if your AUR helper has the ability to set those for `mak…

I'll have to experiment with that, thanks. Still, I don't understand why they don't just do it for me. Maybe they're aggressively caching packages and don't want to take the size hit or something.

Re: Now using Zstandard instead of xz for package compression

#88

Earlier quoted context omitted.

I love the AUR, but every single time I have to wait for it to compress Firefox nightly, and then wait for it to immediately decompress it again because the only reason I was building the package in the first place was to install it I about lose my mind. Hopefully this helps, but I really wish AUR helpers would just disable compression and call it a day so I don't have to go mess with config files that would also cha…

This isn't a function of an AUR helper but rather makepkg itself. In makepkg.conf, ommit compression by specifying: PKGEXT='.pkg.tar' More information can be found at https://wiki.archlinux.org/index.php/Makepkg#Tips_and_tricks

Then it happens for packages I build myself (and want compressed) and those that I just want to install.

Re: Now using Zstandard instead of xz for package compression

#89
post #51

Earlier quoted context omitted.

I'm glad it's not a problem for you. It is for me, though, and the point of accessibility is to enable use by everyone.

You don't get to declare that something is an accessibility problem just because you don't like it, though. You would rate the exact same formatting good or bad based entirely on whether the text next to it is a couple pixels larger. That does not sound like "accessibility". Whether it's hard to read is an accessibility issue. But that's not your complaint.

I have to agree with the previous poster. Sometimes i have absolutely no clue what the article is about because the original title has been edited and the url is not immediately recognizable.

Re: Now using Zstandard instead of xz for package compression

#90
post #37

Earlier quoted context omitted.

With GNU tar, the -a flag is not needed

You don't need it with libarchive-based BSD tar either. It can also extract zipfiles, ISO archives, and many other formats with just "tar xf file.zip".

libarchive is really awesome! I've started using bsdtar everywhere, it's so well done and polished I never felt the need to bother with anything else (same goes for bsdcpio)
Post reply on HN