Live data from Hacker News

Smaller and faster data compression with Zstandard

code.facebook.com

11–20 of 166 posts

Re: Smaller and faster data compression with Zstandard

#11
Looks very interesting, however I'm not impressed by the name. "Zstandard"??? With ".zstd" as the extension? I don't like it.

They should have named it letter-zip, along the lines of gzip, bzip, and xzip, with the extension letterz. "fz" would have been a good one since they work at Facebook.

Re: Smaller and faster data compression with Zstandard

#13

Looks very interesting, however I'm not impressed by the name. "Zstandard"??? With ".zstd" as the extension? I don't like it. They should have named it letter -zip, along the lines of gzip, bzip, and xzip, with the extension letter z. "fz" would have been a good one since they work at Facebook.

Agreed. How did they not call it "Pied Piper"?

Re: Smaller and faster data compression with Zstandard

#14

Looks very interesting, however I'm not impressed by the name. "Zstandard"??? With ".zstd" as the extension? I don't like it. They should have named it letter -zip, along the lines of gzip, bzip, and xzip, with the extension letter z. "fz" would have been a good one since they work at Facebook.

https://en.wiktionary.org/wiki/bikeshedding

Re: Smaller and faster data compression with Zstandard

#15

Looks very interesting, however I'm not impressed by the name. "Zstandard"??? With ".zstd" as the extension? I don't like it. They should have named it letter -zip, along the lines of gzip, bzip, and xzip, with the extension letter z. "fz" would have been a good one since they work at Facebook.

It used to be called just 'zstd' but I guess that wasn't very pronounceable.

Re: Smaller and faster data compression with Zstandard

#16

Looks very interesting, however I'm not impressed by the name. "Zstandard"??? With ".zstd" as the extension? I don't like it. They should have named it letter -zip, along the lines of gzip, bzip, and xzip, with the extension letter z. "fz" would have been a good one since they work at Facebook.

...yeah, why didn't they claim part of a namespace that only has room for 26 (or 36) things? Everyone else is doing it!

Re: Smaller and faster data compression with Zstandard

#17
post #2

Some more benchmarks on this[0] page Also, I actually discovered something very interesting (to me at least). At the bottom of the link mentioned below, the link attached says https://github.com/Cyan4973/zstd but then redirects to https://github.com/facebook/zstd . Anyone know why? [0]: http://facebook.github.io/zstd/ EDIT: After a little bit of sleuthing, it looks like the author of zstd (github.com/Cyan4973) is now…

> the link attached says https://github.com/Cyan4973/zstd but then redirects to https://github.com/facebook/zstd . Anyone know why? I believe this is just GitHub's standard behavior when a repository is moved to another namespace. We recently renamed https://github.com/D-Programming-Language to https://github.com/dlang , and e.g. https://github.com/D-Programming-Language/dmd redirects in the same way.

Oh, D language is dlang now? Looks like you guys did a redesign of the website and stuff too! I hear good things about D, keep up the great work!

Re: Smaller and faster data compression with Zstandard

#18
There is just so much awesome stuff in this article. Finite State Entropy and Asymmetric Numeral System are completely new concepts to me (I've got 7 open tabs just from references FB supplied in the article), as is repcode modeling. I love that they've already built in granular control over the compression tradeoffs you can make, and I can't wait to look into Huff0. If anyone outside of Facebook has started playing with it or is planning to put it into production right away I'd love to hear about it.

Re: Smaller and faster data compression with Zstandard

#19
post #6
post #2

Some more benchmarks on this[0] page Also, I actually discovered something very interesting (to me at least). At the bottom of the link mentioned below, the link attached says https://github.com/Cyan4973/zstd but then redirects to https://github.com/facebook/zstd . Anyone know why? [0]: http://facebook.github.io/zstd/ EDIT: After a little bit of sleuthing, it looks like the author of zstd (github.com/Cyan4973) is now…

It hasn't been long that I fetched zstd from https://github.com/Cyan4973/zstd Upon looking further, it turns out that the author Yann Collet [0] works with Facebook now; the repo would have been thus transferred to Facebook. Author's blog [1]. [0] https://twitter.com/Cyan4973 [1] http://fastcompression.blogspot.in/

>It hasn't been long that I fetched zstd from https://github.com/Cyan4973/zstd

Yep, and Google too

https://www.google.co.in/search?q=zstd

> it turns out that the author Yann Collet [0] works with Facebook now

yep https://www.facebook.com/yann.collet.73

On another note, wonder what happens when personal and professional lives can interfere. The profile of Yann Collet in the blog post is the above link and I can't help but think, why am I on faceook looking at a baby's picture on someone's profile instead of Github or at least LinkedIn? Seems like something he might want to keep private (and yes, I'm totally assuming here, I don't know his preferences)

Now, I know you can restrict the stuff you post to friends only instead of public like he did, but it is still something people(and facebook for its employees) should consider if they want their facebook profile to become their professional contact page.

Re: Smaller and faster data compression with Zstandard

#20
The plot of compression ratio against speed for the various compression levels is pretty helpful for understanding its performance: https://scontent.fsnc1-3.fna.fbcdn.net/t39.2365-6/14146892_9...

"The x-axis is a decreasing logarithmic scale in megabytes per second; the y-axis is the compression ratio achieved."

I'd love to see a version of this chart that also included Brotli. (And I'm somewhat surprised Brotli isn't mentioned at all.)

(Disclaimer: I work at Google, which made Brotli)

Post reply on HN