Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zlib-Rs Is Not Only Safer but Now Outperforming Zlib C Implementations

phoronix.com

Post reply on HN
Print view View original

11–19 of 19 posts

Re: Zlib-Rs Is Not Only Safer but Now Outperforming Zlib C Implementations

#11

Post by 0cf8612b2e1e »

TinkersW wrote: post #9
bennettnate5 wrote: post #6

Fascinating--I wonder to what extent the performance improvements are due to the better constraints Rust hands over to LLVM for optimization (specifically thinking `noalias` rules) vs aggressiv code optimization by the developers to be to the point where they beat the C zlib. The latter could be more difficult to achieve given how long the original zlib has been getting incrementally improved, though.

zlib is really old and out of date, no real reason to use it except legacy. I think most optimization effort goes into newer compression libs like zstd.

Legacy is a bit much. Deflate is probably the only supported compression format in many existing/live standards. For a greenfield project where you can control all clients, sure, use whatever, but zlib is likely going to continue to be used for decades.

Re: Zlib-Rs Is Not Only Safer but Now Outperforming Zlib C Implementations

#12

Post by yjftsjthsd-h »

maplant wrote: post #4
giancarlostoro wrote: post #2

Really interesting, I've never used the Rust version, I can only assume I've used the C implementation indirectly dozens of times if not hundreds, but does anyone know how 1:1 the Rust implementation is? The one area where C will always beat Rust (at least today) will probably always be portability, C compilers run everywhere, on everything, and to anything... Unless Rust has similar capabilities via LLVM?

From the GitHub page, they provide libz-rs-sys, a zlib-compatible C API for usage in non-Rust applications.

That's the most interesting thing to me; is this to the point where you could ex. drop it in to a Gentoo box with nothing but a USE flag and switch everything over to use it? (Sadly it's API compatible but not ABI compatible so we're still talking about needing to recompile, but sometimes that's a minor matter)

Re: Zlib-Rs Is Not Only Safer but Now Outperforming Zlib C Implementations

#13

Post by dralley »

TinkersW wrote: post #9
bennettnate5 wrote: post #6

Fascinating--I wonder to what extent the performance improvements are due to the better constraints Rust hands over to LLVM for optimization (specifically thinking `noalias` rules) vs aggressiv code optimization by the developers to be to the point where they beat the C zlib. The latter could be more difficult to achieve given how long the original zlib has been getting incrementally improved, though.

zlib is really old and out of date, no real reason to use it except legacy. I think most optimization effort goes into newer compression libs like zstd.

The comparison is against zlib-ng, not the OG zlib. So this is still very impressive.

I suspect the results won't be quite as good on Aarch64 or other architectures though. zlib-ng has a pretty wide range of hand-optimized intrinsics whereas zlib-rs seems to only really have x86_64.

Re: Zlib-Rs Is Not Only Safer but Now Outperforming Zlib C Implementations

#14

Post by daveguy »

0cf8612b2e1e wrote: post #11
TinkersW wrote: post #9

Earlier quoted context omitted.

zlib is really old and out of date, no real reason to use it except legacy. I think most optimization effort goes into newer compression libs like zstd.

Legacy is a bit much. Deflate is probably the only supported compression format in many existing/live standards. For a greenfield project where you can control all clients, sure, use whatever, but zlib is likely going to continue to be used for decades.

EDIT: doh! Mixed up Zstd and Zlib. Nothing to see here. (Can't delete with a reply.)

Re: Zlib-Rs Is Not Only Safer but Now Outperforming Zlib C Implementations

#15

Post by dralley »

daveguy wrote: post #14
0cf8612b2e1e wrote: post #11

Earlier quoted context omitted.

Legacy is a bit much. Deflate is probably the only supported compression format in many existing/live standards. For a greenfield project where you can control all clients, sure, use whatever, but zlib is likely going to continue to be used for decades.

EDIT: doh! Mixed up Zstd and Zlib. Nothing to see here. (Can't delete with a reply.)

I think you've confused Zstd (which they were not discussing) for Zlib (the primary implementation of DEFLATE, which is most commonly used by Gzip)

Gzip/DEFLATE is mediocre on most metrics relevant to compression but it's widely used of course. Zstd is better all-around, but much newer and thus doesn't have the same level of adoption.

Re: Zlib-Rs Is Not Only Safer but Now Outperforming Zlib C Implementations

#17

Post by sanxiyn »

dralley wrote: post #13
TinkersW wrote: post #9

Earlier quoted context omitted.

zlib is really old and out of date, no real reason to use it except legacy. I think most optimization effort goes into newer compression libs like zstd.

The comparison is against zlib-ng, not the OG zlib. So this is still very impressive. I suspect the results won't be quite as good on Aarch64 or other architectures though. zlib-ng has a pretty wide range of hand-optimized intrinsics whereas zlib-rs seems to only really have x86_64.

zlib-rs does use ARM CRC32 instructions though.

Re: Zlib-Rs Is Not Only Safer but Now Outperforming Zlib C Implementations

#18

Post by alfiedotwtf »

giancarlostoro wrote: post #2

Really interesting, I've never used the Rust version, I can only assume I've used the C implementation indirectly dozens of times if not hundreds, but does anyone know how 1:1 the Rust implementation is? The one area where C will always beat Rust (at least today) will probably always be portability, C compilers run everywhere, on everything, and to anything... Unless Rust has similar capabilities via LLVM?

To me, the point of using a Rust version over a C version of anything would be for memory safety not performance, so a 1:1 rewrite doesn’t make any sense ie. because what’s the point - you’ve now reproduced any memory dangers!

Re: Zlib-Rs Is Not Only Safer but Now Outperforming Zlib C Implementations

#19

Post by vlovich123 »

alfiedotwtf wrote: post #18
giancarlostoro wrote: post #2

Really interesting, I've never used the Rust version, I can only assume I've used the C implementation indirectly dozens of times if not hundreds, but does anyone know how 1:1 the Rust implementation is? The one area where C will always beat Rust (at least today) will probably always be portability, C compilers run everywhere, on everything, and to anything... Unless Rust has similar capabilities via LLVM?

To me, the point of using a Rust version over a C version of anything would be for memory safety not performance, so a 1:1 rewrite doesn’t make any sense ie. because what’s the point - you’ve now reproduced any memory dangers!

It’s never either or nor does a 1:1 rewrite imply the use of unsafe or reproduced memory dangers. For example, vec[v.len()] has no memory safety issues so even if the C code did that in rust it would just panic. Similarly, the bounds checks can often end up being elided by converting to idiomatic rust with iterators and for loops. While the point for you may not be performance, the reason many systems engineers are excited about Rust replacing c/C++ is precisely because it shows promise in memory safety with equivalent or better performance which makes it an and rather than or proposition
Post reply on HN