Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zlib-rs is faster than C

trifectatech.org

Post reply on HN
Print view View original

161–170 of 492 posts

Re: Zlib-rs is faster than C

#161

Post by gf000 »

kahlonel wrote: post #7

You mean the implementation is faster than the one in C. Because nothing is “faster than C”.

Wtf, since when?

Besides the famous "C is not a low-level language" blog post.. I don't even get what you are thinking. C is not even the performance queen for large programs (the de facto standard today is C++ for good reasons), let alone for tiny ultra hot loops like codecs and stuff, which are all hand-written assembly.

It's not even hard to beat C with something like Rust or C++, because you can properly do high level optimizations as the language is expressive enough for that.

Re: Zlib-rs is faster than C

#162

Post by capitainenemo »

wavemode wrote: post #160
gf000 wrote: post #145

Earlier quoted context omitted.

Even innocent looking C code can be chock-full of UBs that can invalidate your "local reasoning" capabilities. So, not even close.

Care to share an example?

sorting floats with NaN ? almost anything involving threading and mutation where people either don't realise how important locks are, or don't realise their code has suddenly been threaded?

Re: Zlib-rs is faster than C

#163

Post by NobodyNada »

pclmulqdq wrote: post #95
cmrdporcupine wrote: post #77

Earlier quoted context omitted.

There's no standardization of simd in Rust yet, they've been sitting in nightly unstable for years: https://doc.rust-lang.org/std/intrinsics/simd/index.html So I suspect it's a matter of two things: 1. You're calling out to what's basically assembly, so buyer beware. This is basically FFI into C/asm. 2. There's no guarantee on what comes out of those 128-bit vectors after to follow any sanity or expectations, so... b…

> they've been sitting in nightly unstable for years So many very useful features of Rust and its core library spend years in "nightly" because the maintainers of those features don't have the discipline to see them through.

Before I started working with Rust, I spent a lot of time using Swift for systems-y/server-side code, outside of the Apple ecosystem. There is a lot I like about that language, but one of the biggest factors that drove me away was just how fast the Apple team was to add more and more compiler-magic features without considering whether they were really the best possible design. (One example: adding compiler-magic derived implementations of specific protocols instead of an extensible macro system like Rust has.) When these concerns were raised on the mailing lists, the response from leadership was "yes, something like that would be better in the long run, but we want to ship this now." Or even in one case, "yes, that tweak to the design would be better, but we already showed off the old design at the WWDC keynote and we don't want to break code we put in a keynote slide."

When I started working in Rust, I'd want some feature or function, look it up, and find it was unstable, sometimes for years. This was frustrating at first, but then I'd go read the GitHub issue thread and find that there was some design or implementation concern that needed to be overcome, and that people were actively working on it and unwilling to stabilize the feature until they were sure it was the best possible design. And the result of that is that features that do get stabilized are well thought out, generalize, and compose well with everything else in the language.

Yes, I really want things like portable SIMD, allocators, generators, or Iterator::intersperse. But programming languages are the one place I really do want perfect to be the enemy of good. I'd rather it take 5+ years to stabilize features than for us to end up with another Swift or C++.

Re: Zlib-rs is faster than C

#164

Post by gf000 »

dijit wrote: post #79
kahlonel wrote: post #7

You mean the implementation is faster than the one in C. Because nothing is “faster than C”.

The kind of code you can write in rust can indeed be faster than C, but someone will wax poetic about how anything is possible in C and they would be valid. The major reason that rust can be faster than C though, is because due to the way the compiler is constructed, you can lean on threading idiomatically. The same can be true for Go, coroutines vs no coroutines in some cases is going to be faster for the use case.…

> but someone will wax poetic about how anything is possible in C and they would be valid.

Not at all would that be valid.

C has a semantic model which was close to how early CPUs worked, but a lot has changed since. It's more like CPUs deliberately expose an API so that C programmers could feel at home, but stuff like SIMD and the like is non-existent in C besides as compiler extensions. But even just calling conventions, the stack, etc are all stuff you have no real control over in the C language, and a more optimal version of your code might want to do so. Sure, the compiler might be sufficiently smart, but then it might as well convert my Python script to that ultra-efficient machine code, right?

So no, you simply can't write everything in C, something like simd-json is just not possible. Can you put inline assembly into C? Yeah, but I can also call inline assembly from Scratch and JS, that's not C at all.

Also, Go is not even playing in the same ballpark as C/C++/Rust.

Re: Zlib-rs is faster than C

#165

Post by jeroenhd »

jrockway wrote: post #55

Chromium is kind of stuck with zlib because it's the algorithm that's in the standards, but if you're making your own protocol, you can do even better than this by picking a better algorithm. Zstandard is faster and compresses better. LZ4 is much faster, but not quite as small. Some reading: https://jolynch.github.io/posts/use_fast_data_algorithms/ (As an aside, at my last job container pushes / pulls were in the dev…

`Content-Encoding: zstd` was added to Chromium a while ago: https://chromestatus.com/feature/6186023867908096

You can still use deflate for compression, but Brotli and Zstd have been available in all modern browsers for quite some time.

Re: Zlib-rs is faster than C

#166

Post by [deleted] »

YZF wrote: post #18

I found out I already know Rust: unsafe { let x_tmp0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x10); xmm_crc0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x01); xmm_crc1 = _mm_xor_si128(xmm_crc1, x_tmp0); xmm_crc1 = _mm_xor_si128(xmm_crc1, xmm_crc0); Kidding aside, I thought the purpose of Rust was for safety but the keyword unsafe is sprinkled liberally throughout this library. At what point does it really stop mat…

[deleted]

Re: Zlib-rs is faster than C

#167

Post by ajross »

steveklabnik wrote: post #94
ajross wrote: post #52

Earlier quoted context omitted.

Are there examples you're thinking about? The only good ones I can think of are bits about undefined behavior semantics, which frankly are very well covered in modern C code via tools like ubsan, etc...

They're just fundamentally different languages. There's semantics that exist in all four of these quadrants: * defined in C, undefined in Rust * undefined in C, undefined in Rust * defined in Rust, undefined in C * defined in Rust, defined in C

That doesn't seem responsive. The question wasn't whether Rust and C are literally the same language ("duh", as it were), it was effectively "are there meaningful safety features provided to the unsafe zlib-rs code in question in that aren't already available in C toolchains/ecosystems?"

And there really aren't. The abbreviated/limited safety environment being exploited by this non-idiomatic Rust code seems to me to be basically isomorphic to the way you'd solve the problem in C.

Re: Zlib-rs is faster than C

#168

Post by ajross »

sedatk wrote: post #91
ajross wrote: post #46

Earlier quoted context omitted.

It doesn't exploit (and in fact deliberately evades) Rust's signature memory safety features. The impression from the headline is "Rust is as fast as C now!", but in fact the subset of the language that has been shown to be as fast as C is the subset that is basically isomorphic to C. The impression a naive reader might take is that idiomatic/safe/best-practices Rust has now closed the performance gap. But clearly th…

Rust's many memory safety features (including the borrow checker) are still enabled in unsafe Rust blocks. For more information: https://news.ycombinator.com/item?id=43382176

But again, not exploited by the code in question. This isn't using the Rust runtime heap, it's doing its own thing with raw pointers/indexing, and even seems to have its own allocator.

Re: Zlib-rs is faster than C

#169

Post by cb321 »

lern_too_spel wrote: post #111
cb321 wrote: post #40

I think this may not be a very high bar. zippy in Nim claims to be about 1.5x to 2.0x faster than zlib: https://github.com/guzba/zippy I think there are also faster zlib's around in C than the standard install one, such as https://github.com/ebiggers/libdeflate (EDIT: also mentioned elsethread https://news.ycombinator.com/item?id=43381768 by mananaysiempre) zlib itself seems pretty antiquated/outdated these days, but…

They're comparing against zlib-ng, not zlib. zlib-ng is more than twice as fast as zlib for decompression. https://github.com/zlib-ng/zlib-ng/discussions/871 libdeflate is not zlib compatible. It doesn't support streaming decompression.

Thanks (to all correctors). FWIW, that zlib-ng discussion page you link to has way more information about what machine the benchmarks were run on than TFA. It's also a safe bet that Google timed their chromium lib (which seems really close) on a much larger diversity of core architectures than these 3..4 guys have with zlib-rs. So, you know, very early days in terms of perf claims, IMO.

Also, FWIW, that zippy Nim library has essentially zero CPU-specific optimizations that I could find. Maybe one tiny one in some checksumming bit. Optimization is specialization. So, I'd guess it's probably a little slower than zlib-ng now that this is pointed out, but as @hinkley observed, portability can also be a meaningful goal/axis.

Re: Zlib-rs is faster than C

#170

Post by NobodyNada »

formerly_proven wrote: post #118
api wrote: post #45

Earlier quoted context omitted.

The idea is that you can trivially search the code base for "unsafe" and closely examine all unsafe code, and unless you are doing really low-level stuff there should not be much of it. Higher level code bases should ideally have none. It tends to be found in drivers, kernels, vector code, and low-level implementations of data structures and allocators and similar things. Not typical application code. As a general ru…

My understanding from Aria Beingessner's and some other writings is that unsafe{} rust is significantly harder to get right in "non-trivial cases" than C, because the semantics are more complex and less specified.

This is definitely true right now, but I don't think it will always be the case.

Unsafe Rust is currently extremely underspecified and underdocumented, but it's designed to be far more specifiable than C. For example: aliasing rules. When and how you're allowed to alias references in unsafe code is not at all documented and under much active discussion; whereas in C pointer aliasing rules are well defined but also completely insane (casting pointers to a different type in order to reinterpret the bytes of an object is often UB even in completely innocuous cases).

Once Rust's memory model is fully specified and written down, unsafe Rust is trying to go for something much simpler, more teachable, and with less footguns than C.

Huge props to Ralf Jung and the opsem team who are working on answering these questions & creating a formal specification: https://github.com/rust-lang/unsafe-code-guidelines/issues

Post reply on HN