Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zlib-rs is faster than C

trifectatech.org

Post reply on HN
Print view View original

141–150 of 492 posts

Re: Zlib-rs is faster than C

#141

Post by dwattttt »

formerly_proven wrote: post #118
api wrote: post #45

Earlier quoted context omitted.

The idea is that you can trivially search the code base for "unsafe" and closely examine all unsafe code, and unless you are doing really low-level stuff there should not be much of it. Higher level code bases should ideally have none. It tends to be found in drivers, kernels, vector code, and low-level implementations of data structures and allocators and similar things. Not typical application code. As a general ru…

My understanding from Aria Beingessner's and some other writings is that unsafe{} rust is significantly harder to get right in "non-trivial cases" than C, because the semantics are more complex and less specified.

It's hard to compare. Rust has stricter requirements than C, but looser requirements don't mean easier: ever bit shifted by a variable amount? Hope you never relied on shifting "entirely" out of a variable zeroing it.

Re: Zlib-rs is faster than C

#142

Post by gf000 »

YZF wrote: post #18

I found out I already know Rust: unsafe { let x_tmp0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x10); xmm_crc0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x01); xmm_crc1 = _mm_xor_si128(xmm_crc1, x_tmp0); xmm_crc1 = _mm_xor_si128(xmm_crc1, xmm_crc0); Kidding aside, I thought the purpose of Rust was for safety but the keyword unsafe is sprinkled liberally throughout this library. At what point does it really stop mat…

Rust's borrow checker still checks within unsafe blocks, so unless you are only operating with raw pointers (and not accessing certain references as raw pointers in some small, well-defined blocks) across the whole program it will be significantly more safe than C. Especially given all the other language benefits, like a proper type system that can encode a bunch of invariants, no footguns at every line/initialization/cast, etc.

Re: Zlib-rs is faster than C

#143

Post by gf000 »

uecker wrote: post #93
timschmidt wrote: post #59

Earlier quoted context omitted.

It seems like you've got it backwards. Even unsafe rust is still more strict than C. Here's what the book has to say ( https://doc.rust-lang.org/book/ch20-01-unsafe-rust.html ) "You can take five actions in unsafe Rust that you can’t in safe Rust, which we call unsafe superpowers. Those superpowers include the ability to: Dereference a raw pointer Call an unsafe function or method Access or modify a mutable static va…

This description is still misleading. The preconditions for the correctness of an unsafe block can very much depend on the correctness of the code outside and it is easy to find Rust bugs where exactly this was the cause. This is very similar where often C out of bounds accesses are caused by some logic error elsewhere. Also an unsafe block has to maintain all the invariants the safe Rust part needs to maintain corre…

This is technically correct, but a bit pedantic.

Sure, you can technically just write your own vulnerability for your own program and inject it at an unsafe and see the whole world crumble... but the exact same is true for any form of FFI calls in any language. Is Java memory safe? Yeah, just because I can grab a random pointer and technically break anything I want won't change that.

The fact that a memory vulnerability error may either appear at no place at all OR at the couple hundred lines of code thorough the whole project is a night and day difference.

Re: Zlib-rs is faster than C

#144

Post by asveikau »

YZF wrote: post #18

I found out I already know Rust: unsafe { let x_tmp0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x10); xmm_crc0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x01); xmm_crc1 = _mm_xor_si128(xmm_crc1, x_tmp0); xmm_crc1 = _mm_xor_si128(xmm_crc1, xmm_crc0); Kidding aside, I thought the purpose of Rust was for safety but the keyword unsafe is sprinkled liberally throughout this library. At what point does it really stop mat…

> At what point does it really stop mattering if this is C or Rust?

If I read TFA correctly, they came up with a library that is API compatible with the C one, but they've measured to be faster.

At that point I think in addition to safety benefits in other parts of the library (apart from unsafe micro optimizations as quoted), what they're leveraging is better compiler technology. Intuitively, I start to assume that the rust compiler can perhaps get away with more optimizations that might not be safe to assume in C.

Re: Zlib-rs is faster than C

#145

Post by gf000 »

uecker wrote: post #128
lambda wrote: post #119

Earlier quoted context omitted.

So, it's true that unsafe code can depend on preconditions that need to be upheld by safe code. But using ordinary module encapsulation and private fields, you can scope the code that needs to uphold those preconditions to a particular module. So the "trusted computing base" for the unsafe code can still be scoped and limited, allowing you to reduce the amount of code you need to audit and be particularly careful abo…

And you think one can not modularize C code and encapsulate critical buffer operations in much safer APIs? One can, the problem is that a lot of legacy C code was not written this way. Also lot of newly written C code is not written this way, but the reason is often that people cut corners when they need to get things done with limited time and resources. The same you will see with Rust.

Even innocent looking C code can be chock-full of UBs that can invalidate your "local reasoning" capabilities. So, not even close.

Re: Zlib-rs is faster than C

#146

Post by acdha »

gf000 wrote: post #142
YZF wrote: post #18

I found out I already know Rust: unsafe { let x_tmp0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x10); xmm_crc0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x01); xmm_crc1 = _mm_xor_si128(xmm_crc1, x_tmp0); xmm_crc1 = _mm_xor_si128(xmm_crc1, xmm_crc0); Kidding aside, I thought the purpose of Rust was for safety but the keyword unsafe is sprinkled liberally throughout this library. At what point does it really stop mat…

Rust's borrow checker still checks within unsafe blocks, so unless you are only operating with raw pointers (and not accessing certain references as raw pointers in some small, well-defined blocks) across the whole program it will be significantly more safe than C. Especially given all the other language benefits, like a proper type system that can encode a bunch of invariants, no footguns at every line/initializatio…

Yes. I think it’s easy to underestimate how much the richer language and library ecosystem chip away at the attack surface area. So many past vulnerabilities have been in code which isn’t dealing with low-level interfaces or weird performance optimizations and wouldn’t need to use unsafe. There’ve been so many vulnerabilities in crypto code which weren’t the encryption or hashing algorithms but things like x509/ASN parsing, logging, or the kind of option/error handling logic a Rust programmer would use the type system to validate.

Re: Zlib-rs is faster than C

#147

Post by onnimonni »

timschmidt wrote: post #59
chongli wrote: post #47

Earlier quoted context omitted.

Isn't it the case that once you use unsafe even a single time, you lose all of Rust's nice guarantees? As far as I'm aware, inside the unsafe block you can do whatever you want which means all of the nice memory-safety properties of the language go away. It's like letting a wet dog (who'd just been swimming in a nearby swamp) run loose inside your hermetically sealed cleanroom.

It seems like you've got it backwards. Even unsafe rust is still more strict than C. Here's what the book has to say ( https://doc.rust-lang.org/book/ch20-01-unsafe-rust.html ) "You can take five actions in unsafe Rust that you can’t in safe Rust, which we call unsafe superpowers. Those superpowers include the ability to: Dereference a raw pointer Call an unsafe function or method Access or modify a mutable static va…

Would someone with more experience be able to explain to me why can't these operations be "safe"? What is blocking rust from producing the same machine code in a "safe" way?

Re: Zlib-rs is faster than C

#148

Post by gf000 »

uecker wrote: post #126
dwattttt wrote: post #117

Earlier quoted context omitted.

It's true, but I think it's only fair if you hold Rust to this analysis, other languages should too; the scrutiny you're implying you need in an unsafe Rust block needs to be applied to all C code, because all C code could depend on code anywhere else for its safety characteristics. In practice (in both languages) you check what the actual unsafe code does (or "all" code in C's case), note code that depends on extern…

What is true is that there are more operations in C which can cause undefined behavior and those are more densely distributed over the C code, making it harder to screen for undefined behavior. This is true and Rust certainly has an advantage, but it not nearly as big of an advantage as the "Rust is safe" (please do not look at all the unsafe blocks we need to make it also fast!) and "all C is unsafe" story wants you…

Rust is plenty fast, in fact there are countless examples of safe rust that will trivially beat out C in performance due to no aliasing, enabling better vectorization among others. Let alone being simply a more expressive language and allowing writing better optimizations (e.g. small strings, vs the absolutely laughable c-strings that perform terribly, but also you can actually get away with sharing more stuff in memory vs doing defensive copies everywhere because it is safe to do so, etc)

And there is not many things we have statistics on in CS, but memory vulnerabilities being absolutely everywhere in unsafe languages, and Rust cleaning up the absolute majority of them even when only the new parts are written in Rust are some of the few we do know, based on actual, real life projects at Google/Microsoft among others.

A memory safe low-level language is as novel as it gets. Rust is absolutely not just hype, it actually delivers and you might want to get on with the times.

Re: Zlib-rs is faster than C

#149

Post by jeffbee »

jrockway wrote: post #55

Chromium is kind of stuck with zlib because it's the algorithm that's in the standards, but if you're making your own protocol, you can do even better than this by picking a better algorithm. Zstandard is faster and compresses better. LZ4 is much faster, but not quite as small. Some reading: https://jolynch.github.io/posts/use_fast_data_algorithms/ (As an aside, at my last job container pushes / pulls were in the dev…

Yeah I just discovered this a few days ago. All the docker-era tools default to gzip but if using, say, bazel rules_oci instead of rules_docker you can turn on zstd for large speedups in push/pull time.
Post reply on HN