Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zlib-rs is faster than C

trifectatech.org

Post reply on HN
Print view View original

51–60 of 492 posts

Re: Zlib-rs is faster than C

#51

Post by pinkmuffinere »

tdiff wrote: post #32
johnisgood wrote: post #4

"faster than C" almost always boils down to different designs, implementations, algorithms, etc. Perhaps it is faster than already-existing implementations, sure, but not "faster than C", and it is odd to make such claims.

Nevertheless Russinovich actually says something in the lines of "simple rewriting in rust made some our code 5-15% faster (without deliberate optimizations)": https://www.youtube.com/watch?v=1VgptLwP588&t=351s

I’m sure I’m missing context, and presumably there are other benefits, but 5-15% improvement is such a small step to justify rewriting codebases.

I also wonder how much of an improvement you’d get by just asking for a “simple rewrite” in the existing language. I suspect there are often performance improvements to be had with simple changes in the existing language

Re: Zlib-rs is faster than C

#52

Post by ajross »

hermanradtke wrote: post #15
ajross wrote: post #8

Earlier quoted context omitted.

It's... basically written in C. I'm no expert on zlib/deflate or related algorithms, but digging around https://github.com/trifectatechfoundation/zlib-rs/ almost every block with meaningful logic is marked unsafe. There's raw allocation management, raw slicing of arrays, etc... This code looks and smells like C, and very much not like rust. I don't know that this is a direct transcription of the C code, but if you we…

> basically written in C Unsafe Rust still has to conform to many of Rust’s rules. It is meaningfully different than C.

Are there examples you're thinking about? The only good ones I can think of are bits about undefined behavior semantics, which frankly are very well covered in modern C code via tools like ubsan, etc...

Re: Zlib-rs is faster than C

#53

Post by timschmidt »

api wrote: post #45
Aurornis wrote: post #29

Earlier quoted context omitted.

Using unsafe blocks in Rust is confusing when you first see it. The idea is that you have to opt-out of compiler safety guarantees for specific sections of code, but they’re clearly marked by the unsafe block. In good practice it’s used judiciously in a codebase where it makes sense. Those sections receive extra attention and analysis by the developers. Of course you can find sloppy codebases where people reach for u…

The idea is that you can trivially search the code base for "unsafe" and closely examine all unsafe code, and unless you are doing really low-level stuff there should not be much of it. Higher level code bases should ideally have none. It tends to be found in drivers, kernels, vector code, and low-level implementations of data structures and allocators and similar things. Not typical application code. As a general ru…

To the extent that it's even possible to write bare metal microcontroller firmware in Rust without unsafe, as the embedded hal ecosystem wraps unsafe hardware interfaces in a modular fairly universal safe API.

Re: Zlib-rs is faster than C

#54

Post by [deleted] »

johnisgood wrote: post #4

"faster than C" almost always boils down to different designs, implementations, algorithms, etc. Perhaps it is faster than already-existing implementations, sure, but not "faster than C", and it is odd to make such claims.

[deleted]

Re: Zlib-rs is faster than C

#55

Post by jrockway »

Chromium is kind of stuck with zlib because it's the algorithm that's in the standards, but if you're making your own protocol, you can do even better than this by picking a better algorithm. Zstandard is faster and compresses better. LZ4 is much faster, but not quite as small.

Some reading: https://jolynch.github.io/posts/use_fast_data_algorithms/

(As an aside, at my last job container pushes / pulls were in the development critical path for a lot of workflows. It turns out that sha256 and gzip are responsible for a lot of the time spent during container startup. Fortunately, Zstandard is allowed, and blake3 digests will be allowed soon.)

Re: Zlib-rs is faster than C

#56

Post by LegionMammal978 »

johnisgood wrote: post #33
Filligree wrote: post #25

Earlier quoted context omitted.

The usual answer is: You only need to verify the unsafe blocks, not every block. Though 'unsafe' in Rust is actually even less safe than regular C, if a bit more predictable, so there's a crossover point where you really shouldn't have bothered. The Rust compiler is indeed better than the C one, largely because of having more information and doing full-program optimisation. A `vec_foo = vec_foo.into_iter().map(...).c…

I have been told that "unsafe" affects code outside of that block, but hopefully steveklabnik may explain it better (again). > isn't going to do any bounds checks or allocate. You need to add explicit bounds check or explicitly allocate in C though. It is not there if you do not add it yourself.

> I have been told that "unsafe" affects code outside of that block, but hopefully stevelabnik may explain it better (again).

Poorly-written unsafe code can have effects extending out into safe code. But correctly-written unsafe code does not have any effects on safe code w.r.t. memory safety. So to ensure memory safety, you just have to verify the correctness of the unsafe code (and any helper functions, etc., it depends on), rather than the entire codebase.

Also, some forms of unsafe code are far less dangeous than others in practice. E.g., most of the SIMD functions are practically safe to call in every situation, but they all have 'unsafe' slapped on them due to being intrinsics.

> You need to add explicit bounds check or explicitly allocate in C though. It is not there if you do not add it yourself.

Unfortunately, you do need to allocate a new buffer in C if you change the type of the elements. The annoying side of strict aliasing is that every buffer has a single type that's set in stone for all time. (Unless you preemptively use unions for everything.)

Re: Zlib-rs is faster than C

#57

Post by qweqwe14 »

water9 wrote: post #21

[flagged]

> Add a borrow checker to C++ and put rust to bed once and for all

Ah yes, C++ is just one safety feature away from replacing Rust, surely, any moment now. The bizzare world C++ fanboys live in.

Every single person that had been writing C++ for a while and isn't a victim of Stockholm syndrome would be happy when C++ is put to bed once and for all. It's a horrible language only genuinely enjoyed by bad programmers.

Re: Zlib-rs is faster than C

#58

Post by xxs »

YZF wrote: post #18

I found out I already know Rust: unsafe { let x_tmp0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x10); xmm_crc0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x01); xmm_crc1 = _mm_xor_si128(xmm_crc1, x_tmp0); xmm_crc1 = _mm_xor_si128(xmm_crc1, xmm_crc0); Kidding aside, I thought the purpose of Rust was for safety but the keyword unsafe is sprinkled liberally throughout this library. At what point does it really stop mat…

oddly enough that's not the most optimal version of crc32, e.g. it's not an avx512 variant.

Re: Zlib-rs is faster than C

#59

Post by timschmidt »

chongli wrote: post #47
Aurornis wrote: post #29

Earlier quoted context omitted.

Using unsafe blocks in Rust is confusing when you first see it. The idea is that you have to opt-out of compiler safety guarantees for specific sections of code, but they’re clearly marked by the unsafe block. In good practice it’s used judiciously in a codebase where it makes sense. Those sections receive extra attention and analysis by the developers. Of course you can find sloppy codebases where people reach for u…

Isn't it the case that once you use unsafe even a single time, you lose all of Rust's nice guarantees? As far as I'm aware, inside the unsafe block you can do whatever you want which means all of the nice memory-safety properties of the language go away. It's like letting a wet dog (who'd just been swimming in a nearby swamp) run loose inside your hermetically sealed cleanroom.

It seems like you've got it backwards. Even unsafe rust is still more strict than C. Here's what the book has to say (https://doc.rust-lang.org/book/ch20-01-unsafe-rust.html)

"You can take five actions in unsafe Rust that you can’t in safe Rust, which we call unsafe superpowers. Those superpowers include the ability to:

    Dereference a raw pointer
    Call an unsafe function or method
    Access or modify a mutable static variable
    Implement an unsafe trait
    Access fields of a union
It’s important to understand that unsafe doesn’t turn off the borrow checker or disable any other of Rust’s safety checks: if you use a reference in unsafe code, it will still be checked. The unsafe keyword only gives you access to these five features that are then not checked by the compiler for memory safety. You’ll still get some degree of safety inside of an unsafe block.

In addition, unsafe does not mean the code inside the block is necessarily dangerous or that it will definitely have memory safety problems: the intent is that as the programmer, you’ll ensure the code inside an unsafe block will access memory in a valid way.

People are fallible, and mistakes will happen, but by requiring these five unsafe operations to be inside blocks annotated with unsafe you’ll know that any errors related to memory safety must be within an unsafe block. Keep unsafe blocks small; you’ll be thankful later when you investigate memory bugs."

Re: Zlib-rs is faster than C

#60

Post by cmrdporcupine »

timschmidt wrote: post #36
Aurornis wrote: post #29

Earlier quoted context omitted.

Using unsafe blocks in Rust is confusing when you first see it. The idea is that you have to opt-out of compiler safety guarantees for specific sections of code, but they’re clearly marked by the unsafe block. In good practice it’s used judiciously in a codebase where it makes sense. Those sections receive extra attention and analysis by the developers. Of course you can find sloppy codebases where people reach for u…

Unsafe is a very distinct code smell. Like the hydrogen sulfide added to natural gas to allow folks to smell a gas leak. If you smell it when you're not working on the gas lines, that's a signal.

Look, no. Just go read the unsafe block in question. It's just SIMD intrinsics. No memory access. No pointers. It's unsafe in name only.

No need to get all moral about it.

Post reply on HN