Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zlib-rs is faster than C

trifectatech.org

Post reply on HN
Print view View original

211–220 of 492 posts

Re: Zlib-rs is faster than C

#211

Post by dralley »

pclmulqdq wrote: post #208
dralley wrote: post #205

Earlier quoted context omitted.

>The fact that the Rust maintainers allow people to put in half-baked features before they are fully designed is the biggest cultural failing of the language, IMO. In nightly? Hard disagree. Letting people try things out in the real world is how you avoid half-baked features. Easy availability of nightly compilers with unstable features allows way more people to get involved in the pre-stabilization polishing phase o…

Yeah, we're going to have to agree to disagree on the C++ flow (really the flow for any language that has a written standard) being better. That flow is usually: 1. Big library/compiler does a thing, and people really like it 2. Other compilers and libraries copy that thing, sometimes putting their own spin on it 3. All the kinks get worked out and they write a white paper 4. Eventually the thing becomes standard Tha…

>That way, everything in the standard library is something that is fully-thought-out and feature-complete

Are C++ features really that much better thought out? Modules were "standardized" half a decade ago, but the list of problems with actually using them in practice is still pretty damn long to the point where adoption is basically non-existent.

I'm not going to pretend to be nearly as knowledgeable about C++ as Rust, but it seems like most new C++ features I hear about are a bit janky or don't actually fit that well with the rest of the language. Something that tends to happen when designing things in an ivory tower without testing them in practice.

Re: Zlib-rs is faster than C

#212

Post by cbarrick »

YZF wrote: post #18

I found out I already know Rust: unsafe { let x_tmp0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x10); xmm_crc0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x01); xmm_crc1 = _mm_xor_si128(xmm_crc1, x_tmp0); xmm_crc1 = _mm_xor_si128(xmm_crc1, xmm_crc0); Kidding aside, I thought the purpose of Rust was for safety but the keyword unsafe is sprinkled liberally throughout this library. At what point does it really stop mat…

Others have already addressed the "unsafe" smell.

I think the bigger point here is that doing SIMD in Rust is still painful.

There are efforts like portable-simd [1] to make this better, but in practice, many people are dropping down to low-level SIMD intrinsics and/or inline assembly, which are no better than their C equivalents.

[1]: https://github.com/rust-lang/portable-simd

Re: Zlib-rs is faster than C

#213

Post by lambda »

uecker wrote: post #128
lambda wrote: post #119

Earlier quoted context omitted.

So, it's true that unsafe code can depend on preconditions that need to be upheld by safe code. But using ordinary module encapsulation and private fields, you can scope the code that needs to uphold those preconditions to a particular module. So the "trusted computing base" for the unsafe code can still be scoped and limited, allowing you to reduce the amount of code you need to audit and be particularly careful abo…

And you think one can not modularize C code and encapsulate critical buffer operations in much safer APIs? One can, the problem is that a lot of legacy C code was not written this way. Also lot of newly written C code is not written this way, but the reason is often that people cut corners when they need to get things done with limited time and resources. The same you will see with Rust.

There is no distinction between safe and unsafe code in C, so it's not possible to make that same distinction that you can in Rust.

And even if you try to provide some kind of safer abstraction, you're limited by the much more primitive type system, that can't distinguish between owned types, unique borrows, and shared borrows, nor can it distinguish thread safety properties.

So you're left to convention and documentation for that kind of information, but nothing checking that you're getting it right, making it easy to make mistakes. And even if you get it right at first, a refactor could change your invariants, and without a type system enforcing them, you never know until someone comes along with a fuzzer and figures out that they can pwn you

Re: Zlib-rs is faster than C

#214

Post by pclmulqdq »

dralley wrote: post #211
pclmulqdq wrote: post #208

Earlier quoted context omitted.

Yeah, we're going to have to agree to disagree on the C++ flow (really the flow for any language that has a written standard) being better. That flow is usually: 1. Big library/compiler does a thing, and people really like it 2. Other compilers and libraries copy that thing, sometimes putting their own spin on it 3. All the kinks get worked out and they write a white paper 4. Eventually the thing becomes standard Tha…

>That way, everything in the standard library is something that is fully-thought-out and feature-complete Are C++ features really that much better thought out? Modules were "standardized" half a decade ago, but the list of problems with actually using them in practice is still pretty damn long to the point where adoption is basically non-existent. I'm not going to pretend to be nearly as knowledgeable about C++ as Ru…

They absolutely are. The reason many features are stupid and janky is because the language and its ecosystem has had almost 40 more years to collect cruft.

The fundamental problem with modules is that build systems for C++ have different abstractions and boundaries. C++ modules are like Rust async - something that just doesn't fit well with the language/system and got hammered in anyway.

The reason it seems like they come from nowhere is probably because you don't know where they come from. Most things go through boost, folly, absl, clang, or GCC (or are vendor-specific features) before going to std.

That being said, it's not just C++ that has this flow for adding features to the language. Almost every other major language that is not Rust has an authoritative specification.

Re: Zlib-rs is faster than C

#215

Post by dralley »

pclmulqdq wrote: post #214
dralley wrote: post #211

Earlier quoted context omitted.

>That way, everything in the standard library is something that is fully-thought-out and feature-complete Are C++ features really that much better thought out? Modules were "standardized" half a decade ago, but the list of problems with actually using them in practice is still pretty damn long to the point where adoption is basically non-existent. I'm not going to pretend to be nearly as knowledgeable about C++ as Ru…

They absolutely are. The reason many features are stupid and janky is because the language and its ecosystem has had almost 40 more years to collect cruft. The fundamental problem with modules is that build systems for C++ have different abstractions and boundaries. C++ modules are like Rust async - something that just doesn't fit well with the language/system and got hammered in anyway. The reason it seems like they…

What's a Rust feature that you think suffered from their process in a way that C++ would not have?

Re: Zlib-rs is faster than C

#216

Post by thrance »

kccqzy wrote: post #66
cmrdporcupine wrote: post #60

Earlier quoted context omitted.

Look, no. Just go read the unsafe block in question. It's just SIMD intrinsics. No memory access. No pointers. It's unsafe in name only. No need to get all moral about it.

By your line of reasoning, SIMD intrinsics functions should not be marked as unsafe in the first place. Then why are they marked as unsafe?

For now the caller has to ensure proper alignment of SMID lines. But in the future a safe API will be made available, once the kinks are ironed out. You can already use it in fact, by enabling a specific compiler feature [1].

[1] https://doc.rust-lang.org/std/simd/index.html

Re: Zlib-rs is faster than C

#217

Post by ComputerGuru »

ajross wrote: post #168
sedatk wrote: post #91

Earlier quoted context omitted.

Rust's many memory safety features (including the borrow checker) are still enabled in unsafe Rust blocks. For more information: https://news.ycombinator.com/item?id=43382176

But again, not exploited by the code in question . This isn't using the Rust runtime heap, it's doing its own thing with raw pointers/indexing, and even seems to have its own allocator.

That is not correct; in another comment you can see where the code takes advantage of the rust-specific &mut notation to use a fast memcpy for non-overlapping pointers.

Re: Zlib-rs is faster than C

#218

Post by beacon294 »

throwaway150 wrote: post #207
timschmidt wrote: post #36

Earlier quoted context omitted.

Unsafe is a very distinct code smell. Like the hydrogen sulfide added to natural gas to allow folks to smell a gas leak. If you smell it when you're not working on the gas lines, that's a signal.

> Like the hydrogen sulfide added to natural gas to allow folks to smell a gas leak. I am 100% sure that the smell they add to natural gas does not smell like rotten eggs.

They add mercaptan which is like 1000x the rotten egg smell of H2S.

Re: Zlib-rs is faster than C

#219

Post by anonymoushn »

thrance wrote: post #216
kccqzy wrote: post #66

Earlier quoted context omitted.

By your line of reasoning, SIMD intrinsics functions should not be marked as unsafe in the first place. Then why are they marked as unsafe?

For now the caller has to ensure proper alignment of SMID lines. But in the future a safe API will be made available, once the kinks are ironed out. You can already use it in fact, by enabling a specific compiler feature [1]. [1] https://doc.rust-lang.org/std/simd/index.html

there are no loads in the above unsafe block, in practice loadu is just as fast as load, and even if you manually use the aligned load or store, you get a crash. it's silly to say that crashes are unsafe.

Re: Zlib-rs is faster than C

#220

Post by throwaway2037 »

YZF wrote: post #18

I found out I already know Rust: unsafe { let x_tmp0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x10); xmm_crc0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x01); xmm_crc1 = _mm_xor_si128(xmm_crc1, x_tmp0); xmm_crc1 = _mm_xor_si128(xmm_crc1, xmm_crc0); Kidding aside, I thought the purpose of Rust was for safety but the keyword unsafe is sprinkled liberally throughout this library. At what point does it really stop mat…

    > Is the Rust compiler a better optimizing compiler than C compilers?
First, I assume that the main Rust compiler uses LLVM. I also assume (big leap here!) that the LLVM optimization process is language agnostic (ChatGPT agrees, whatever that is worth). As long as the language frontend can compiler to LLVM language-independent intermediate representation (IR), then all languages can equally benefit from the optimizer.
Post reply on HN