Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zlib-rs is faster than C

trifectatech.org

Post reply on HN
Print view View original

271–280 of 492 posts

Re: Zlib-rs is faster than C

#271

Post by k1t »

branko_d wrote: post #266
timschmidt wrote: post #36

Earlier quoted context omitted.

Unsafe is a very distinct code smell. Like the hydrogen sulfide added to natural gas to allow folks to smell a gas leak. If you smell it when you're not working on the gas lines, that's a signal.

Hydrogen sulfide is highly toxic (it's comparable to carbon monoxide). I doubt anyone in their right mind would put it intentionally in a place where it could leak around humans. But it can occur naturally in natural gas.

I assume GP was referring to mercaptan, or similar. i.e. Something with a distinctive bad smell.

https://en.m.wikipedia.org/wiki/Methanethiol

Re: Zlib-rs is faster than C

#272

Post by maccard »

pinkmuffinere wrote: post #51
tdiff wrote: post #32

Earlier quoted context omitted.

Nevertheless Russinovich actually says something in the lines of "simple rewriting in rust made some our code 5-15% faster (without deliberate optimizations)": https://www.youtube.com/watch?v=1VgptLwP588&t=351s

I’m sure I’m missing context, and presumably there are other benefits, but 5-15% improvement is such a small step to justify rewriting codebases. I also wonder how much of an improvement you’d get by just asking for a “simple rewrite” in the existing language. I suspect there are often performance improvements to be had with simple changes in the existing language

There are hopefully very few things that can be done to low level building blocks. A 15% improvement is absolutely worth it for a library as widely used as a compression library.

Re: Zlib-rs is faster than C

#273

Post by johnisgood »

anon-3988 wrote: post #171
layer8 wrote: post #81

Earlier quoted context omitted.

C has the restrict type qualifier to express non-aliasing, hence it shouldn’t be a fundamental impediment.

> fundamental impediment This is an interesting word. I wonder why no one has written high performance library code in assembly yet at this point?

Well, most of them is written in C/C++ and Fortran, is it not the case?

Re: Zlib-rs is faster than C

#274

Post by exDM69 »

kccqzy wrote: post #66
cmrdporcupine wrote: post #60

Earlier quoted context omitted.

Look, no. Just go read the unsafe block in question. It's just SIMD intrinsics. No memory access. No pointers. It's unsafe in name only. No need to get all moral about it.

By your line of reasoning, SIMD intrinsics functions should not be marked as unsafe in the first place. Then why are they marked as unsafe?

They are marked as unsafe because there are hundreds and hundreds of intrinsics, some of which do memory access, some have side effects and others are arithmetic only. Someone would have to individually review them and explicitly mark the safe ones.

There was a bug open about it and the rationale was that no one with the expertise (some of these are quite arcane) was stepping up to do it. (edit: other comments in this thread suggest that this effort is now underway and first changes were committed a few weeks ago)

You can do safe SIMD using std::simd but it is nightly only at this point.

Re: Zlib-rs is faster than C

#275

Post by PhilipRoman »

techjamie wrote: post #44
jdefr89 wrote: post #34

Earlier quoted context omitted.

Not to mention they link to libc.. All rust code does last I checked…

There is an option to not link to it for instances like OS writing and embedded. Writing everything in pure Rust without libc is entirely possible, even if an effort in losing sanity when you're reimplementing every syscall you need from scratch. But even then, your code is calling out to kernel functions which are probably written in C or assembly, and therefore "dangerous." Rust code safety is overhyped frequently,…

Ironically using C without libc turns out to be easier (except for portability of course). The kernel ABI is much more sane than . The only useful parts of libc are DNS resolution and text formatting, both of which it does rather poorly.

Re: Zlib-rs is faster than C

#276

Post by PhilipRoman »

hyperbrainer wrote: post #269
pdimitar wrote: post #191

Earlier quoted context omitted.

I am hopping on Rust threads on HN very regularly and I have to tell you my anecdotal experience. Which is: people complaining about Rust zealots are much more than actual Rust zealots. Thinking of it, I haven't seen a proper Rust zealot on HN for at least a year at this point. So I don't know, maybe do less cheap digs. Tearing down straw men is pretty boring to watch.

It is not limited to rust zealots or whatever. I have seen this phenomenon of people criticising a certain category for doing something in the comment section on reddit/youtube, but never actually seen the people doing that thing, even browsing the most heavily downvoted comments. Some weird group mentality victimisation.

I think in those cases usually the uploader has deleted said comments.

Re: Zlib-rs is faster than C

#277

Post by amaranth »

jeroenhd wrote: post #165
jrockway wrote: post #55

Chromium is kind of stuck with zlib because it's the algorithm that's in the standards, but if you're making your own protocol, you can do even better than this by picking a better algorithm. Zstandard is faster and compresses better. LZ4 is much faster, but not quite as small. Some reading: https://jolynch.github.io/posts/use_fast_data_algorithms/ (As an aside, at my last job container pushes / pulls were in the dev…

`Content-Encoding: zstd` was added to Chromium a while ago: https://chromestatus.com/feature/6186023867908096 You can still use deflate for compression, but Brotli and Zstd have been available in all modern browsers for quite some time.

Safari doesn't support zstd, that means if you want to use it you have to support multiple formats.

Re: Zlib-rs is faster than C

#278

Post by sunshowers »

rat87 wrote: post #252
sunshowers wrote: post #122

Earlier quoted context omitted.

What language is the JVM written in? All safe code in existence running on von Neumann architectures is built on a foundation of unsafe code. The goal of all memory-safe languages is to provide safe abstractions on top of an unsafe core.

I don't think what something was written in should count. Baring bugs it should still be memory safe. But I believe JVM has ffi and as soon as you use ffi you risk messing up that memory safety.

Does it help to think of "safe Rust" as a language that's written in "unsafe Rust"? That's basically what it is.

Re: Zlib-rs is faster than C

#279

Post by TheDong »

no_wizard wrote: post #236
TheDong wrote: post #210

Earlier quoted context omitted.

And while we're in the hypothetical extreme world somewhat separated from reality, a series of solar flares could flip a memory bit and all the error-correction bits in my ECC ram at once to change a pointer in memory, causing my safe rust to do an out of bounds write. Until we design perfectly correct computer hardware, processors, and a sun which doesn't produce solar radiation, we can't rely on totally uniform cor…

I’m simply positing how do we know the safety guarantees hold, not a hypothetical extreme. Not really sure where the extreme comes in. If you take Rust at face value, than this to me seems like an obvious question to ask

Sorry, it's just that I have an allergic reaction to what sounds like people trying to make debate-bro arguments.

Like, when I say "use signal, it's secure", someone could respond "Ahh, but technically you can't prove the absence of bugs, signal could have serious bugs, so it's not secure, you fool", but like everyone reading this already knew "it's secure" means "based on current evidence and my opinion it seems likely to be more secure than alternatives", and it got shortened. Interpreting things as absolutes that are true or false is pointless debate-bro junk which lets you create strawmen out of normal human speech.

When someone says "1+1 = 2", and a debate-bro responds "ahh but in base-2 it's 10 you fool", it's just useless internet noise. Sure, it's correct, but it's irrelevant, everyone already knows it, the original comment didn't mean otherwise.

Responding to "safe Rust should never cause out-of-bounds access, use-after-free" with "ahh but we can't prove the compiler is safe, so rust isn't safe is it??" is a similarly sorta response. Everyone already knows it. It's self-evident. It adds nothing. It sounds like debate-bro "I want to argue with you so I'm saying something that's true, but we both already know and doesn't actually matter".

I think that allergic response came out, apologies if it was misguided in this case and you're not being a debate-bro.

Re: Zlib-rs is faster than C

#280

Post by pjmlp »

Aurornis wrote: post #29
YZF wrote: post #18

I found out I already know Rust: unsafe { let x_tmp0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x10); xmm_crc0 = _mm_clmulepi64_si128(xmm_crc0, crc_fold, 0x01); xmm_crc1 = _mm_xor_si128(xmm_crc1, x_tmp0); xmm_crc1 = _mm_xor_si128(xmm_crc1, xmm_crc0); Kidding aside, I thought the purpose of Rust was for safety but the keyword unsafe is sprinkled liberally throughout this library. At what point does it really stop mat…

Using unsafe blocks in Rust is confusing when you first see it. The idea is that you have to opt-out of compiler safety guarantees for specific sections of code, but they’re clearly marked by the unsafe block. In good practice it’s used judiciously in a codebase where it makes sense. Those sections receive extra attention and analysis by the developers. Of course you can find sloppy codebases where people reach for u…

It is also an idea that traces back to the 1960's system languages, that apparently was unknown at Bell Labs.
Post reply on HN