Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zlib-rs is faster than C

trifectatech.org

Post reply on HN
Print view View original

111–120 of 492 posts

Re: Zlib-rs is faster than C

#111

Post by lern_too_spel »

cb321 wrote: post #40

I think this may not be a very high bar. zippy in Nim claims to be about 1.5x to 2.0x faster than zlib: https://github.com/guzba/zippy I think there are also faster zlib's around in C than the standard install one, such as https://github.com/ebiggers/libdeflate (EDIT: also mentioned elsethread https://news.ycombinator.com/item?id=43381768 by mananaysiempre) zlib itself seems pretty antiquated/outdated these days, but…

They're comparing against zlib-ng, not zlib. zlib-ng is more than twice as fast as zlib for decompression. https://github.com/zlib-ng/zlib-ng/discussions/871

libdeflate is not zlib compatible. It doesn't support streaming decompression.

Re: Zlib-rs is faster than C

#112

Post by mastax »

cb321 wrote: post #40

I think this may not be a very high bar. zippy in Nim claims to be about 1.5x to 2.0x faster than zlib: https://github.com/guzba/zippy I think there are also faster zlib's around in C than the standard install one, such as https://github.com/ebiggers/libdeflate (EDIT: also mentioned elsethread https://news.ycombinator.com/item?id=43381768 by mananaysiempre) zlib itself seems pretty antiquated/outdated these days, but…

The benchmarks in the parent post are comparing to zlib-ng, which is substantially faster than zlib. The zippy claims are against "zlib found on a fresh Linux install" which at least for Debian is classic zlib.

Re: Zlib-rs is faster than C

#113

Post by iknowstuff »

uecker wrote: post #93
timschmidt wrote: post #59

Earlier quoted context omitted.

It seems like you've got it backwards. Even unsafe rust is still more strict than C. Here's what the book has to say ( https://doc.rust-lang.org/book/ch20-01-unsafe-rust.html ) "You can take five actions in unsafe Rust that you can’t in safe Rust, which we call unsafe superpowers. Those superpowers include the ability to: Dereference a raw pointer Call an unsafe function or method Access or modify a mutable static va…

This description is still misleading. The preconditions for the correctness of an unsafe block can very much depend on the correctness of the code outside and it is easy to find Rust bugs where exactly this was the cause. This is very similar where often C out of bounds accesses are caused by some logic error elsewhere. Also an unsafe block has to maintain all the invariants the safe Rust part needs to maintain corre…

No. Correctness of code outside unsafe depends on correctness inside those blocks, not the other way around

Re: Zlib-rs is faster than C

#114

Post by JoshTriplett »

cb321 wrote: post #40

I think this may not be a very high bar. zippy in Nim claims to be about 1.5x to 2.0x faster than zlib: https://github.com/guzba/zippy I think there are also faster zlib's around in C than the standard install one, such as https://github.com/ebiggers/libdeflate (EDIT: also mentioned elsethread https://news.ycombinator.com/item?id=43381768 by mananaysiempre) zlib itself seems pretty antiquated/outdated these days, but…

The bar here is not zlib, it's zlib-ng, which aims primarily for performance.

libdeflate is an impressive library, but it doesn't help if you need to stream data rather than having it all in memory at once.

Re: Zlib-rs is faster than C

#115

Post by tonyhart7 »

CooCooCaCha wrote: post #82
chongli wrote: post #47

Earlier quoted context omitted.

Isn't it the case that once you use unsafe even a single time, you lose all of Rust's nice guarantees? As far as I'm aware, inside the unsafe block you can do whatever you want which means all of the nice memory-safety properties of the language go away. It's like letting a wet dog (who'd just been swimming in a nearby swamp) run loose inside your hermetically sealed cleanroom.

I wouldn’t go that far. Bevy for example, uses unsafe internally but is VERY strict about it, and every use of unsafe requires a comment explaining why the code is safe. In other words, unsafe works if you use it carefully and keep it contained.

right, the point is raising awareness and assumption its not 100 and 0 problem

Re: Zlib-rs is faster than C

#116

Post by burntsushi »

nindalf wrote: post #98
kahlonel wrote: post #7

You mean the implementation is faster than the one in C. Because nothing is “faster than C”.

Why can’t something be faster than C? If a language is able to convey more information to a backend like LLVM, the backend could use that to produce more optimised code than what it could do for C. For example, if the language is able to say, for any two pointers, the two pointers will not overlap - that would enable the backend to optimise further. In C this requires an explicit restrict keyword. In Rust, it’s the d…

Narrow correction on two points:

First, I would say that "ripgrep is generally faster than GNU grep" is a true statement. But sometimes GNU grep is faster than ripgrep and in many cases, performance is comparable or only a "little" slower than ripgrep.

Secondly, VS Code using ripgrep because of its speed is only one piece of the picture. Licensing was also a major consideration. There is an issue about this where they originally considered ripgrep (and ag if I recall correctly), but I'm on mobile so I don't have the link handy.

Re: Zlib-rs is faster than C

#117

Post by dwattttt »

uecker wrote: post #93
timschmidt wrote: post #59

Earlier quoted context omitted.

It seems like you've got it backwards. Even unsafe rust is still more strict than C. Here's what the book has to say ( https://doc.rust-lang.org/book/ch20-01-unsafe-rust.html ) "You can take five actions in unsafe Rust that you can’t in safe Rust, which we call unsafe superpowers. Those superpowers include the ability to: Dereference a raw pointer Call an unsafe function or method Access or modify a mutable static va…

This description is still misleading. The preconditions for the correctness of an unsafe block can very much depend on the correctness of the code outside and it is easy to find Rust bugs where exactly this was the cause. This is very similar where often C out of bounds accesses are caused by some logic error elsewhere. Also an unsafe block has to maintain all the invariants the safe Rust part needs to maintain corre…

It's true, but I think it's only fair if you hold Rust to this analysis, other languages should too; the scrutiny you're implying you need in an unsafe Rust block needs to be applied to all C code, because all C code could depend on code anywhere else for its safety characteristics.

In practice (in both languages) you check what the actual unsafe code does (or "all" code in C's case), note code that depends on external actors for safety (it's not all C code, nor is it all unsafe Rust blocks), and check their callers (and callers callers, etc).

Re: Zlib-rs is faster than C

#118

Post by formerly_proven »

api wrote: post #45
Aurornis wrote: post #29

Earlier quoted context omitted.

Using unsafe blocks in Rust is confusing when you first see it. The idea is that you have to opt-out of compiler safety guarantees for specific sections of code, but they’re clearly marked by the unsafe block. In good practice it’s used judiciously in a codebase where it makes sense. Those sections receive extra attention and analysis by the developers. Of course you can find sloppy codebases where people reach for u…

The idea is that you can trivially search the code base for "unsafe" and closely examine all unsafe code, and unless you are doing really low-level stuff there should not be much of it. Higher level code bases should ideally have none. It tends to be found in drivers, kernels, vector code, and low-level implementations of data structures and allocators and similar things. Not typical application code. As a general ru…

My understanding from Aria Beingessner's and some other writings is that unsafe{} rust is significantly harder to get right in "non-trivial cases" than C, because the semantics are more complex and less specified.

Re: Zlib-rs is faster than C

#119

Post by lambda »

uecker wrote: post #93
timschmidt wrote: post #59

Earlier quoted context omitted.

It seems like you've got it backwards. Even unsafe rust is still more strict than C. Here's what the book has to say ( https://doc.rust-lang.org/book/ch20-01-unsafe-rust.html ) "You can take five actions in unsafe Rust that you can’t in safe Rust, which we call unsafe superpowers. Those superpowers include the ability to: Dereference a raw pointer Call an unsafe function or method Access or modify a mutable static va…

This description is still misleading. The preconditions for the correctness of an unsafe block can very much depend on the correctness of the code outside and it is easy to find Rust bugs where exactly this was the cause. This is very similar where often C out of bounds accesses are caused by some logic error elsewhere. Also an unsafe block has to maintain all the invariants the safe Rust part needs to maintain corre…

So, it's true that unsafe code can depend on preconditions that need to be upheld by safe code.

But using ordinary module encapsulation and private fields, you can scope the code that needs to uphold those preconditions to a particular module.

So the "trusted computing base" for the unsafe code can still be scoped and limited, allowing you to reduce the amount of code you need to audit and be particularly careful about for upholding safety guarantees.

Basically, when writing unsafe code, the actual unsafe operations are scoped to only the unsafe blocks, and they have preconditions that you need to scope to a particular module boundary to ensure that there's a limited amount of code that needs to be audited to ensure it upholds all of the safety invariants.

Ralf Jung has written a number of good papers and blog posts on this topic.

Re: Zlib-rs is faster than C

#120

Post by timschmidt »

steveklabnik wrote: post #106
timschmidt wrote: post #104

Earlier quoted context omitted.

The compiler won't allow you to compile such code without the unsafe. The unsafe is *you* promising the compiler that *you* have checked to ensure that the address will always be legal. So that the compiler will allow you to compile the code.

Right, I'm saying "allow" has two different connotations, and only one of them, the one that you're talking about, applies.

I gotcha. I misread and misunderstood. Yes, we agree.
Post reply on HN