Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust Performance Pitfalls

llogiq.github.io

Post reply on HN
Print view View original

61–70 of 112 posts

Re: Rust Performance Pitfalls

#61

Post by johncolanduoni »

corn_dog wrote: post #56
steveklabnik wrote: post #41

Earlier quoted context omitted.

unsafe litearally means "you, compiler, cannot verify that this does not violate memory safety, but I have done so, so please trust me." Violating memory safety in unsafe code is UB.

Is it the case that "unsafe" tells the compiler to not perform its memory safety checks on that section of code, presumably because it's not possible? (What would happen if you put only code that could be verified by the compiler in an unsafe block?) If so couldn't you also think of it as a message for the next programmer who looks at the code? This section has not/cannot be verified by the compiler, approach with ca…

No, if you call a safe function that does memory checks (like accessing a Vec by index) in an unsafe block the compiler still emits checks. You need to explicitly call the unsafe versions of those operations to remove the check.

Re: Rust Performance Pitfalls

#62

Post by steveklabnik »

corn_dog wrote: post #56
steveklabnik wrote: post #41

Earlier quoted context omitted.

unsafe litearally means "you, compiler, cannot verify that this does not violate memory safety, but I have done so, so please trust me." Violating memory safety in unsafe code is UB.

Is it the case that "unsafe" tells the compiler to not perform its memory safety checks on that section of code, presumably because it's not possible? (What would happen if you put only code that could be verified by the compiler in an unsafe block?) If so couldn't you also think of it as a message for the next programmer who looks at the code? This section has not/cannot be verified by the compiler, approach with ca…

> Is it the case that "unsafe" tells the compiler to not perform its memory safety checks on that section of code,

So, unsafe Rust is a superset of safe Rust. Adding `unsafe` around some code lets you do four things:

* Dereferencing a raw pointer

* Calling an unsafe function or method

* Accessing or modifying a mutable static variable

* Implementing an unsafe trait

That's it. Nothing else changes, you get these additional abilities. This is very important, conceptually. Tons of other checks are still on, etc.

With that in mind,

> (What would happen if you put only code that could be verified by the compiler in an unsafe block?)

It would function identically.

Re: Rust Performance Pitfalls

#63

Post by Sharlin »

Asooka wrote: post #36
firethief wrote: post #17

Earlier quoted context omitted.

That's the kind of vagueness that should sound like a warning to anyone considering that approach.

To me "your code breaks in surprising ways" means something more like "the user sees gibberish/wrong results" and not "your program segfaults". Segfaulting is not actually a surprising result to me - I have seen it over and over on out of bounds access.

A buffer overflow doesn't necessarily cause a segfault - that's the problem! A guaranteed segfault would be a completely valid and safe way to handle a buffer overflow. But segfaults only happen when your program tries to access memory it does not own. It is improbable that an overflowing buffer is straight at a page boundary, doubly so if the buffer is allocated on the stack. Instead, you get gibberish output, an invalid and unexpected program state, no effect at all (except once in a blue moon) or in the worst case a code injection vulnerability. That is, undefined behavior.

Re: Rust Performance Pitfalls

#64

Post by kibwen »

kbenson wrote: post #55
kibwen wrote: post #50

Earlier quoted context omitted.

> No, I don't believe it does. This is mistaken, and an impression that we continually strive very hard to counter. The `unsafe` keyword is to be used in the process of writing (and therefore consuming) APIs if and only if those APIs have external unenforced invariants which, if broken, could cause memory unsafety. The reason that we strive to reinforce this so fervently is precisely because we want people to see the…

> > No, I don't believe it does. > This is mistaken, and an impression that we continually strive very hard to counter. > In particular, this means that `unsafe` is not to be used for operations that may be dangerous but that have nothing to do with memory safety. That's not what I'm talking about. I think you've misinterpreted my point. I'm talking about unsafe for memory access, but in ways that are provably (or it…

We must be talking past each other, because I'm still not certain what you're trying to say.

If you're trying to say "there exists Rust code that contains `unsafe` blocks that is memory-safe", then this is obviously (hopefully!) correct, because 100% of the time we hope that our `unsafe` blocks are correctly implemented. In the same sense that "valid" C code does not contain undefined behavior, "valid" Rust code doesn't either; therefore the correctness of `unsafe` blocks in Rust must be tautologically (if uselessly) guaranteed. But that's not a very useful statement. Bugs happen.

Conversely, if you're trying to say "there exists code that can only be written in `unsafe` blocks but that can never cause memory unsafety despite any modification to the surrounding code", then I'd say this is trivially refutable; I can modify any unsafe block to exhibit memory unsafety.

Finally, if you're just saying "static analysis must, by its very definition, reject some correct programs in order to guarantee that the programs it does accept are correct, and Rust's static analyses are no different" then, of course, this is true (again, by the nature of static analysis), but again this is not an especially useful statement, especially since this isn't what anyone here is disputing. Your original comment was made in reply to this statement by rabidferret: "`unsafe` literally means "this might violate memory safety". This is a true statement, both in a social context (see my original comment) and in an implementation context (see the second example from this comment).

The bottom line is, if you see an `unsafe` block, assume memory safety is at risk unless you're absolutely sure the author knows what they're doing.

Re: Rust Performance Pitfalls

#65

Post by zigzigzag »

kibwen wrote: post #51
zigzigzag wrote: post #48

Earlier quoted context omitted.

That seems like a really unfortunate design decision. I used to think that Java's use of UTF-16 for strings was just a problematic legacy thing, but compared to this it seems quite good. Strings are pretty high performance and there are no complex calculations to do indexing or bounds checks. And in Java 9 the JVM can switch between UTF-16 or Latin1 encodings on the fly, which both uses less RAM and speeds things up…

What seems like an unfortunate design decision? I'm unclear what concerns you're seeing that would suggest UTF-16 as preferable to UTF-8, either in terms of performance or memory safety.

To not only use UTF-8 as the internal string encoding but practically mandate it, if you want to remain safe.

UTF-8 is a fine transport format, but for raw runtime performance it's obviously going to be an issue if you ever need to iterate over characters, do substring matches, things like that because you can't do constant time "next char" or indexing.

UTF-16 doesn't let you do that either in the presence of combining characters, but they're pretty rare and for many operations it doesn't really matter.

Re: Rust Performance Pitfalls

#66

Post by vvanders »

smitherfield wrote: post #54

>Sometimes, when changing an enum, we want to keep parts of the old value. Use mem::replace to avoid needless clones. use std::mem; enum MyEnum { A { name: String, x: u8 }, B { name: String } } fn a_to_b(e: &mut MyEnum) { // we mutably borrow `e` here. This precludes us from changing it directly // as in `*e = ...`, because the borrow checker won't allow it. Therefore // the assignment to `e` must be outside the `if…

> Similarly, the functional abstractions almost feel like a step backwards from the venerable C++ , abstraction and composition-wise. Very nitty-gritty, low-level implementation details leak out like a sieve — you can't have your maps or folds without a generous sprinking of `as_slice()`, `unwrap()`, `iter()` / `iter_mut()` / `into_iter()` and `collect()` everywhere

That's not been my experience at all. In fact I've found to not only be extremely limiting but also being painfully verbose due to the need of begin/end pairs everywhere.

Re: Rust Performance Pitfalls

#67

Post by kbenson »

kibwen wrote: post #64
kbenson wrote: post #55

Earlier quoted context omitted.

> > No, I don't believe it does. > This is mistaken, and an impression that we continually strive very hard to counter. > In particular, this means that `unsafe` is not to be used for operations that may be dangerous but that have nothing to do with memory safety. That's not what I'm talking about. I think you've misinterpreted my point. I'm talking about unsafe for memory access, but in ways that are provably (or it…

We must be talking past each other, because I'm still not certain what you're trying to say. If you're trying to say "there exists Rust code that contains `unsafe` blocks that is memory-safe", then this is obviously (hopefully!) correct, because 100% of the time we hope that our `unsafe` blocks are correctly implemented. In the same sense that "valid" C code does not contain undefined behavior, "valid" Rust code does…

> If you're trying to say "there exists Rust code that contains `unsafe` blocks that is memory-safe", then this is obviously (hopefully!) correct, because 100% of the time we hope that our `unsafe` blocks are correctly implemented.

Yes, and additionally there are some algorithms that are safe, but to implement require unsafe blocks. I think it's obvious that there are patterns of memory access that are safe, possibly provably so, but not by the compiler at this time.

Since a recommendation for someone to use unsafe could be either a general recommendation with a very broad caveat that quite a bit of additional care and thought, or a fairly benign recommendation to "do X, Y, and Z in this order, and while it requires unsafe, it's no less safe than when we do the same in C/C++", I think it's important to distinguish those, lest someone mistake the former for the latter.

What that boils down to in practice is that stating something requires unsafe is not sufficient by itself as a warning to denote the level of care someone should take. Different people will interpret that statement differently at different times on different topics. There is no need to leave that ambiguity standing when it is easy to clarify. That's all I was trying to express, in response to '"requires unsafe" already implies everything in your comment.'

Re: Rust Performance Pitfalls

#68

Post by smitherfield »

pcwalton wrote: post #60
smitherfield wrote: post #54

>Sometimes, when changing an enum, we want to keep parts of the old value. Use mem::replace to avoid needless clones. use std::mem; enum MyEnum { A { name: String, x: u8 }, B { name: String } } fn a_to_b(e: &mut MyEnum) { // we mutably borrow `e` here. This precludes us from changing it directly // as in `*e = ...`, because the borrow checker won't allow it. Therefore // the assignment to `e` must be outside the `if…

One problem with what you propose (as I understand it) is that it's not safe to have uninitialized data in the presence of panics. A panic can cause control flow to unwind and destructors to be invoked, and if you have partially uninitialized data you have a recipe for use after free. In C++ you just get use-after-free hazards everywhere (which is one of the many reasons why exception safety in C++ is extremely hard)…

Ah, I wasn't aware that Rust reorders structure members. Although I still feel it should be possible, whether the compiler just needs to change the tag or has to shift data around.

What I was proposing was that common subsequences of enum fields be treated by the compiler as synonyms, but accesses to uninitialized data would still be illegal. One way this might be implemented is by, when necessary, creating additional tags behind the scenes, e.g. MyEnum::__AB__ meaning A was initialized but B is the active member, so only accesses to their common subsequence are legal, MyEnum::__BA__ meaning the reverse, and so forth. Or it could be restricted to fields that contain no members outside their common subsequence with nontrivial destructors. Or the compiler could null out any uninitialized references to such.

>That said, it sounds like what you're looking for is one of the various inheritance proposals, which have safe access to common enum fields as one of the main guarantees. This would make this pattern much more ergonomic.

Interesting. Link?

Re: Rust Performance Pitfalls

#69

Post by kccqzy »

zigzigzag wrote: post #65
kibwen wrote: post #51

Earlier quoted context omitted.

What seems like an unfortunate design decision? I'm unclear what concerns you're seeing that would suggest UTF-16 as preferable to UTF-8, either in terms of performance or memory safety.

To not only use UTF-8 as the internal string encoding but practically mandate it, if you want to remain safe. UTF-8 is a fine transport format, but for raw runtime performance it's obviously going to be an issue if you ever need to iterate over characters, do substring matches, things like that because you can't do constant time "next char" or indexing. UTF-16 doesn't let you do that either in the presence of combini…

This has been repeated so many times but UTF-16 does not allow constant time indexing either. Combining characters are one case and they are not rare at all. What about surrogates? What about grapheme clusters that are a complicated sequence of emoji and emoji modifiers with ZWJ?

A better suggestion is to rethink why you need those operations in the first place.

Re: Rust Performance Pitfalls

#70

Post by smitherfield »

vvanders wrote: post #66
smitherfield wrote: post #54

>Sometimes, when changing an enum, we want to keep parts of the old value. Use mem::replace to avoid needless clones. use std::mem; enum MyEnum { A { name: String, x: u8 }, B { name: String } } fn a_to_b(e: &mut MyEnum) { // we mutably borrow `e` here. This precludes us from changing it directly // as in `*e = ...`, because the borrow checker won't allow it. Therefore // the assignment to `e` must be outside the `if…

> Similarly, the functional abstractions almost feel like a step backwards from the venerable C++ , abstraction and composition-wise. Very nitty-gritty, low-level implementation details leak out like a sieve — you can't have your maps or folds without a generous sprinking of `as_slice()`, `unwrap()`, `iter()` / `iter_mut()` / `into_iter()` and `collect()` everywhere That's not been my experience at all. In fact I've…

Slight hyperbole there; it's more that I feel the contrast between the two could be much greater.
Post reply on HN