> To get rid of the checks, we can either use bytes directly (usually via Vec / &[u8]) or, if we are absolutely sure the input will be valid UTF-8, use str::from_utf8_unchecked(_) (note that this will require unsafe and break your code in surprising ways should the input not be valid UTF-8). I believe this needs a stronger warning. Functions that operate on strings are allowed to assume that their input is valid UTF-…
That seems like a really unfortunate design decision. I used to think that Java's use of UTF-16 for strings was just a problematic legacy thing, but compared to this it seems quite good. Strings are pretty high performance and there are no complex calculations to do indexing or bounds checks. And in Java 9 the JVM can switch between UTF-16 or Latin1 encodings on the fly, which both uses less RAM and speeds things up…
Rust Performance Pitfalls
51–60 of 112 posts
Re: Rust Performance Pitfalls
#52> To get rid of the checks, we can either use bytes directly (usually via Vec / &[u8]) or, if we are absolutely sure the input will be valid UTF-8, use str::from_utf8_unchecked(_) (note that this will require unsafe and break your code in surprising ways should the input not be valid UTF-8). I believe this needs a stronger warning. Functions that operate on strings are allowed to assume that their input is valid UTF-…
That seems like a really unfortunate design decision. I used to think that Java's use of UTF-16 for strings was just a problematic legacy thing, but compared to this it seems quite good. Strings are pretty high performance and there are no complex calculations to do indexing or bounds checks. And in Java 9 the JVM can switch between UTF-16 or Latin1 encodings on the fly, which both uses less RAM and speeds things up…
I do have my unrelated niche complaints about Rust's string story (and I have vague plans to resolve them), but Rust's string implementation is my favorite among any other language I've used.
Re: Rust Performance Pitfalls
#53Earlier quoted context omitted.
The nature of warnings means that any time you make assumptions about those that might need the warnings having enough knowledge to correctly assess an ambiguity, you've likely just failed a portion of the people the warning was meant to help. The correct response when warning people of potential problems is never "oh, they should be able to figure out whether this applies to their case".
Right. The correct answer here is to convert external data into UTF-8 once and keep it that way. You're probably not going to become compute-bound, even if you're just reading in text and writing it out again. The UTF-8 check is linear time.
Re: Rust Performance Pitfalls
#54 use std::mem;
enum MyEnum {
A { name: String, x: u8 },
B { name: String }
}
fn a_to_b(e: &mut MyEnum) {
// we mutably borrow `e` here. This precludes us from changing it directly
// as in `*e = ...`, because the borrow checker won't allow it. Therefore
// the assignment to `e` must be outside the `if let` clause.
*e = if let MyEnum::A { ref mut name, x: 0 } = *e {
// this takes out our `name` and put in an empty String instead
// (note that empty strings don't allocate).
// Then, construct the new enum variant (which will
// be assigned to `*e`, because it is the result of the `if let` expression).
MyEnum::B { name: mem::replace(name, String::new()) }
// In all other cases, we return immediately, thus skipping the assignment
} else { return }
}
Don't get me wrong, I think Rust is an incredibly impressive language, but this is nuts. For the first time ever, I had a moment of appreciation for the "simple clarity" of C++11 move constructors. If it weren't for the comments and the documentation[1] I wouldn't have had the slightest clue what this code is doing (a hack to fool the borrow checker while allowing for Sufficiently Smart Compilation to a no-op[2] ... I think).This is a good example of the main conceptual aspect of Rust where I feel it could use improvement.[3] A lot of its features marry very high-level concepts (like algebraic data types) to exposed, low-level implementations (like tagged unions). Now, there's nothing wrong with the obvious choice to implement ADTs as tagged unions, but the nature of Rust as a language that exposes low-level control over allocation and addressing, in combination with the strictures of the borrow checker, means enums and other high-level features live in a sort of uncanny valley, falling short of either the high-level expressiveness of ADTs or the low-level flexibility of tagged unions (without expert-level knowledge or using `unsafe`).
Similarly, the functional abstractions almost feel like a step backwards from the venerable C++ , abstraction and composition-wise. Very nitty-gritty, low-level implementation details leak out like a sieve — you can't have your maps or folds without a generous sprinking of `as_slice()`, `unwrap()`, `iter()` / `iter_mut()` / `into_iter()` and `collect()` everywhere, although at least for this case you would be able to figure out the idioms by reading the official docs. But nevertheless it seems like reasonable defaults could be inferred from context (with the possible exception of `collect()` since it allocates), while still allowing explicitness as an option when you want low-level control over the code the compiler generates.
In this case, enums are a language-level construct, not a library, so the borrow-checker really shouldn't be rejecting reasonable patterns. It (legal to alias between the structural and nominal common subsequence of several members of an enum) should be the default behavior and not require an nonobvious, unreadable hack such as the above. At the very least that behavior (invaluable for many low-level tasks) should be easy to opt in to with e.g. an attribute.[4]
[1] https://doc.rust-lang.org/std/mem/fn.replace.html
[2] Or rather since it is a tagged union, in MASMish pseudocode something like
jnz x_nonzero
mov MyEnum.B, e.tag
x_nonzero:
ret
Although it could still be a no-op depending on what else Sufficiently Smart Compiler/LLVM inlines.[3] And I'm not saying I have the solution or even that all-around-better solutions exist.
[4] Something like
#![safe_alias_common_subsequence(structural)]
#![safe_alias_common_subsequence(nominal_and_structural)]Re: Rust Performance Pitfalls
#55Earlier quoted context omitted.
No, I don't believe it does. There are perfectly safe operations that must be done in unsafe, because the compiler is not smart enough to determine they are safe. Thus when describing a potential operation, saying it "requires unsafe" does not imply it can blow up in your face, just that the compiler is not smart enough to determine that at this time. This is exactly why warnings should strive to be as clear as possi…
> No, I don't believe it does. This is mistaken, and an impression that we continually strive very hard to counter. The `unsafe` keyword is to be used in the process of writing (and therefore consuming) APIs if and only if those APIs have external unenforced invariants which, if broken, could cause memory unsafety. The reason that we strive to reinforce this so fervently is precisely because we want people to see the…
> This is mistaken, and an impression that we continually strive very hard to counter.
> In particular, this means that `unsafe` is not to be used for operations that may be dangerous but that have nothing to do with memory safety.
That's not what I'm talking about. I think you've misinterpreted my point. I'm talking about unsafe for memory access, but in ways that are provably (or it not provable, are accepted as) safe. For example, the standard libs that are unsafe under the covers, but expose a safe API.
Saying "you must use unsafe to accomplish this" is not equivalent to saying "this can cause memory unsafety". That latter is a subset of the former, and the only time that isn't true is if Rust is capable of completely identifying every case of safe memory access and only requiring unsafe for actual unsafe operations.
Since unsafe could be required for what is an entirely safe, and possibly provably so, set of actions, it's dependent on what the context the recommendation is whether you believe someone stating an action requires unsafe implies that actual problems could occur.
Re: Rust Performance Pitfalls
#56Earlier quoted context omitted.
"requires unsafe" already implies everything in your comment. `unsafe` literally means "this might violate memory safety"
unsafe litearally means "you, compiler, cannot verify that this does not violate memory safety, but I have done so, so please trust me." Violating memory safety in unsafe code is UB.
Re: Rust Performance Pitfalls
#57Earlier quoted context omitted.
No, I don't believe it does. There are perfectly safe operations that must be done in unsafe, because the compiler is not smart enough to determine they are safe. Thus when describing a potential operation, saying it "requires unsafe" does not imply it can blow up in your face, just that the compiler is not smart enough to determine that at this time. This is exactly why warnings should strive to be as clear as possi…
Saying something "requires unsafe". Literally means "something violates the safety model". --- Here is the truth about safety guarantees. You can do a lot valid things type systems prevent you from doing. You can do a lot of valid things if/else/for/while prevent you from doing. But 99% of the time is isn't worth it. Following the rules, even with strange BS they create is easier than managing the mess of GOTO's you'…
Yes. But the safety model is not capable of identifying whether something is or is not safe in every case, just that it can't prove that it is safe. There are plenty of things you must do in unsafe that are safe, just not provably by the compiler. Thus, needing to use unsafe does not always imply actual unsafe operations.
If someone was under the impression that they were recommended to use unsafe but in a safe way but there were other caveats to the usage they weren't aware of, bad things could happen. I don't believe it's sufficient in a guide to rely on the fact that unsafe is recommended to convey the level of danger that recommendation entails.
Re: Rust Performance Pitfalls
#58Earlier quoted context omitted.
'break your code in surprising ways' is super vague. Saying 'may cause out of bounds memory access/writes should the input not be valid UTF-8' explicitly is probably more scary.
The nature of undefined behavior is in fact super vague; it's not actually possible to say what will happen.
Re: Rust Performance Pitfalls
#59Writing a iterator to provide the individual bits supplied by an iterator of bytes means you can count them with
fn count_bits>(it : I) -> i32{
let mut a=0;
for i in it {
if i {a+=1};
}
return a;
}
Counting bits in an array of bytes would need something like this let p:[u8;6] = [1,2,54,2,3,6];
let result = count_bits(bits(p.iter().cloned()));
Checking what that generates in asm https://godbolt.org/g/iTyfapThe core of the code is
.LBB0_4:
mov esi, edx ;edx has the current mask of the bit we are looking at
and esi, ecx ;ecx is the byte we are examining
cmp esi, 1 ;check the bit to see if it is set (note using carry not zero flag)
sbb eax, -1 ;fun way to conditionally add 1
.LBB0_1:
shr edx ;shift mask to the next bit
jne .LBB0_4 ;if mask still has a bit in it, go do the next bit otherwise continue to get the next byte
cmp rbx, r12 ;r12 has the memory location of where we should stop. Are we there yet?
je .LBB0_5 ; if we are there, jump out. we're all done
movzx ecx, byte ptr [rbx] ;get the next byte
inc rbx ; advance the pointer
mov edx, 128 ; set a new mask starting at the top bit
jmp .LBB0_4 ; go get the next bit
.LBB0_5:
Apart from magical bit counting instructions this is close to what I would have written in asm mysef. That really impressed me. I'm still a little wary of hitting a performance cliff. I worry that I can easily add something that will mean the optimiser bails on the whole chain, but so far I'm trusting Rust more than I have trusted any other Optimiser.If this produces simiarly nice code (I haven't checked yet) I'll be very happy
for (dest,source) in self.buffer.iter_mut().zip(data) { *dest=source }Re: Rust Performance Pitfalls
#60>Sometimes, when changing an enum, we want to keep parts of the old value. Use mem::replace to avoid needless clones. use std::mem; enum MyEnum { A { name: String, x: u8 }, B { name: String } } fn a_to_b(e: &mut MyEnum) { // we mutably borrow `e` here. This precludes us from changing it directly // as in `*e = ...`, because the borrow checker won't allow it. Therefore // the assignment to `e` must be outside the `if…
Another problem with your proposal is that it's not safe to access the structural and nominal common subsequence of several members of an enum in the same way, because the Rust compiler can and will reorder the fields differently in different variants in order to fill padding.
That said, it sounds like what you're looking for is one of the various inheritance proposals, which have safe access to common enum fields as one of the main guarantees. This would make this pattern much more ergonomic.