Live data from Hacker News

Rust Performance Pitfalls

llogiq.github.io

101–110 of 112 posts

Re: Rust Performance Pitfalls

#101
"Similarly, the Read::lines() iterator is very easy to use, but it has one downside: It allocates a String for each line. Manually allocating and reusing a String will reduce memory churn and may gain you a bit of performance."

Back when Rust used "internal" iterators, this could be the default. Now, you can encapsulate it by passing a function to handle each line.

Re: Rust Performance Pitfalls

#102

Earlier quoted context omitted.

but that's only a comfort for the programmer's ability to reason about the code A way to mark functions as pure for this purpose would be great! Especially if it's not as fraught as const in C++.

We actually did have this once, but it wasn't really worth it, so it was removed. https://news.ycombinator.com/item?id=6940624 is the HN discussion, but it looks like the link might now be wrong? It was also a very, very long time ago, and so today's Rust might be different enough that those reasons don't apply any more.

I think that being able to enforce purity would be fantastic for actors, agents, and/or game loops.

https://news.ycombinator.com/item?id=8609775

Re: Rust Performance Pitfalls

#103
post #59

I have been doing some exploration of how well Rust optimizes Iterators and have been quite impressed. Writing a iterator to provide the individual bits supplied by an iterator of bytes means you can count them with fn count_bits >(it : I) -> i32{ let mut a=0; for i in it { if i {a+=1}; } return a; } Counting bits in an array of bytes would need something like this let p:[u8;6] = [1,2,54,2,3,6]; let result = count_bi…

Okay, wait. Every modern CPU has a popcount instruction, so any hand-coded implementation would use that, meaning the compiler output is actually pretty bad in an absolute sense.

But if you find popcount too "magical", the commonly-known fast way to count bits is via masking, shifts and adds, so that you do it in log(n) steps. Which also would perform much better than this solution.

So what you're really saying is "the compiler managed to make a pretty efficient representation of the naive solution" which is fine but it does not mean your code is fast.

Re: Rust Performance Pitfalls

#104
post #98
post #63

Earlier quoted context omitted.

A buffer overflow doesn't necessarily cause a segfault - that's the problem! A guaranteed segfault would be a completely valid and safe way to handle a buffer overflow. But segfaults only happen when your program tries to access memory it does not own. It is improbable that an overflowing buffer is straight at a page boundary, doubly so if the buffer is allocated on the stack. Instead, you get gibberish output, an in…

Yes, the overflow either segfaults your program, or doesn't. It's not undefined. Computers don't perform random operations. Just say "it will produce buffer overflow" - that is absolutely a defined operation: the CPU will execute a load from an address that hasn't been defined at that place in code and will either contain leftover data, or refer to an unmapped page, triggering a segfault. This isn't quantum mechanics…

As soon as you use more than one clock domain it's no longer completely deterministic.

https://en.wikipedia.org/wiki/Metastability_in_electronics

Re: Rust Performance Pitfalls

#105
post #58

Earlier quoted context omitted.

The nature of undefined behavior is in fact super vague; it's not actually possible to say what will happen.

Just say the worst thing that can happen, out of bounds access probably qualifies. :-)

"Could potentially emit code to rm -rf /"

Re: Rust Performance Pitfalls

#106
post #103
post #59

I have been doing some exploration of how well Rust optimizes Iterators and have been quite impressed. Writing a iterator to provide the individual bits supplied by an iterator of bytes means you can count them with fn count_bits >(it : I) -> i32{ let mut a=0; for i in it { if i {a+=1}; } return a; } Counting bits in an array of bytes would need something like this let p:[u8;6] = [1,2,54,2,3,6]; let result = count_bi…

Okay, wait. Every modern CPU has a popcount instruction, so any hand-coded implementation would use that, meaning the compiler output is actually pretty bad in an absolute sense. But if you find popcount too "magical", the commonly-known fast way to count bits is via masking, shifts and adds, so that you do it in log(n) steps. Which also would perform much better than this solution. So what you're really saying is "t…

> Okay, wait. Every modern CPU has a popcount instruction

What do you consider a "modern CPU"? Atom chips sold less than a decade ago didn't support popcnt. AMD shipped some C-series chips without support for it as recently as 2012. The low-end chips were likely to be sold later without refreshes to newer features in some cases, and those are also likely the ones to be repurposed for small and cheap x86 devices later. I wouldn't want the default compilation settings without specifying CPU extensions to use an extension that might not exist on my target platform.

Re: Rust Performance Pitfalls

#107
post #98
post #63

Earlier quoted context omitted.

A buffer overflow doesn't necessarily cause a segfault - that's the problem! A guaranteed segfault would be a completely valid and safe way to handle a buffer overflow. But segfaults only happen when your program tries to access memory it does not own. It is improbable that an overflowing buffer is straight at a page boundary, doubly so if the buffer is allocated on the stack. Instead, you get gibberish output, an in…

Yes, the overflow either segfaults your program, or doesn't. It's not undefined. Computers don't perform random operations. Just say "it will produce buffer overflow" - that is absolutely a defined operation: the CPU will execute a load from an address that hasn't been defined at that place in code and will either contain leftover data, or refer to an unmapped page, triggering a segfault. This isn't quantum mechanics…

I'm sorry but this is useless pedantry that adds nothing to the discussion.

Re: Rust Performance Pitfalls

#108
post #84
post #79

Earlier quoted context omitted.

Even if someone that you trust is telling you to use a specific unsafe API and that "it's no less safe than when we do the same in C/C++", proper responsible usage of that API always requires reading the documentation to determine which invariants must be upheld, if only because you ought to be documenting those exact same invariants (and describing the measures that you take to uphold them) in your own code. Don't t…

If someone says "here's a simple mergesort implementation. It requires unsafe in a few sports, for performance"[1] or even "you can implement a fairly normal mergesort, but oyu may want to use unsafe in areas A and B for speed", I view that differently than if someone says "you can access strings without confirming they are UTF-8 encoded with str::from_utf8_unchecked, but it requires unsafe". One, has, for the most p…

For what it's worth, I've used Rust for years and understand what you're saying and completely agree.

Re: Rust Performance Pitfalls

#109
post #103

Earlier quoted context omitted.

Okay, wait. Every modern CPU has a popcount instruction, so any hand-coded implementation would use that, meaning the compiler output is actually pretty bad in an absolute sense. But if you find popcount too "magical", the commonly-known fast way to count bits is via masking, shifts and adds, so that you do it in log(n) steps. Which also would perform much better than this solution. So what you're really saying is "t…

> Okay, wait. Every modern CPU has a popcount instruction What do you consider a "modern CPU"? Atom chips sold less than a decade ago didn't support popcnt. AMD shipped some C-series chips without support for it as recently as 2012. The low-end chips were likely to be sold later without refreshes to newer features in some cases, and those are also likely the ones to be repurposed for small and cheap x86 devices later…

This is why you have a fallback to a generic version.

Re: Rust Performance Pitfalls

#110
post #103
post #59

I have been doing some exploration of how well Rust optimizes Iterators and have been quite impressed. Writing a iterator to provide the individual bits supplied by an iterator of bytes means you can count them with fn count_bits >(it : I) -> i32{ let mut a=0; for i in it { if i {a+=1}; } return a; } Counting bits in an array of bytes would need something like this let p:[u8;6] = [1,2,54,2,3,6]; let result = count_bi…

Okay, wait. Every modern CPU has a popcount instruction, so any hand-coded implementation would use that, meaning the compiler output is actually pretty bad in an absolute sense. But if you find popcount too "magical", the commonly-known fast way to count bits is via masking, shifts and adds, so that you do it in log(n) steps. Which also would perform much better than this solution. So what you're really saying is "t…

As I stated in another reply. I don't actually want to count bits at all. Counting is just the simplest thing I could do with the bits in the test case. Turning a stream of bytes into a stream of bits can be quite useful.
Post reply on HN