Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust std fs slower than Python? No, it's hardware

xuanwo.io

Post reply on HN
Print view View original

101–110 of 255 posts

Re: Rust std fs slower than Python? No, it's hardware

#101

Post by rasz »

diamondlovesyou wrote: post #89

AMD's string store is not like Intel's. Generally, you don't want to use it until you are past the CPU's L2 size (L3 is a victim cache), making ~2k WAY too small. Once past that point, it's profitable to use string store, and should run at "DRAM speed". But it has a high startup cost, hence 256bit vector loads/stores should be used until that threshold is met.

Or you leave it as is forcing AMD to fix their shit. "fast string mode" has been strongly hinted as _the_ optimal way over 30 years ago with Pentium Pro, further enforced over 10 years ago with ERMSB and FSRM 4 years ago. AMD get with the program.

Re: Rust std fs slower than Python? No, it's hardware

#102

Post by topaz0 »

meneer_oke wrote: post #40
Pop_- wrote: post #23

Earlier quoted context omitted.

The root cause is AMD's bad support for rep movsb (which is a hardware problem). However, python by default has a small offset when reading memories while lower level language (rust and c) does not, which is why python seems to perform better than c/rust. It "accidentally" avoided the hardware problem.

It doesn't seem faster. Seem would imply that it isn't the case. It is faster currently on that setup. But since python runtime is written in C, the issue can't be Python vs C.

It's obviously not python vs c -- the time difference turns out to be in kernel code (system call) and not user code at all, and the post explicitly constructs a c program that doesn't have the slowdown by adding a memory offset. It just turns up by default in a comparison of python vs c code because python reads have a memory offset by default (for completely unrelated reasons) and analogous c reads don't by default. In principle you could also construct python code that does see this slowdown, it would just be much less likely to show up at random. So the python vs c comp is a total red herring here, it just happened to be what the author noticed and used as a hook to understand the problem.

Re: Rust std fs slower than Python? No, it's hardware

#103

Post by sound1 »

pornel wrote: post #38
3cats-in-a-coat wrote: post #36

Earlier quoted context omitted.

What's the TLDR on how... hardware performs differently on two software runtimes?

AMD's implementation of `rep movsb` instruction is surprisingly slow when addresses are page aligned. Python's allocator happens to add a 16-byte offset that avoids the hardware quirk/bug.

thank you, upvoted!

Re: Rust std fs slower than Python? No, it's hardware

#104

Post by qd011 »

quietbritishjim wrote: post #18

I'm a bit confused about the premise. This is not comparing pure Python code against some native (C or Rust) code. It's comparing one Python wrapper around native code (Python's file read method) against another Python wrapper around some native code (OpenDAL). OK it's still interesting that there's a difference in performance, but it's very odd to describe it as "slower than Python". Did they expect that the Python…

I don't understand why Python gets shit for being a slow language when it's slow but no credit for being fast when it's fast just because "it's not really Python".

If I write Python and my code is fast, to me that sounds like Python is fast, I couldn't care less whether it's because the implementation is in another language or for some other reason.

Re: Rust std fs slower than Python? No, it's hardware

#105

Post by saghm »

mwcampbell wrote: post #28
drtgh wrote: post #21

>Rust std fs slower than Python!? No, it's hardware! >... >Python features three memory domains, each representing different allocation strategies and optimized for various purposes. >... >Rust is slower than Python only on my machine. if one library performs wildly better than the other in the same test, on the same hardware, how can that not be a software-related problem? sounds like a contradiction. Maybe should b…

Years ago, Rust's standard library used jemalloc. That decision substantially increased the minimum executable size, though. I didn't publicly complain about it back then (as far as I can recall), but perhaps others did. So the Rust library team switched to using the OS's allocator by default. Maybe using an alternative allocator only solves the problem by accident and there's another way to solve it intentionally; I…

> I didn't publicly complain about it back then (as far as I can recall), but perhaps others did. So the Rust library team switched to using the OS's allocator by default.

I've honestly never worked in a domain where binary size ever really mattered beyond maybe invoking `strip` on a binary before deploying it, so I try to keep an open mind. That said, this has always been a topic of discussion around Rust[0], and while I obviously don't have anything against binary sizes being smaller, bugs like this do make me wonder about huge changes like switching the default allocator where we can't really test all of the potential side effects; next time, the unintended consequences might not be worth the tradeoff.

[0]: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...

Re: Rust std fs slower than Python? No, it's hardware

#106

Post by vlovich123 »

masklinn wrote: post #99
vlovich123 wrote: post #87

Earlier quoted context omitted.

I don’t know why people still look to jemalloc. Mimalloc outperforms the standard allocator on nearly every single benchmark. Glibc’s allocator & jemalloc both are long in the tooth & don’t actually perform as well as state of the art allocators. I wish Rust would switch to mimalloc or the latest tcmalloc (not the one in gperftools).

> I wish Rust would switch to mimalloc or the latest tcmalloc (not the one in gperftools). That's nonsensical. Rust uses the system allocators for reliability, compatibility, binary bloat, maintenance burden, ..., not because they're good (they were not when Rust switched away from jemalloc, and they aren't now). If you want to use mimalloc in your rust programs, you can just set it as global allocator same as jemall…

I know it’s easy to change but the arguments for using glibc’s allocator are less clear to me:

1. Reliability - how is an alternate allocator less reliable? Seems like a FUD-based argument. Unless by reliability you mean performance in which case yes - jemalloc isn’t reliably faster than standard allocators, but mimalloc is.

2. Compatibility - again sounds like a FUD argument. How is compatibility reduced by swapping out the allocator? You don’t even have to do it on all systems if you want. Glibc is just unequivocally bad.

3. Binary bloat - This one is maybe an OK argument although I don’t know what size difference we’re talking about for mimalloc. Also, most people aren’t writing hello world applications so the default should probably be for a good allocator. I’d also note that having a dependency of the std runtime on glibc in the first place likely bloats your binary more than the specific allocator selected.

4. Maintenance burden - I don’t really buy this argument. In both cases you’re relying on a 3rd party to maintain the code.

Re: Rust std fs slower than Python? No, it's hardware

#108

Post by js2 »

diamondlovesyou wrote: post #89

AMD's string store is not like Intel's. Generally, you don't want to use it until you are past the CPU's L2 size (L3 is a victim cache), making ~2k WAY too small. Once past that point, it's profitable to use string store, and should run at "DRAM speed". But it has a high startup cost, hence 256bit vector loads/stores should be used until that threshold is met.

Isn't the high startup cost what FSRM is intended to solve?

> With the new Zen3 CPUs, Fast Short REP MOV (FSRM) is finally added to AMD’s CPU functions analog to Intel’s X86_FEATURE_FSRM. Intel had already introduced this in 2017 with the Ice Lake Client microarchitecture. But now AMD is obviously using this feature to increase the performance of REP MOVSB for short and very short operations. This improvement applies to Intel for string lengths between 1 and 128 bytes and one can assume that AMD’s implementation will look the same for compatibility reasons.

https://www.igorslab.de/en/cracks-on-the-core-3-yet-the-5-gh...

Re: Rust std fs slower than Python? No, it's hardware

#109

Post by kragen »

a1o wrote: post #33

> Rust developers might consider switching to jemallocator for improved performance I am curious if this is something that everyone can do to get free performance or if there are caveats. Can C codebases benefit from this too? Is this performance that is simply left on table currently?

basically that's why jason wrote it in the first place, but other allocators have caught up since then to some extent. so jemalloc might make your c either slower or faster, you'll have to test to know. it's pretty reliable at being close to the best choice

does tend to use more ram tho

Re: Rust std fs slower than Python? No, it's hardware

#110

Post by mrweasel »

formerly_proven wrote: post #32
Pop_- wrote: post #23

Earlier quoted context omitted.

The root cause is AMD's bad support for rep movsb (which is a hardware problem). However, python by default has a small offset when reading memories while lower level language (rust and c) does not, which is why python seems to perform better than c/rust. It "accidentally" avoided the hardware problem.

That extra 0x20 (32 byte) offset is the size of the PyBytes object header for anyone wondering; 64 bits each for type object pointer, reference count, base pointer and item count.

Thank you, because I was wondering if some Python developer found the same issue and decided to just implement the offset. It makes much more sense that it just happens to work out that way in Python.
Post reply on HN