[flagged]
In this case it is a hardware bug and in no way attributable to Python being fast.
Rust std fs slower than Python? No, it's hardware
11–20 of 255 posts
Re: Rust std fs slower than Python? No, it's hardware
#12[flagged]
reminds me of HP48 programming, there was sysrpl which was near asm speed, and then user rpl which was slooow
Re: Rust std fs slower than Python? No, it's hardware
#13[flagged]
It's worth reading the article. In this case, it seems to have been a hardware issue - as such, not directly related to Rust, C, or Python, but triggered by an instruction that was only called by some file loading routines. It's a very cool deep dive into debugging these sorts of issues.
Re: Rust std fs slower than Python? No, it's hardware
#14[flagged]
It's worth reading the article. In this case, it seems to have been a hardware issue - as such, not directly related to Rust, C, or Python, but triggered by an instruction that was only called by some file loading routines. It's a very cool deep dive into debugging these sorts of issues.
Although true that it's great article.
It states that python is faster then c, that is not possible since python is build with c. There could be other reasons such libs or implementation.
Also note that the issue he had was not resolved.
The comment was about that python is seen as slow. But that is not always the case.
Once a dev is able to understand the difference between the python and c parts. Python can be quite performant, and efficient with memory.
But if one would actually create a application that does more then just read a file it will be slow again compared to c and rust.
Re: Rust std fs slower than Python? No, it's hardware
#15Earlier quoted context omitted.
It's worth reading the article. In this case, it seems to have been a hardware issue - as such, not directly related to Rust, C, or Python, but triggered by an instruction that was only called by some file loading routines. It's a very cool deep dive into debugging these sorts of issues.
Although true that it's great article. It states that python is faster then c, that is not possible since python is build with c. There could be other reasons such libs or implementation. Also note that the issue he had was not resolved. The comment was about that python is seen as slow. But that is not always the case. Once a dev is able to understand the difference between the python and c parts. Python can be quit…
It's not stating python is faster than c in general. This is just one very specific case where non-page-aligned memeory reading on AMD is involved.
Re: Rust std fs slower than Python? No, it's hardware
#16Earlier quoted context omitted.
Although true that it's great article. It states that python is faster then c, that is not possible since python is build with c. There could be other reasons such libs or implementation. Also note that the issue he had was not resolved. The comment was about that python is seen as slow. But that is not always the case. Once a dev is able to understand the difference between the python and c parts. Python can be quit…
It's not stating python is faster than c in general. This is just one very specific case where non-page-aligned memeory reading on AMD is involved.
The context of my initial comment is that python is slow, but can be fast.
From the article.
> In conclusion, the issue isn't software-related. Python outperforms C/Rust due to an AMD CPU bug.
Re: Rust std fs slower than Python? No, it's hardware
#17Earlier quoted context omitted.
Although true that it's great article. It states that python is faster then c, that is not possible since python is build with c. There could be other reasons such libs or implementation. Also note that the issue he had was not resolved. The comment was about that python is seen as slow. But that is not always the case. Once a dev is able to understand the difference between the python and c parts. Python can be quit…
It's not stating python is faster than c in general. This is just one very specific case where non-page-aligned memeory reading on AMD is involved.
It does make me wonder why pymallov and jemalloc used page aligned memory, but glibc didn't. That is odd. Other questions never answered, why did pyo3 add so much overhead? it was over half the difference between the two.
Re: Rust std fs slower than Python? No, it's hardware
#18I'm a bit confused about the premise. This is not comparing pure Python code against some native (C or Rust) code. It's comparing one Python wrapper around native code (Python's file read method) against another Python wrapper around some native code (OpenDAL). OK it's still interesting that there's a difference in performance, but it's very odd to describe it as "slower than Python". Did they expect that the Python standard library is all written in pure Python? On the contrary, I would expect the implementations of functions in Python's standard library to be native and, individually, highly optimised.
I'm not surprised the conclusion had something to do with the way that native code works. Admittedly I was surprised at the specific answer - still a very interesting article despite the confusing start.
Edit: The conclusion also took me a couple of attempts to parse. There's a heading "C is slower than Python with specified offset". To me, as a native English speaker, this reads as "C is slower (than Python) with specified offset" i.e. it sounds like they took the C code, specified the same offset as Python, and then it's still slower than Python. But it's the opposite: once the offset from Python was also specified in the C code, the C code was then faster. Still very interesting once I got what they were saying though.
Re: Rust std fs slower than Python? No, it's hardware
#19Earlier quoted context omitted.
It's not stating python is faster than c in general. This is just one very specific case where non-page-aligned memeory reading on AMD is involved.
It does make me wonder why pymallov and jemalloc used page aligned memory, but glibc didn't. That is odd. Other questions never answered, why did pyo3 add so much overhead? it was over half the difference between the two.
> It does make me wonder why pymallov and jemalloc used page aligned memory, but glibc didn't.
The root cause is not about page alignment. In fact, all allocators are aligned.
The root cause is AMD CPU didn't implement FSRM correctly while copying data from 0x1000 * n ~ 0x1000 * n + 0x10.
> Other questions never answered, why did pyo3 add so much overhead? it was over half the difference between the two.
OpenDAL Python Binding v0.42 does have many place to improve, like we can alloc the buffer in advance or using `read_buf` into uninit vec. I skipped this part since they are not the root cause.