Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust std fs slower than Python? No, it's hardware

xuanwo.io

Post reply on HN
Print view View original

211–220 of 255 posts

Re: Rust std fs slower than Python? No, it's hardware

#211

Post by int_19h »

munch117 wrote: post #202
jwueller wrote: post #194

Earlier quoted context omitted.

How is it pure Python if it delegates all of the actual work to the Kernel?

All I/O delegates to the kernel, eventually. It's pure Python in that there's no cffi, no ctypes, no Cython, no C extensions of any kind.

It's pretty hard to draw this line in Python because all built-in types and functions are effectively C extensions, just compiled directly into the interpreter.

Conversely, you can have pure C code just using PyObjects (this is effectively what Cython does), with the Python bytecode interpreter completely out of the picture. But the perf improvement is nowhere near what people naively expect from compiled code, usually.

Re: Rust std fs slower than Python? No, it's hardware

#212

Post by JonChesterfield »

lmm wrote: post #195
kbenson wrote: post #128

Earlier quoted context omitted.

Because for any nontrivial case you would expect python+compiled library and associated marshaling of data to be slower than that library in its native implementation without any inyerop/marshaling required. When you see an interpreted language faster than a compiled one, it's worth looking at why, because most the time it's because there's some hidden issue causing the other to be slow (which could just be a differe…

> Because for any nontrivial case you would expect python+compiled library and associated marshaling of data to be slower than that library in its native implementation without any inyerop/marshaling required. > When you see an interpreted language faster than a compiled one, it's worth looking at why, because most the time it's because there's some hidden issue causing the other to be slow (which could just be a dif…

> On the contrary, the compiled languages tend to only be faster in trivial benchmarks. In real-world systems the Python-based systems tends to be faster because they haven't had to spend so long twiddling which integers they're using and debugging crashes and memory leaks, and got to spend more time on the problem.

This is an interesting premise.

Python in particular gets an absolute kicking for being slow. Hence all the libraries written in C or C++ then wrapped in a python interface. Also why "python was faster than rust at anything" is headline worthy.

I note your claim is that python systems in general tend to be faster (outside of trivial benchmarks, whatever the scope of that is). Can you cite any single example where this is the case?

Re: Rust std fs slower than Python? No, it's hardware

#213

Post by int_19h »

crabbone wrote: post #140
codr7 wrote: post #118

Earlier quoted context omitted.

Have you ever attempted to write a scripting language that performs better? I have, several, and it's far from trivial. The basics are seriously optimized for typical use cases, take a look at the source code for the dict type.

> Have you ever attempted to write a scripting language that performs better? No, because "scripting language" is not a thing. But, if we are talking about implementing languages, then I worked with many language implementations. The most comparable one that I know fairly well, inside-and-out would be the AVM, i.e. the ActionScript Virtual Machine. It's not well-written either unfortunately. I've looked at implementa…

For starters, since everything in Python is a pass-by-ref object, dicts store pointers to values, which then have to be allocated on the heap and refcounted, whereas std::map can store values directly. But this is the consequence of a very-high-level object model used by CPython, not its dict implementation that has to adapt to that.

Re: Rust std fs slower than Python? No, it's hardware

#214

Post by nh2 »

masklinn wrote: post #96
nh2 wrote: post #75

Earlier quoted context omitted.

Be aware `jemalloc` will make you suffer the observability issues of `MADV_FREE`. `htop` will no longer show the truth about how much memory is in use. * https://github.com/jemalloc/jemalloc/issues/387#issuecomment... * https://gitlab.haskell.org/ghc/ghc/-/issues/17411 Apparently now `jemalloc` will call `MADV_DONTNEED` 10 seconds after `MADV_FREE`: https://github.com/JuliaLang/julia/issues/51086#issuecomment... So w…

The musl remark is funny, because jemalloc's use of pretty fine-grained arenas sometimes leads to better memory utilisation through reduced fragmentation. For instance Aerospike couldn't fit in available memory under (admittedly old) glibc, and jemalloc fixed the issue: http://highscalability.com/blog/2015/3/17/in-memory-computin... And this is not a one-off: https://hackernoon.com/reducing-rails-memory-use-on-amazon…

Yes, almost everybody who looks at memory usage in production will eventually discover glibc's memory fragmentation issues. This is how I learned about this topic.

Setting the env var MALLOC_MMAP_THRESHOLD_=65536 usually solves these problems instantaneously.

Most programmers seem to not bother to understand what is going on (thus arriving at the above solution) but follow "we switched to jemalloc and it fixes the issue".

(I have no opinion yet on whether jemalloc is better or worse than glibc malloc. Both have tunables, and will create problematic corner cases if the tunables are not set accordingly. The fact that jemalloc has /more/ tunables, and more observability / debugging features, seems like a pro point for those that read the documentation. For users that "just want low memory usage", both libraries' defaults look bad, and the musl attitude seems like the best default, since OOM will cause a crash vs just having the program be some percent slower.)

Re: Rust std fs slower than Python? No, it's hardware

#215

Post by JonChesterfield »

lmm wrote: post #197
kbenson wrote: post #196

Earlier quoted context omitted.

I don't doubt that can happen, but I'm also highly doubtful that it's the norm for large, established, mature projects with lots of attention, such as popular libraries and the standard library of popular languages. As time spent on the project increases, I suspect that any gain an interpreted language has over an (efficient) compiled one not only gets smaller, but eventually reverses in most cases. So, like in most…

> I don't doubt that can happen, but I'm also highly doubtful that it's the norm for large, established, mature projects with lots of attention, such as popular libraries and the standard library of popular languages. Code that has lots of attention is different, certainly, but it's also the exception rather than the rule; the last figure I saw was that 90% of code is internal business applications that are never eve…

> a tiny interpreter that sits in L1 and takes its instructions in a very compact form ends up saving you more memory bandwidth

There's a paper on this you might like. https://www.researchgate.net/publication/2749121_When_are_By...

I think there's something to the idea of keeping the program in the instruction cache by deliberately executing parts of it via interpreted bytecode. There should be an optimum around zero instruction cache misses, either from keeping everything resident, or from deliberately paging instructions in and out as control flow in the program changes which parts are live.

There are complicated tradeoffs between code specialisation and size. Translating some back and forth between machine code and bytecode adds another dimension to that.

I fear it's either the domain of extremely specialised handwritten code - luajit's interpreter is the canonical example - of the the sufficiently smart compiler. In this case a very smart compiler.

Re: Rust std fs slower than Python? No, it's hardware

#216

Post by JonChesterfield »

codr7 wrote: post #150
crabbone wrote: post #140

Earlier quoted context omitted.

> Have you ever attempted to write a scripting language that performs better? No, because "scripting language" is not a thing. But, if we are talking about implementing languages, then I worked with many language implementations. The most comparable one that I know fairly well, inside-and-out would be the AVM, i.e. the ActionScript Virtual Machine. It's not well-written either unfortunately. I've looked at implementa…

Except it is, because everyone knows sort of what it means, an interpreted language that prioritizes convenience over performance; Perl/Python/Ruby/Lua/PHP/etc. SBCL is definitely a different beast. I would expect Emacs Lisp & Lua to be more similar. Erlang had plenty more funding and stricter requirements. C++'s std::map has most likely gotten even more attention than Python's dict, but I'm not sure from your commen…

(std::map is famously rubbish, to the extent that a common code review fix is to replace it with std::unordered_map. Map is a tree, unordered map is a linked-list-collision hashtable. Both are something of a performance embarrassment for C++. So std::map outperforming a given hashtable is a strongly negative judgement)

Re: Rust std fs slower than Python? No, it's hardware

#217

Post by nh2 »

the8472 wrote: post #148
nh2 wrote: post #75

Earlier quoted context omitted.

Be aware `jemalloc` will make you suffer the observability issues of `MADV_FREE`. `htop` will no longer show the truth about how much memory is in use. * https://github.com/jemalloc/jemalloc/issues/387#issuecomment... * https://gitlab.haskell.org/ghc/ghc/-/issues/17411 Apparently now `jemalloc` will call `MADV_DONTNEED` 10 seconds after `MADV_FREE`: https://github.com/JuliaLang/julia/issues/51086#issuecomment... So w…

Aiming to please people who panic about their RSS numbers seems... misguided? It seems like worrying about RAM being "used" as file cache[0]. If you want to gauge whether your system is memory-limited look at the PSI metrics instead. [0] https://www.linuxatemyram.com/

Those are not the same.

You can see cache usage in htop; it has a different colour.

With MADV_FREE, it looks like the process is still using the memory.

That sucks: If you have some server that's slow, you want to SSH into a server and see how much memory each process takes. That's a basic, and good, observability workflow. Memory leaks exist, and tools should show them easily.

The point of RES is to show resident memory, not something else.

If you change htop to show the correct memory, that'd fix the issue of course.

Re: Rust std fs slower than Python? No, it's hardware

#218

Post by dzaima »

the8472 wrote: post #143
giancarlostoro wrote: post #114

Earlier quoted context omitted.

So correct me if I am wrong but does this mean you need to compile two executables for a specific compile time build? Or is it just you need to compile it from specific hardware? Wondering what the fix would be, some sort of runtime check?

The sibling comments mention the hardware specific dynamic linking in glibc that's used for function calls. But if your compiler inlines memcpy (usually for short, fixed-sized copies) into the binary then yes you'll have to compile it for a specific CPU to get optimal performance. But that's true for all target-dependent optimizations. More broadly compatible routines will still work on newer CPUs, they just won yiel…

Some quick searching gives that FSRM is used for at least 128 bytes or so (ERMS for ≥~2048 bytes for reference); in base x86 (i.e. SSE2) that's 8 loads & 8 stores, ~62 bytes of code. At that point calling into a library function isn't too unreasonable (at the very least it could utilize AVX and cut that in half to 4 loads+4 stores, though at the cost of function call overhead & some (likely correctly-predicted) branches).

Re: Rust std fs slower than Python? No, it's hardware

#219

Post by saagarjha »

the8472 wrote: post #206
saagarjha wrote: post #179

Earlier quoted context omitted.

They are, glibc already has an ERMS code path for memcpy.

Did you miss the part about inlined calls? I was saying that if it weren't for these issues we could have them turn into a single instruction without worrying about perf.

Generally the compiler will not inline a memcpy if it doesn't know the size it is dealing with.

Re: Rust std fs slower than Python? No, it's hardware

#220

Post by saagarjha »

JonChesterfield wrote: post #210
saagarjha wrote: post #178

Earlier quoted context omitted.

This is generally a bad idea because it requires code modification, which has security implications. Most implementations will bring in multiple implementations and select the right one at startup (amortizing the indirect call into something like the GOT which already exists).

There are security hazards around writable + executable code. They don't apply to patching before execution (e.g. the install step) since nothing needs to be executed at that point. I don't think the security concerns apply during load time either - what does it matter if the text section is edited before it gets marked read-only&executable? It just means you're running a slightly different program, exactly as if it…

On certain platforms, it would break code signatures if they are tied to the pages the code is on.
Post reply on HN