Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why does musl make my Rust code so slow?

andygrove.io

Post reply on HN
Print view View original

71–77 of 77 posts

Re: Why does musl make my Rust code so slow?

#71

Post by dalias »

littlestymaar wrote: post #58
harrygeez wrote: post #49

Earlier quoted context omitted.

well one of the stated goals of musl is to be simple and correct, and all those mallocs are anything but simple

That's true for jemalloc, but mimalloc is pretty simple. The reference paper is pretty short and really accessible and IIRC the implementation is around 5klocs. I doubt musl's implementation would much simpler than this.

5kloc is about 10x larger than musl's existing (old) malloc in source lines. I suspect lots of that is low code density, comments, etc.

I have to lookup what exactly mimalloc is/does every time someone mentions it, because the readme/documentation isn't very descriptive except discussing extensions outside the normal API. I didn't have time to dig through this again today. But I did look at it in some depth on several occasions in the past and it really wasn't suitable for or comparable to what we're doing in musl.

Re: Why does musl make my Rust code so slow?

#73

Post by fluffything »

dalias wrote: post #69
scott_s wrote: post #63

Earlier quoted context omitted.

Curiously, it doesn't adopt the now-standard approach for multithreaded support: per-thread memory pools, allowing one thread allocating and deallocating the same memory to avoid synchronization. This uses one lock guarding allocation, which means that it can be a bottleneck in a multithreaded workload.

The justifications are partly the same as what Daniel Micay has written extensively on in the rational for hardened_malloc ( https://github.com/GrapheneOS/hardened_malloc ) - unsynchronized per-thread state inherently sacrifices global consistency for performance and makes it impossible to detect a lot of types of memory usage errors (DF/UAF, etc) that could otherwise be caught. However musl has the additional constr…

> However musl has the additional constraint of being compatible with small/very-low-memory environments.

How many threads do these have ?

If they only have one thread, they'll use 72x less memory than if they would have 72 threads.

The thing is that if you are using 72 threads you probably would like your application to be 72x faster than if you are using only one. So synchronizing all allocations and killing scalability doesn't solve these users problems.

Most allocators, including jemalloc, tcmalloc, mimalloc, etc. have a "hardened" mode, that people can opt into if they want.

If I'm using Rust like the user in the blog post, double frees are caught at compile-time, so I'd rather not pay for them at run-time.

Re: Why does musl make my Rust code so slow?

#74

Post by scott_s »

dalias wrote: post #69
scott_s wrote: post #63

Earlier quoted context omitted.

Curiously, it doesn't adopt the now-standard approach for multithreaded support: per-thread memory pools, allowing one thread allocating and deallocating the same memory to avoid synchronization. This uses one lock guarding allocation, which means that it can be a bottleneck in a multithreaded workload.

The justifications are partly the same as what Daniel Micay has written extensively on in the rational for hardened_malloc ( https://github.com/GrapheneOS/hardened_malloc ) - unsynchronized per-thread state inherently sacrifices global consistency for performance and makes it impossible to detect a lot of types of memory usage errors (DF/UAF, etc) that could otherwise be caught. However musl has the additional constr…

Thanks for the clear explanation. Looking at the source code, it looks similar to modern allocators, just without the per-thread heaps. (I think all modern allocators use size-class slab allocators for small objects.) Curiously, I don't think the academic community has much literature on hardened allocators. It's been a while since I've worked in the area, but I wasn't aware of any other than DieHard from 2006 [1]. I did some searched on the ACM Digital Library (I love that it's all free right now so I can easily provide links in forums), and the only other thing I could find was FreeGuard from 2017 [2]. Maybe the issue there is that academics who design memory allocators tend to be on the systems side of CS, and such people tend to use raw performance as a part of the evaluation. Better security for a new thing does not show up in a graph. (Even that FreeGuard paper from 2017 claims security with better performance.)

In the non-academic world, I found the one we're discussing, but also Scudo (https://llvm.org/docs/ScudoHardenedAllocator.html). And that's it. If I still worked in the area, I would try to go after scalable hardened allocators. I wonder if there's still some clever stuff we haven't thought of there.

[1] https://github.com/emeryberger/DieHard, https://dl.acm.org/doi/abs/10.1145/1133981.1134000

[2] https://github.com/UTSASRG/FreeGuard, https://dl.acm.org/doi/abs/10.1145/3133956.3133957

Re: Why does musl make my Rust code so slow?

#75

Post by jessermeyer »

fluffything wrote: post #73
dalias wrote: post #69

Earlier quoted context omitted.

The justifications are partly the same as what Daniel Micay has written extensively on in the rational for hardened_malloc ( https://github.com/GrapheneOS/hardened_malloc ) - unsynchronized per-thread state inherently sacrifices global consistency for performance and makes it impossible to detect a lot of types of memory usage errors (DF/UAF, etc) that could otherwise be caught. However musl has the additional constr…

> However musl has the additional constraint of being compatible with small/very-low-memory environments. How many threads do these have ? If they only have one thread, they'll use 72x less memory than if they would have 72 threads. The thing is that if you are using 72 threads you probably would like your application to be 72x faster than if you are using only one. So synchronizing all allocations and killing scalab…

Not less than two weeks ago the DragonFly kernel allocator made related improvements for very high core CPUs.

https://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/018...

Re: Why does musl make my Rust code so slow?

#77

Post by littlestymaar »

dalias wrote: post #71
littlestymaar wrote: post #58

Earlier quoted context omitted.

That's true for jemalloc, but mimalloc is pretty simple. The reference paper is pretty short and really accessible and IIRC the implementation is around 5klocs. I doubt musl's implementation would much simpler than this.

5kloc is about 10x larger than musl's existing (old) malloc in source lines. I suspect lots of that is low code density, comments, etc. I have to lookup what exactly mimalloc is/does every time someone mentions it, because the readme/documentation isn't very descriptive except discussing extensions outside the normal API. I didn't have time to dig through this again today. But I did look at it in some depth on severa…

Thanks for clarification, didn't imagine that musl's malloc was that minimalist.

You should definitely have a look at the paper [1] it's only ten page long! (Excluding benchmarks and references)

[1]: https://www.microsoft.com/en-us/research/uploads/prod/2019/0...

Post reply on HN