Why does musl make my Rust code so slow?

#1

Post by andygrove » Tue, May 05, 2020, 2:23 PM UTC

Why does musl make my Rust code so slow?
andygrove.io

Re: Why does musl make my Rust code so slow?

#2

Post by pedrocr » Tue, May 05, 2020, 2:42 PM UTC

Swapping out the allocator for jemalloc would be my first try. It's easy to do and often results in better performance. 30x requires some kind of pathological case though.

Re: Why does musl make my Rust code so slow?

#3

Post by iou » Tue, May 05, 2020, 2:42 PM UTC

Swap out the allocator https://users.rust-lang.org/t/optimizing-rust-binaries-obser...

Re: Why does musl make my Rust code so slow?

#4

Post by BubRoss » Tue, May 05, 2020, 2:43 PM UTC

This actually someone asking and not an investigation and explanation. There isn't even a lot of due diligence to figuring it out - no profiling or resource usage other than CPUs. Also it is musl combined with docker causing a 30x slowdown.

If something is running 30x slower from linking in a different libc, I'm guessing it should not be that difficult to narrow down the cause at least a little bit.

Re: Why does musl make my Rust code so slow?

#5

Post by termie » Tue, May 05, 2020, 2:43 PM UTC

run a perf trace on both and see what jumps out

Re: Why does musl make my Rust code so slow?

#6

Post by pjc50 » Tue, May 05, 2020, 2:44 PM UTC

.. where's the profile output?

Re: Why does musl make my Rust code so slow?

#7

Post by asveikau » Tue, May 05, 2020, 2:46 PM UTC

iou wrote: post #3

Swap out the allocator https://users.rust-lang.org/t/optimizing-rust-binaries-obser...

Without familiarity with rust, I wasn't sure what they meant by "system allocator". Apparently that means libc's malloc. (Or HeapAlloc on Windows)

So I guess they statically link jemalloc but can optionally use libc malloc.

Re: Why does musl make my Rust code so slow?

#8

Post by underdeserver » Tue, May 05, 2020, 2:47 PM UTC

...Not the Intel guy, if anyone else had to pause for a second.

Re: Why does musl make my Rust code so slow?

#9

Post by curtis3389 » Tue, May 05, 2020, 2:53 PM UTC

https://lmgtfy.com/?q=musl+libc+performance

I'm fairly sure musl is used because it's a smaller alternative to glibc, so it's better suited to Docker containers. It's goals are security, not performance.

Re: Why does musl make my Rust code so slow?

#10

Post by mrits » Tue, May 05, 2020, 2:54 PM UTC

pedrocr wrote: post #2

Swapping out the allocator for jemalloc would be my first try. It's easy to do and often results in better performance. 30x requires some kind of pathological case though.

In a commercial product I worked on I went against the vendors advice to try out jemalloc. It took a 100GB memory hold (that took 48 hours to happen) to staying steady at around 2-4GB and only peaking at 100GB for a few seconds a day.

Same exact code but just swapped out the jemalloc at the command line.