Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spotting and avoiding heap fragmentation in Rust applications

svix.com

Post reply on HN
Print view View original

11–20 of 70 posts

Re: Spotting and avoiding heap fragmentation in Rust applications

#11

Post by Hooray_Darakian »

MuffinFlavored wrote: post #7
Hooray_Darakian wrote: post #3

Earlier quoted context omitted.

I'm not sure it's the case that there's something "wrong" with the default allocator, but rather that there are different tradeoffs at play. There's an old but good discussion of the issue here https://github.com/rust-lang/rfcs/blob/master/text/1183-swap...

A sharp increase that never comes back down over time seems more "wrong" than right to me, no?

Without more context I think it's unclear. I know linux tends to avoid freeing memory until/unless the system is near capacity, so this test may be running on a system with low memory pressure.

Re: Spotting and avoiding heap fragmentation in Rust applications

#12

Post by chc »

MuffinFlavored wrote: post #7
Hooray_Darakian wrote: post #3

Earlier quoted context omitted.

I'm not sure it's the case that there's something "wrong" with the default allocator, but rather that there are different tradeoffs at play. There's an old but good discussion of the issue here https://github.com/rust-lang/rfcs/blob/master/text/1183-swap...

A sharp increase that never comes back down over time seems more "wrong" than right to me, no?

It's not good, but only looking at the bad half of a tradeoff will always look bad. Without diving into the glibc source code to understand what it's doing, it is hard to say if this is actually wrong.

Re: Spotting and avoiding heap fragmentation in Rust applications

#13

Post by mcronce »

MuffinFlavored wrote: post #2

This article would've been a bit cooler if the conclusion wasn't "switch from default allocator to jemalloc" but instead "use jemalloc to prove something is wrong in the default allocator and track down + find a fix for what's wrong in the default allocator" Unless I misunderstood that the default Rust allocator, with high request bodies and concurrency, is always going to suffer unfixable heap fragmentation like dis…

I agree that there could have been a more satisfying conclusion, but it is worth noting that jemalloc isn't a panacea. I've seen issues similar to Svix in both Rust and C++ applications that were heavy on ephemeral allocations, and have fixed it by doing all of the following, depending on the specific process:

  * Switching from libc malloc to jemalloc
  * Switching from libc malloc to tcmalloc (dating myself a little bit)
  * Switching from libc malloc to mimalloc
  * Switching from jemalloc to mimalloc
  * Switching from jemalloc to libc malloc
  * Switching from mimalloc to jemalloc
Possibly others; I only want to list cases I'm 100% certain of.

Heap fragmentation is just a reality of some allocation patterns without a GC runtime.

One certainly can (and, in some cases, should) make their application more allocator-friendly, but - aside from some often-low-hanging fruit - this is a time-intensive process involving a bit of, for lack of a better word, arcane knowledge (I should inline all my fields and allocate on the stack as much as possible, right? Yes, well, except ...)

If you already have a halfway decent benchmark suite or workload generator, which you'll want for other purposes anyway, it's often a lot quicker to just try a few other allocators and select the one that handles your workload best.

Re: Spotting and avoiding heap fragmentation in Rust applications

#14

Post by zackangelo »

If you're doing memory-intensive tasks that are clearly bounded (e.g., handling an http request), I wonder if it's worth looking at something like an arena allocator [0] that can free the entire task's memory at once.

[0] https://docs.rs/bumpalo/latest/bumpalo/

Re: Spotting and avoiding heap fragmentation in Rust applications

#16

Post by tasn »

nikeee wrote: post #15

Shouldn't this also be a much larger problem when deploying rust on embedded systems or in the kernel? How is that solved in these cases? Or is this not a problem at all?

Both the kernel and embedded systems manage memory very differently to "normal" applications. The kernel has its own allocation functions, and embedded systems often don't "allocate" but rather just have fixed memory regions they use for things.

Re: Spotting and avoiding heap fragmentation in Rust applications

#17

Post by yakubin »

dgacmu wrote: post #10
cryptonector wrote: post #9

Earlier quoted context omitted.

The failure mode for filesystems is to get slow. Same as for heap allocation.

No - heap fragmentation actually leaves memory unusable by the workload. It may or may not slow down allocation depending on the design of the allocator. (The analogy to file system fragmentation for memory is that when physical pages are allocated in a discontinuous manner, it prevents some optimizations like coalescing them into hugepages, which for some workloads can help with TLB hit rate.)

Filesystem may also run out of inodes, even though there is still plenty of space.

Re: Spotting and avoiding heap fragmentation in Rust applications

#18

Post by moomin »

chc wrote: post #12
MuffinFlavored wrote: post #7

Earlier quoted context omitted.

A sharp increase that never comes back down over time seems more "wrong" than right to me, no?

It's not good, but only looking at the bad half of a tradeoff will always look bad. Without diving into the glibc source code to understand what it's doing, it is hard to say if this is actually wrong.

I’ve recently been doing a lot of work that involves algorithmic design with non-technical people and this is the fun thing. It’s very easy to identify expected scenarios, harder to identify how to achieve them and sometimes actually impossible on e you consider things like “the algorithm doesn’t know what happens next”.

Re: Spotting and avoiding heap fragmentation in Rust applications

#19

Post by nikeee »

tasn wrote: post #16
nikeee wrote: post #15

Shouldn't this also be a much larger problem when deploying rust on embedded systems or in the kernel? How is that solved in these cases? Or is this not a problem at all?

Both the kernel and embedded systems manage memory very differently to "normal" applications. The kernel has its own allocation functions, and embedded systems often don't "allocate" but rather just have fixed memory regions they use for things.

I recall from playing with Arduino that using heap-allocated strings will cause heap fragmentation, which is why they should be avoided. This was C++, doesn't this count for rust as well?

Do kmalloc/kzmalloc do magic to circumvent this problem, similar to jemalloc?

Re: Spotting and avoiding heap fragmentation in Rust applications

#20

Post by jamboca »

"The specific cause for the fragmentation could be any number of things: JSON parsing with serde, something at the framework-level in axum, something deeper in tokio, or even just a quirk of the specific allocator implementation for the given system. Even without knowing the root cause (if there is such a thing) the behavior is observable in our environment and somewhat reproducible in a bare-bones app."

So what is the ultimate cause of the fragmentation in this case? You can blame your allocator implementation, but how do you know it's not your use of the allocator? It feels like you are just slapping jemalloc on to solve this, which I suppose shows that the allocator you were using was causing problems before, but it doesn't really explain how? I suppose what I'm wondering is why specifically the allocator you were using before was causing this fragmentation... isn't that a bigger problem than you're making it sound?

Also, what else can an allocator do other than coalescing free blocks to decrease fragmentation? Does it involve occasional checks to defragment the heap ie moving separated blocks so they are adjacent?

Post reply on HN