Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spotting and avoiding heap fragmentation in Rust applications

svix.com

Post reply on HN
Print view View original

51–60 of 70 posts

Re: Spotting and avoiding heap fragmentation in Rust applications

#51

Post by harpiaharpyja »

tayo42 wrote: post #42
tasn wrote: post #40

Earlier quoted context omitted.

> 99.999 is miserable to support. how do you even get the granularity to measure, freedom to try anything? i dont think cloud services even provide slas like that for their services. A lot of redundancies and testing. > 1 billion a year would average out to 32 a second. if its 10 billion? 320/second. surprising to see rust for this. gc languages should be able to handle it. This assumes even distribution, though traf…

Oh maybe I missed but I didn't think that info was there. Would have been interesting to see thr extent of fragmentation or what the cause was. 5 9s of uptime means 800ms of down time a day. Is this not in a cloud, a load balancer health check can't react that fast. Vms will get randomly killed if something is wrong.

Yet again, averages don't always represent real life accurately.

Re: Spotting and avoiding heap fragmentation in Rust applications

#52

Post by tayo42 »

harpiaharpyja wrote: post #51
tayo42 wrote: post #42

Earlier quoted context omitted.

Oh maybe I missed but I didn't think that info was there. Would have been interesting to see thr extent of fragmentation or what the cause was. 5 9s of uptime means 800ms of down time a day. Is this not in a cloud, a load balancer health check can't react that fast. Vms will get randomly killed if something is wrong.

Yet again, averages don't always represent real life accurately.

theyre usually close enough, especially for estimating needs. 1 billion/year averages out to 30/second like i said before. You think they're bursting to 1000? maybe i could believe that. 10,000?100,000/second? its unlikely. this is all still scale that is handled by stable, gc languages.

Re: Spotting and avoiding heap fragmentation in Rust applications

#53

Post by tayo42 »

tasn wrote: post #43
tayo42 wrote: post #42

Earlier quoted context omitted.

Oh maybe I missed but I didn't think that info was there. Would have been interesting to see thr extent of fragmentation or what the cause was. 5 9s of uptime means 800ms of down time a day. Is this not in a cloud, a load balancer health check can't react that fast. Vms will get randomly killed if something is wrong.

The SLA is monthly, not daily.

that's still like 30 seconds. <1 of 100,000 requests ever failing. im surprised the company finds the tradeoff of providing that worth the engineering cost to attempt to provide something like that.

Re: Spotting and avoiding heap fragmentation in Rust applications

#55

Post by mr_00ff00 »

solomatov wrote: post #29
munificent wrote: post #25

In some ways Rust is in the worst possible position in terms of language design when it comes to fragmentation. In a completely manually managed language like C or C++, you can handle fragmentation problems yourself by writing your own allocators or doing object pools to reuse previously allocated memory. You have control over fragmentation. In a garbage collected language like Java or C#, the runtime is able to move…

>In a completely manually managed language like C or C++, you can handle fragmentation problems yourself by writing your own allocators or doing object pools to reuse previously allocated memory. You have control over fragmentation. It's done constantly in Rust. Create a vector of items you want to allocate. Reference to them by their id, i.e. int representing them. There're libraries supporting this style of develop…

I assume there might be performance issues using a vector that’s gonna reallocate when capacity is full?

That being said vectors seem like a gift in rust. I feel like it’s the easiest way to get around some ownership rules when you need to work with many pointers. You can just use indices.

Re: Spotting and avoiding heap fragmentation in Rust applications

#56

Post by astrange »

tasn wrote: post #37
astrange wrote: post #34

Earlier quoted context omitted.

Heap fragmentation often comes from allocating objects with different lifetimes at the same time on the same pages. Parsing is a common case of this because you allocate the whole object tree then only keep some of it.

Not in the context of an HTTP server. As we just parse, use it in the request, and then return (freeing all the memory). I think the problem is because we have multiple requests being handled in tandem and Rust doesn't know it's probably better off allocating all of the data together and then freeing this big block. That's what's nice about jemalloc, it has a more generic algorithm for reusing allocated blocks.

Well, it's good as a drop in solution but an arena allocator or malloc zones would really be best.

Re: Spotting and avoiding heap fragmentation in Rust applications

#57

Post by cozzyd »

mr_00ff00 wrote: post #55
solomatov wrote: post #29

Earlier quoted context omitted.

>In a completely manually managed language like C or C++, you can handle fragmentation problems yourself by writing your own allocators or doing object pools to reuse previously allocated memory. You have control over fragmentation. It's done constantly in Rust. Create a vector of items you want to allocate. Reference to them by their id, i.e. int representing them. There're libraries supporting this style of develop…

I assume there might be performance issues using a vector that’s gonna reallocate when capacity is full? That being said vectors seem like a gift in rust. I feel like it’s the easiest way to get around some ownership rules when you need to work with many pointers. You can just use indices.

You can just allocate a large buffer and serialize/deserialize everything at various indices in that buffer

Re: Spotting and avoiding heap fragmentation in Rust applications

#59

Post by gmokki »

mcronce wrote: post #13
MuffinFlavored wrote: post #2

This article would've been a bit cooler if the conclusion wasn't "switch from default allocator to jemalloc" but instead "use jemalloc to prove something is wrong in the default allocator and track down + find a fix for what's wrong in the default allocator" Unless I misunderstood that the default Rust allocator, with high request bodies and concurrency, is always going to suffer unfixable heap fragmentation like dis…

I agree that there could have been a more satisfying conclusion, but it is worth noting that jemalloc isn't a panacea. I've seen issues similar to Svix in both Rust and C++ applications that were heavy on ephemeral allocations, and have fixed it by doing all of the following, depending on the specific process: * Switching from libc malloc to jemalloc * Switching from libc malloc to tcmalloc (dating myself a little bi…

All of the allocators are improved over time (against their own their own goals). It is always important to specify the libc version and version of alternative allocators. Otherwise the results might not be reproducible a year later.

Re: Spotting and avoiding heap fragmentation in Rust applications

#60

Post by mr_00ff00 »

Tuna-Fish wrote: post #23
nikeee wrote: post #15

Shouldn't this also be a much larger problem when deploying rust on embedded systems or in the kernel? How is that solved in these cases? Or is this not a problem at all?

Rust by default uses the platform allocator. Anything that's true of it's allocator performance is also true of C. Also, as the sibling comment said, when you are doing tight embedded, the solution is not to allocate things dynamically.

I’m curious when not allocating things dynamically, does that just get around all memory safety (other than temporal with array bounds checking).

Essentially any language that does array bounds checking would be as safe as rust? Or am I missing anything?

C does not technically do that, so rust is still safer. But zig would be the same in the embedded space as rust? Or is there anything I am missing?

Post reply on HN