Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Identifying Rust's collect: >() memory leak footgun

blog.polybdenum.com

Post reply on HN
Print view View original

41–50 of 129 posts

Re: Identifying Rust's collect:<Vec<_>>() memory leak footgun

#41

Post by LegionMammal978 »

vlovich123 wrote: post #21
cmrdporcupine wrote: post #18

Earlier quoted context omitted.

What we need is a page-based Vec that mmaps (anon) for the storage but leaves the unused portions zero-bytes and therefore not part of RSS until actually required. (And when clearing/shrinking sections, madvise DONTNEED the pages). That is, the vec could expand to areas much larger than the actual used size, but this would have no effect on process RSS until those pages get dirtied with actual data.

That can be not a great idea for subtle reasons even though it does seem like a better design at first. When you do the madvise, based on the contact of the API, the kernel has to do a TLB shoot down which is really expensive. And not just “expensive for my process” but expensive in terms of a significant slowdown for entire machine. Honestly you could probably DDOS a machine if you could reliably trigger that shootd…

Is MADV_FREE on Linux any better in this regard? It's what the libc allocators tend to use, if I recall correctly.

Re: Identifying Rust's collect:<Vec<_>>() memory leak footgun

#42

Post by vlovich123 »

cmrdporcupine wrote: post #23
vlovich123 wrote: post #21

Earlier quoted context omitted.

That can be not a great idea for subtle reasons even though it does seem like a better design at first. When you do the madvise, based on the contact of the API, the kernel has to do a TLB shoot down which is really expensive. And not just “expensive for my process” but expensive in terms of a significant slowdown for entire machine. Honestly you could probably DDOS a machine if you could reliably trigger that shootd…

Yes, well, I think you're right it's a lot about the contract of the API and the expectation of the user. I'd certainly want intelligence (either by the user or by the framework) in how frequently you release pages back to the OS. But the DONTNEED is really not the core of the value. It's being able to create vectors that don't fragment as badly. And yes, you're right a decent allocator could help with this, and what…

I don’t follow. Fragmentation wasn’t the problem here - it’s that an optimization to trying to reuse the underlying allocation resulted in a severe over-allocation in a specific path. That would happen even with your idea unless you use DONTNEED to release all the unused pages.

I think fragmentation is an over focused on problem when in practice it’s rarely the problem. It’s also a problem that can be solved by an application reset and making sure to periodically reset your running application in a way that doesn’t result in severe disruption is a better practice that works around many more issues than just fragmentation.

Re: Identifying Rust's collect:<Vec<_>>() memory leak footgun

#43

Post by loeg »

This is a pretty surprising behavior. Reusing the allocation without shrinking when the resulting capacity will be within 1.0-2.0x the length: seems reasonable, not super surprising. Reusing the allocation without shrinking when the resulting capacity will be a large multiple of the (known) length: pretty surprising! My intuition is that this is too surprising (at capacity >2) to be worth the possible optimization but probably still worth doing at capacity small multiples of length.

Re: Identifying Rust's collect:<Vec<_>>() memory leak footgun

#44

Post by vlovich123 »

WirelessGigabit wrote: post #26

I don't think it's a memory leak. It's re-using the same space of the underling array. It's allocated. Dropping the Vec will release the memory. In v1 you put in 128 containers is with each 1024 boxes. Then v2 is taking out the first box out of each container, tossing the container and putting the box at the space where the container was, packing them. The fact that capacity doubles when you remove the as u8 is... no…

It seems like you have a better grasp on the problem than I. I understand why capacity in terms of number of elements doubles. But why did he end up using 200x more memory? Is it that the transient memory used the 18GiB and then he reduced it to vector of elements that were 200x smaller? Or is it that there’s some other problem that when you collect, it’s reallocating 2x using the wrong type to do the computation?

Re: Identifying Rust's collect:<Vec<_>>() memory leak footgun

#45

Post by phkahler »

binary132 wrote: post #37
phkahler wrote: post #28

Earlier quoted context omitted.

>> Programmers’ expectations are not codified and do not need to be respected That's pretty negative attitude - my immediate gut response was "and neither do yours". But Rust isn't an ISO standard and is still in development. Even if we do think in those terms, people developing a standard have IMHO an obligation to the people who will be using the standard.

To be a bit pedantic, the people using the C++ standard are mostly compiler engineers.

Everyone writing C++ is following the standard. The people that often reference it directly are the compiler people. But if we are going to try and be stupid about it we can ignore the compiler engineers opinions too - they're just implementing an arbitrary specification that won't impact anyone but them anyway right?

Re: Identifying Rust's collect:<Vec<_>>() memory leak footgun

#46

Post by [deleted] »

WirelessGigabit wrote: post #26

I don't think it's a memory leak. It's re-using the same space of the underling array. It's allocated. Dropping the Vec will release the memory. In v1 you put in 128 containers is with each 1024 boxes. Then v2 is taking out the first box out of each container, tossing the container and putting the box at the space where the container was, packing them. The fact that capacity doubles when you remove the as u8 is... no…

[deleted]

Re: Identifying Rust's collect:<Vec<_>>() memory leak footgun

#47

Post by paulddraper »

WirelessGigabit wrote: post #26

I don't think it's a memory leak. It's re-using the same space of the underling array. It's allocated. Dropping the Vec will release the memory. In v1 you put in 128 containers is with each 1024 boxes. Then v2 is taking out the first box out of each container, tossing the container and putting the box at the space where the container was, packing them. The fact that capacity doubles when you remove the as u8 is... no…

No, if that's all it was, the excess memory usage would be 2x.

But it's 200x.

Right?

Re: Identifying Rust's collect:<Vec<_>>() memory leak footgun

#48

Post by vlovich123 »

LegionMammal978 wrote: post #41
vlovich123 wrote: post #21

Earlier quoted context omitted.

That can be not a great idea for subtle reasons even though it does seem like a better design at first. When you do the madvise, based on the contact of the API, the kernel has to do a TLB shoot down which is really expensive. And not just “expensive for my process” but expensive in terms of a significant slowdown for entire machine. Honestly you could probably DDOS a machine if you could reliably trigger that shootd…

Is MADV_FREE on Linux any better in this regard? It's what the libc allocators tend to use, if I recall correctly.

It would have to be the same problem because there’s no way to free memory without a shootdown AFAIK. As I describe elsewhere, I don’t think there’s any way to free memory to the OS without one as otherwise you can run into race conditions where another process allocating memory gets access to the memory being freed in another process through a stale TLB entry.

Here’s a discussion [1] of a hypothetical lazy_munmap option that would initiate the unmap without an immediate shoot down (i.e. the CPUs would lazily evict from the TLB when they notice the request) but no such option exists. It’s also not immediately clear such a hypothetical API could exist on current CPU architectures or if it would require new HW support as I don’t have enough hands-on expertise at that level. Anyway, [1] is an interesting discussion of this idea and the note about huge pages making these even less valuable is interesting to keep in mind.

[1] https://news.ycombinator.com/item?id=23216590

Re: Identifying Rust's collect:<Vec<_>>() memory leak footgun

#50

Post by paulddraper »

pif wrote: post #3

> It’s also an illustration of how an optimization in one place can lead to bugs downstream by violating programmers’ expectations. This touched upon a pet peeve of mine for its resemblance with all the talk about undefined behaviour in C and C++. Programmers’ expectations are not codified and do not need to be respected: international standards do.

> Programmers’ expectations are not codified and do not need to be respected: international standards do.

What if...stick with me...the standards sought to codify and respect programmers' expectations?

https://en.wikipedia.org/wiki/Principle_of_least_astonishmen...

Post reply on HN