I'm messing around with implementing a parallel runtime system right now^, and I also had to implement a shared-nothing allocator. I ended up making each thread have a Chase-Lev Deque of 2MB pages. If you want to allocate a page, you do the usual pop-or-try-to-steal thing, and if that doesn't work, THEN you do the mmap to get another 2MB page. I was going to benchmark it and write a post about it, but I never got aro…
Snmalloc: A Message Passing Allocator
21–30 of 30 posts
Re: Snmalloc: A Message Passing Allocator
#22I'm messing around with implementing a parallel runtime system right now^, and I also had to implement a shared-nothing allocator. I ended up making each thread have a Chase-Lev Deque of 2MB pages. If you want to allocate a page, you do the usual pop-or-try-to-steal thing, and if that doesn't work, THEN you do the mmap to get another 2MB page. I was going to benchmark it and write a post about it, but I never got aro…
Re: Snmalloc: A Message Passing Allocator
#23The real question is: how does this deal with freeing allocations from a thread that exited ? The implications of thread lifetime seems to be one of the biggest differences between existing allocators.
We have not had complaints about this heuristic, yet. We have a solution that involves marking the remote free queue as sleeping, and a thread that sends to it and observes "sleeping" would then have to do some work for that "sleeping" allocator. This is fairly easy, but I haven't had time to implement it.
Re: Snmalloc: A Message Passing Allocator
#24The real question is: how does this deal with freeing allocations from a thread that exited ? The implications of thread lifetime seems to be one of the biggest differences between existing allocators.
Hi, I am one of the authors of snmalloc. There is a pool of allocators. When a thread exits it returns it to the pool. When a thread is created it first checks the pool for an allocator and uses that in preference to creating a new one. If your application has generally a uniform number of threads over time, then this works well. If you have a spike of threads at start up, then are purely single threaded after that,…
> Allocations on one thread are freed by a different thread
I can imagine one use-case for this: a task that is scheduled from and executed by a work-stealing thread-pool can allocate memory in one thread but by design there's no guarantee that the memory will be necessarily freed from that exact thread. Would that be a good use-case for snmalloc?
> Deallocations occur in large batches
This sounds much like a bump allocator use-case but which can do this exact thing by calling a single munmap(addr, len) and unmap multiple allocations all at once.
Re: Snmalloc: A Message Passing Allocator
#25Earlier quoted context omitted.
Hi, I am one of the authors of snmalloc. There is a pool of allocators. When a thread exits it returns it to the pool. When a thread is created it first checks the pool for an allocator and uses that in preference to creating a new one. If your application has generally a uniform number of threads over time, then this works well. If you have a spike of threads at start up, then are purely single threaded after that,…
https://github.com/microsoft/snmalloc#snmalloc mentions two biggest motivations as: > Allocations on one thread are freed by a different thread I can imagine one use-case for this: a task that is scheduled from and executed by a work-stealing thread-pool can allocate memory in one thread but by design there's no guarantee that the memory will be necessarily freed from that exact thread. Would that be a good use-case…
Yes. A work stealing runtime is a perfect use case for this. Also, cases where you might have a dedicated IO ingress thread that allocates work items, these get picked up by worker threads, and then sent to a dedicated IO egress thread. The ingress thread does a lot of allocation, the egress thread does a lot of deallocation, and the workers do a bit of both. The deallocations typically flow the opposite direction to the work flow, and this works extremely well with snmalloc.
> This sounds much like a bump allocator use-case but which can do this exact thing by calling a single munmap(addr, len) and unmap multiple allocations all at once.
Where suitable bump allocated arena management will be fast, but some times the lifetime are more complex. Certain algorithms for memory management can often release a lot of memory in a batch that is not necessarily allocated in batch, i.e. Lock-free epoch based memory reclamation, or Linux RCU, both batch deciding memory is no longer required. There are also application level things like periodic flushes of a cache/log that can cause batching of deallocations, but are not well suited to bump allocators.
Re: Snmalloc: A Message Passing Allocator
#26The real question is: how does this deal with freeing allocations from a thread that exited ? The implications of thread lifetime seems to be one of the biggest differences between existing allocators.
I'm not a contributor to snmalloc or affiliated in any way. I'm going to reimplement snmalloc in C for my own code. I'm going to avoid that problem by using structured concurrency. Under my system, threads only get data from threads that are ancestors. (There will be a way of getting data from a non-ancestor, but it will effectively keep the allocating thread alive until the using thread is done.) The reason I'm goin…
Re: Snmalloc: A Message Passing Allocator
#27Earlier quoted context omitted.
I'm not a contributor to snmalloc or affiliated in any way. I'm going to reimplement snmalloc in C for my own code. I'm going to avoid that problem by using structured concurrency. Under my system, threads only get data from threads that are ancestors. (There will be a way of getting data from a non-ancestor, but it will effectively keep the allocating thread alive until the using thread is done.) The reason I'm goin…
Have you considered using mimalloc? I gather it has a similar design to snmalloc, but mimalloc is written in C.
Plus, there are some changes I will make to make my snmalloc/mimalloc clone more friendly to structured concurrency. Nothing major, stuff to mostly deal with realloc().
Re: Snmalloc: A Message Passing Allocator
#28Earlier quoted context omitted.
https://github.com/microsoft/snmalloc/blob/main/docs/securit...
While reviewing that doc, I also came across this. Seems very interesting -- I did not know this was possible: > Some architectures, such as CHERI (including Arm's Morello), explicitly consider pointer provenance and bounds in addition to their target addresses. Adding these considerations to the architecture enables software to constrain uses of particular pointers in ways that are not available with traditional pro…
It's great! The CHERI team at U. Cambridge has recently released their initial performance characterization of Morello, Arm's experimental ARMv8 w/ CHERI: https://ctsrd-cheri.github.io/morello-early-performance-resu... . The major take-away there is a little buried, but is:
> The above 1.8% to 3.0% is our current best estimate of the geometric mean overhead that would be incurred for a future optimized design
That seems to be well within people's tolerance for security features, especially as we think having CHERI would also allow us to turn off, and so stop paying for, some existing mitigations.
While there's a wealth of stuff to read about CHERI (https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/cheri...), if you're new to it and want something more presentation flavored than text, you might enjoy my talk from HOPE 2022: https://www.youtube.com/watch?v=dH7QUdXeVrI
Re: Snmalloc: A Message Passing Allocator
#29I'm messing around with implementing a parallel runtime system right now^, and I also had to implement a shared-nothing allocator. I ended up making each thread have a Chase-Lev Deque of 2MB pages. If you want to allocate a page, you do the usual pop-or-try-to-steal thing, and if that doesn't work, THEN you do the mmap to get another 2MB page. I was going to benchmark it and write a post about it, but I never got aro…
If you're backing up your memory through pages stored in N work-stealing queues (chase-lev deque in this case), doesn't that contradict the shared-nothing architecture? When queue is empty, I understand your allocator will try to steal a page from another queue before it asks an OS for a new 2MB page - and that essentially means synchronization and thus sharing. Also, I think this approach is going to result with bad…
Sort of? The stealing happens very rarely, only when the thread doesn’t have a page. It’s done using relaxed atomics, so synchronization is pretty mild in this case.
> bad data locality
This will be relevant when I make it NUMA-aware, but generally speaking this shouldn’t matter as each page is much bigger than a core’s L2 cache anyway. And also memory living in the deque is freed, I don’t think there’s generally expectations on locality of memory returned to you from the allocator.
Re: Snmalloc: A Message Passing Allocator
#30I'm messing around with implementing a parallel runtime system right now^, and I also had to implement a shared-nothing allocator. I ended up making each thread have a Chase-Lev Deque of 2MB pages. If you want to allocate a page, you do the usual pop-or-try-to-steal thing, and if that doesn't work, THEN you do the mmap to get another 2MB page. I was going to benchmark it and write a post about it, but I never got aro…
This is very interesting! I am especially interested in how you share memory between threads in your parallel runtime - what thread safety approach do you use? I've been thinking about static memory as permanent regions that are permanent fixtures and the program flows through them and there must be backpressure when buffers get full. I just wrote a nonblocking multithreaded barrier with a lock free algorithm.