Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

All About Libpas, Phil's Super Fast Malloc

github.com

Post reply on HN
Print view View original

81–87 of 87 posts

Re: All About Libpas, Phil's Super Fast Malloc

#81

Post by kookamamie »

pizlonator wrote: post #4
lidavidm wrote: post #3

I wonder how this compares to jemalloc, mimalloc, snmalloc?

I never got a chance to compare it to those, since I was most interested in beating bmalloc. And I mainly wanted to beat it on Safari workloads. I believe bmalloc was previously compared against jemalloc and tcmalloc, also using Safari workloads, and bmalloc was significantly faster at the time.

tcmalloc and mimalloc are the ones I'd be interested seeing comparisons to. The former is the fastest in my case, which is essentially many-core HPC.

Re: All About Libpas, Phil's Super Fast Malloc

#83

Post by mjp41 »

lidavidm wrote: post #3

I wonder how this compares to jemalloc, mimalloc, snmalloc?

libpas is now in mimalloc-bench (https://github.com/daanx/mimalloc-bench). Thanks to Julien Voisin (https://dustri.org), which makes about 20 allocators that can be compared on a collection of workloads.

Re: All About Libpas, Phil's Super Fast Malloc

#84

Post by mattgreenrocks »

pizlonator wrote: post #54
mattgreenrocks wrote: post #38

I admit I'm a sucker for metacircularity, but have to ask: > The bootstrap heap has hacks to allow itself to allocate that array out of itself. Not quite following that, can someone elaborate?

The bootstrap heap has a freelist that is just an array of entries that tell you where the free memory is. That array has to be allocated somewhere. All pas_simple_large_free_heaps allocate that array from the bootstrap heap. Including the bootstrap heap. So, the pas_simple_large_free_heap has hacks that go something like: "if I'm allocating or freeing something and I need to allocate or free the free list, then very…

Ahhhhhh, makes sense now. Thank you!

Re: All About Libpas, Phil's Super Fast Malloc

#85

Post by pizlonator »

rurban wrote: post #82

pas stands for "Phil's Awesome System" though, not Phil's Superfast AS

At some point I started telling people that the acronym stood for a different thing every time I talked about it. There is a place where I called it “Phil’s Awesome System”.

Re: All About Libpas, Phil's Super Fast Malloc

#86

Post by dragontamer »

thesz wrote: post #75
dragontamer wrote: post #44

Earlier quoted context omitted.

Its very easy to beat the general purpose ones if you know your exact use case. Ex: 16-bit pointers is a 65536-sized heap. Assume 8-bytes per element, that's 512KB of space. A bit small, but large enough to so a lot of things. 65536 elements can be represented as a bitmask. The bitmask only takes up 8192-bytes (8KB), which fits inside of 16 AVX512 registers (Intel offers 32x AVX512/ZMM registers btw). Or it fits insi…

512 in AVX512 is the number of bits per register. You are off by factor of 8.

Agreed. Thanks for pointing out the mistake.

Re: All About Libpas, Phil's Super Fast Malloc

#87

Post by titzer »

pizlonator wrote: post #51
ryanianian wrote: post #46

Are there any projects that surgically augment the memory APIs to be more cooperative? Sure `malloc` implementations will make various tradeoffs, but what if the malloc/free APIs were expanded to expose more programmer-intent or cooperative defrag, etc? - Perhaps `malloc` could ask for an intended lifecycle (think GC generation) that could swap arenas or other algo internals. This opens us up to meta-allocators. - Pe…

I've thought about this a lot. I think that overall, malloc/free/new/delete are already so hard to use that if you added more stuff, it would create too much cognitive load for the programmer. The result would be that the hints the programmer gave you would be more wrong than the malloc's best guess. Let me go through these point by point and offer some thoughts. - Perhaps `malloc` could ask for an intended lifecycle…

I'm even more bullish on computers assisting humans in memory allocation than ever. I think we as humans should just write programs and assume an unboundedly intelligent compiler will study our code, determine what if any lifetimes are clear in the code from usage (e.g. basically type-inferring ownership, rather than type-checking it), do trial region allocation, with dynamic profiling and adaptive optimization, and GC the rest with the best collector based on dynamic tuning.

Basically, just write assuming GC, then whereever a hint could possibly exist, throw computers at it. If anything, such hints should live outside the source code and have no semantic impact (i.e. program works fine if you throw it out).

Post reply on HN