Live data from Hacker News

A single-file C allocator with explicit heaps and tuning knobs

github.com

1–10 of 48 posts

Re: A single-file C allocator with explicit heaps and tuning knobs

#2
I wrote this because I wanted more explicit control over heaps when building different subsystems in C. Standard options like jemalloc and mimalloc are incredibly fast, but they act as black boxes. You can't easily cap a parser's memory at 256MB or wipe it all out in one go without writing a custom pool allocator.

Spaces takes a different approach. It uses 64KB-aligned slabs, and the metadata lookup is just a pointer mask (ptr & ~0xFFFF).

The trade-off is that every free() incurs an L1 cache miss to read the slab header, and there is a 64KB virtual memory floor per slab. But in exchange, you get zero-external-metadata regions, instant teardown of massive structures like ASTs, and performance that surprisingly keeps up with jemalloc on cross-thread workloads (I included the mimalloc-bench scripts in the repo).

It's Linux x86-64 only right now. I'm curious if systems folks think this chunk API is a pragmatic middle ground for memory management, or if the cache-miss penalty on free() makes the pointer-masking approach a dead end for general use.

Re: A single-file C allocator with explicit heaps and tuning knobs

#4
post #2

I wrote this because I wanted more explicit control over heaps when building different subsystems in C. Standard options like jemalloc and mimalloc are incredibly fast, but they act as black boxes. You can't easily cap a parser's memory at 256MB or wipe it all out in one go without writing a custom pool allocator. Spaces takes a different approach. It uses 64KB-aligned slabs, and the metadata lookup is just a pointer…

When dealing with memory in C defaulting to malloc or some opaque structure behind it is unless you just want to allocate and forget it for some one off program that frees memory on proc exit seems bad to me now. For any kind of sophisticated system or module you almost always want to write your own variety of slab, arena, pool, bump whatever it may be allocator.

Re: A single-file C allocator with explicit heaps and tuning knobs

#7
post #5

There's a single commit in the whole repository. Was this AI generated?

You still have things like git squash etc.

That doesn't make any sense. There's 10,000+ lines of code. There shouldn't be a single commit "Initial commit". I'm fine with squashing some commits and creating a clean history, but this isn't a clean history it's obfuscated.

Re: A single-file C allocator with explicit heaps and tuning knobs

#8
What is the reason for the weird `{ code };` blocks everywhere and is the below code machine generated?

```c ((PageSize) (chunk->pageSize - ((PageSize) ((PageSize) ((PageSize) (sizeof(Page) + (sizeof(struct _Block))) + (PageSize) ((sizeof(double)) - 1u)) & ((PageSize) (~((PageSize) ((sizeof(double)) - 1u)))))) - ((PageSize) ((PageSize) ((PageSize) ((sizeof(FreeBlock) + sizeof(PageSize))) + (PageSize) (((((sizeof(double)) > (4)) ? (sizeof(double)) : (4))) - ```

Re: A single-file C allocator with explicit heaps and tuning knobs

#9
post #5

Earlier quoted context omitted.

You still have things like git squash etc.

That doesn't make any sense. There's 10,000+ lines of code. There shouldn't be a single commit "Initial commit". I'm fine with squashing some commits and creating a clean history, but this isn't a clean history it's obfuscated.

I also do this. Lots of weird commit messages because fuck that, I'm busy. Commits that are just there to put some stuff aside, things like that. I don't owe it to anyone to show how messy my kitchen is.

Re: A single-file C allocator with explicit heaps and tuning knobs

#10
post #9

Earlier quoted context omitted.

That doesn't make any sense. There's 10,000+ lines of code. There shouldn't be a single commit "Initial commit". I'm fine with squashing some commits and creating a clean history, but this isn't a clean history it's obfuscated.

I also do this. Lots of weird commit messages because fuck that, I'm busy. Commits that are just there to put some stuff aside, things like that. I don't owe it to anyone to show how messy my kitchen is.

On the other hand, others don’t have to adopt, use or like your stuff which would be the reasons to publish it.

One big commit definitely doesn’t help with creating confidence in this project.

Post reply on HN