Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory Allocation

samwho.dev

Post reply on HN
Print view View original

131–140 of 182 posts

Re: Memory Allocation

#131

Post by eschneider »

bambax wrote: post #112

Excellent, excellent article! I have a question though. > Couldn't we rearrange the memory to get a block of 6 contiguous bytes? Some sort of defragmentation process? > Sadly not. Remember earlier we talked about how the return value of malloc is the address of a byte in memory? Moving allocations won't change the pointers we have already returned from malloc. We would change the value those pointers are pointed at,…

Early MacOS did this with handles (basically pointers to pointers.) You'd lock them to read/write and when they were unlocked, the OS was free to move the underlying memory and change the pointer in the handle.

Re: Memory Allocation

#132

Post by bee_rider »

alex7734 wrote: post #104
meekaaku wrote: post #86

Earlier quoted context omitted.

You mean a bug this deep took one day to debug?

Yes and no, it took me from 8am to 3am once we decided it needed to get fixed but really it sat on the app for years, it only happened on a background process that sent print jobs on a timer, since it used Windows GDI to compose the image we sent to the printer it was affected (our "frontend" should've been affected too but never was, I guess because it had a different memory usage pattern). We just had it restart it…

Owch, I’m guessing that wasn’t -5 hours of debugging.

Re: Memory Allocation

#134

Post by the-smug-one »

winrid wrote: post #130

I was having some fun recently just seeing how fast we can allocate large chunks of memory. There's something refreshing about firing up a C (or maybe now Zig, in my case) compiler and allocating a gigabyte of memory, and seeing that your process is using exactly 1GB.

> I was having some fun recently just seeing how fast we can allocate large chunks of memory.

How did you measure that? What was fastest? I'd imagine sbrk is faster than mmap, but I haven't checked. If you're on Linux, then doing a virtual memory allocation will not cause the memory to have physical representation. Did you populate the pages through looping? Or did you use MAP_POPULATE?

I saw that madvise was going to get MADV_POPULATE_(READ|WRITE) in the Linux kernel mailing lists back in 2021, but I haven't seen it on the kernels that I use (5.4, 5.15)

Re: Memory Allocation

#137

Post by jacobsenscott »

secondcoming wrote: post #26
samsquire wrote: post #7

Thank you for this, this is helpful. I wrote a JIT compiler and I didn't bother calling free much, I just let the operating system free up all allocated memory. I got into this situation often: return_struct = do_something(mystruct); return_struct->inner_struct = malloc(sizeof(struct my_inner_struct)); Now, who owns inner_struct? Who is responsible for freeing it? Do I free it when I assign to it? I feel this ownersh…

> Now, who owns inner_struct? return_struct does since it is the only thing that knows the address. > Who is responsible for freeing it? return_struct is, unless you hand that responsibility over to something else. > Do I free it when I assign to it? Yes, unless you want leaks. > I think if we could visualise memory as cars on a road, we would see obvious traffic jams. That visualisation is helpful for threads, where…

A struct can't own something - it isn't a class with a destructor or anything. So it isn't quite so obvious. There are only two lines of code here, but the implication is a function is running that code and then returning `return_struct`, which might get passed around to more functions, and even returned further up the call stack. Somewhere there needs to be code that knows "hey - nobody else is using return_struct, and by the way you need to free return_struct->inner_struct before freeing return_struct.

Re: Memory Allocation

#139

Post by jacobsenscott »

davidgrenier wrote: post #21
samwho wrote: post #17

Earlier quoted context omitted.

Well shit. I think you're right.

Oh another thing, I'm not a fan of the premise: "As a general-purpose memory allocator, though, we can't get away with having no free implementation." I have a belief that the future of software are short-lived programs that never free memory. Programs allocate and terminate. Short-lived program communicate with each other via blocking CSP-style channels (see Reppy's Concurrent Programming in ML). If you could also e…

I suspect this would cause the same performance issues as a long running application that constantly malloc'd and free'd memory? Many application runtimes allocate but don't free memory - they just reuse it internally - for this reason. For example in a ruby application you'll see memory usage climb after boot, and eventually level off when it has all it will need for its lifetime, but never go down.
Post reply on HN