Live data from Hacker News

Understanding Memory Management, Part 1: C

educatedguesswork.org

41–50 of 95 posts

Re: Understanding Memory Management, Part 1: C

#41

Just no. address = X length = *X address = address + 1 while length > 0 { address = address + 1 print *address }

Author here. You're quite right that this isn't the thing you would normally do. I'm just trying to help people work through the logic of the system with as few dependencies as possible, hence this (admittedly yucky) piece of pseudocode which isn't really C or Rust or Python or anything...

Re: Understanding Memory Management, Part 1: C

#42

This isn't proper usage of realloc: lines = realloc(lines, (num_lines + 1) * sizeof(char *)); In case it cannot service the reallocation and returns NULL, it will overwrite "lines" with NULL, but the memory that "lines" referred to is still there and needs to be either freed or used. The proper way to call it would be: tmp = realloc(lines, (num_lines + 1) * sizeof(char *)); if (tmp == NULL) { free(lines); lines = NUL…

Author here.

Thanks for the flag. As you have probably noticed, I just abort the program a few lines below on realloc failure, so this doesn't leak so much as crash. However, this is a nice example of how fiddly C memory management is.

Re: Understanding Memory Management, Part 1: C

#43
post #27

This isn't proper usage of realloc: lines = realloc(lines, (num_lines + 1) * sizeof(char *)); In case it cannot service the reallocation and returns NULL, it will overwrite "lines" with NULL, but the memory that "lines" referred to is still there and needs to be either freed or used. The proper way to call it would be: tmp = realloc(lines, (num_lines + 1) * sizeof(char *)); if (tmp == NULL) { free(lines); lines = NUL…

There's another bug, related to performance - this involves a quadratic amount of memory copying unless your environment can arrange for zero-copy.

Author here. Quite so. See footnote 3:https://educatedguesswork.org/posts/memory-management-1/#fn3

"If you know you're going to be doing a lot of reallocation like this, many people will themselves overallocate, for instance by doubling the size of the buffer every time they are asked for more space than is available, thus reducing the number of times they need to actually reallocate. I've avoided this kind of trickery to keep this example simple."

Re: Understanding Memory Management, Part 1: C

#44
post #40
post #12

Avoid as much as you can the C standard lib allocator, go directly to mmap system call with your own allocator if you know you won't use CPU without a MMU. If you write a library, let the user code install its own allocator.

Sure, if one doesn't care about portable code.

What modern OS doesn't have the equivalent of mmap? Just som #ifdefs. I didn't know I'd ever hear "use malloc, because it's portable".

sylware is pretty much right anyway. Try to avoid malloc, or write smaller allocators on top of malloc.

Re: Understanding Memory Management, Part 1: C

#45

Earlier quoted context omitted.

> which is just getting things done end-to-end as fast as possible, not careful at every step that we have no memory errors. One horrible but fun thing a former professor of mine pointed out: If your program isn't going to live long, then you never have to deallocate memory. Once it exits, the OS will happily clean it up for you. This works in C or perhaps lazy GC languages, but for stateful objects where destructors…

There is this old chestnut about “null garbage collectors”: https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98... > This sparked an interesting memory for me. I was once working with a customer who was producing on-board software for a missile. In my analysis of the code, I pointed out that they had a number of problems with storage leaks. Imagine my surprise when the customers chief software engineer said…

Rapid disassembly as GC. Love it.

Have you heard the related story about the patriot missile system?

https://www.cs.unc.edu/~smp/COMP205/LECTURES/ERROR/lec23/nod...

Not a GC issue, but fun software bug.

Re: Understanding Memory Management, Part 1: C

#46
post #38

Earlier quoted context omitted.

> which is just getting things done end-to-end as fast as possible, not careful at every step that we have no memory errors. One horrible but fun thing a former professor of mine pointed out: If your program isn't going to live long, then you never have to deallocate memory. Once it exits, the OS will happily clean it up for you. This works in C or perhaps lazy GC languages, but for stateful objects where destructors…

The wonders of corrupted data, stale advisory locks and UNIX IPC leftovers, because they weren't properly flushed, or closed before process termination.

I'll narrow my scope more explicitly:

close(x) is not memory management - not at the user level. This should be done.

free(p) has no O/S side effects like this in C - this can be not-done if you don't malloc all your memory.

You can get away with not de-allocating program memory, but (as mentioned), that has nothing to do with freeing Os/ kernel / networking resources in C.

Re: Understanding Memory Management, Part 1: C

#47
post #40

Earlier quoted context omitted.

Sure, if one doesn't care about portable code.

What modern OS doesn't have the equivalent of mmap? Just som #ifdefs. I didn't know I'd ever hear "use malloc, because it's portable". sylware is pretty much right anyway. Try to avoid malloc, or write smaller allocators on top of malloc.

Why bother with some #ifdefs when the ISO C standard library already does the job?

Re: Understanding Memory Management, Part 1: C

#49
post #27

Earlier quoted context omitted.

There's another bug, related to performance - this involves a quadratic amount of memory copying unless your environment can arrange for zero-copy.

Surely that's only the case if realloc() actually resizes and copies on every call? Which it normally doesn't? I thought that most implementations of realloc() would often "round up" internally to a larger size allocation, maybe power-of-two, maybe page size, or something? So if you ask for 20 bytes, the internal bookkeeping sets aside 32, or 4096, or whatever. And then if you realloc to 24 bytes, realloc will just n…

Some implementations might round up to encourage reuse:

* memory-checking allocators never do.

* purely-size-based allocators always do.

* extent-based allocators try to, but this easily fails if you're doing two interleaving allocations.

* the mmap fallback does only if allowing the kernel to choose addresses rather than keeping virtual addresses together, unless you happen to be on a kernel that allows not leaving a hole

Given that there's approximately zero overhead to do it right, just do it right (you don't need to store capacity, just compute it deterministically from the size).

Re: Understanding Memory Management, Part 1: C

#50
post #38

Earlier quoted context omitted.

The wonders of corrupted data, stale advisory locks and UNIX IPC leftovers, because they weren't properly flushed, or closed before process termination.

I'll narrow my scope more explicitly: close(x) is not memory management - not at the user level. This should be done. free(p) has no O/S side effects like this in C - this can be not-done if you don't malloc all your memory. You can get away with not de-allocating program memory, but (as mentioned), that has nothing to do with freeing Os/ kernel / networking resources in C.

Most kernel resources are fairly well behaved, as they will automatically decrement their refcount when a process exits. Even mutexes have a "robust" flag for this exact reason. Programs which rely on destructors or any other form or orderly exit are always brittle and should be rewritten to use atomic operations.
Post reply on HN