Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding Memory Management, Part 1: C

educatedguesswork.org

Post reply on HN
Print view View original

81–90 of 95 posts

Re: Understanding Memory Management, Part 1: C

#81

Post by dataflow »

returningfory2 wrote: post #80
dataflow wrote: post #79

Earlier quoted context omitted.

> This is an article introducing people to memory management, targeted at beginners I realize, and that's what makes it even worse . First impressions have a heck of a stronger effect than 10th impressions. Beginners need to learn the right way in the beginning, not the wrong way. Whenever did "safety first" stop being a thing? This is like like skipping any mention of goggles when teaching chemistry or woodworking f…

The code in the article is not wrong. It is not unsafe. The author explicitly handles the OOM case correctly. It is true that there are more optimal ways to do it if you do have an OOM handling strategy. And no, you're not supposed to teach your students the best way to do things at the start. That's not how teaching works. You start with the simpler (but still correct) way, and then work towards the best way. This i…

> The code in the article is not wrong. It is not unsafe. The author explicitly handles the OOM case correctly.

And hence we circle back to what I just wrote above: you're confusing the code with the program that it compiles to. Because the code isn't there solely for the purpose of being compiled into a program, it's also serving as a stepping stone for other things (learning, modification, whatever). https://news.ycombinator.com/item?id=42733611

If it helps to phrase it differently: the code might be "compile-safe", but not "modification-safe" or "learning-safe".

Re: Understanding Memory Management, Part 1: C

#83

Post by dundarious »

tptacek wrote: post #63
bluetomcat wrote: post #6

This isn't proper usage of realloc: lines = realloc(lines, (num_lines + 1) * sizeof(char *)); In case it cannot service the reallocation and returns NULL, it will overwrite "lines" with NULL, but the memory that "lines" referred to is still there and needs to be either freed or used. The proper way to call it would be: tmp = realloc(lines, (num_lines + 1) * sizeof(char *)); if (tmp == NULL) { free(lines); lines = NUL…

I was looking for a place to hang this comment and here's as good as any: the right way to handle this problem in most C code is to rig malloc, realloc, and strdup up to explode when they'd return NULL. Proper error handling of a true out-of-memory condition is pretty treacherous, so most of the manual error handling stuff you see on things like realloc and malloc are really just performative. In an application setti…

After using this malloc-auto-abort() style for many many years, I've come to believe that if only for the better error handling properties, manual memory management should primarily be done via explicit up front arena allocation using OS API's like mmap/VirtualAlloc, then a bump allocator within the arena.

It helps in the vast amount of cases where sensible memory bounds are known or can be inferred, and it means that all system memory allocation errors* can be dealt with up front with proper error handling (including perhaps running in a more restrictive mode with less memory), and then all application memory allocation errors (running out of space in the arena) can be auto-abort() as before (and be treated as bugs). The other huge benefit is that there is no free() logic for incremental allocations within the arena, you just munmap/VirtualFree the arena in its entirety when done.

Of course, there are cases where there are no sensible memory bounds (in space or perhaps in time) and where this method is not appropriate without significant modification.

*modulo Linux's overcommit... which is a huge caveat

Re: Understanding Memory Management, Part 1: C

#84

Post by witrak »

ekr____ wrote: post #42
bluetomcat wrote: post #6

This isn't proper usage of realloc: lines = realloc(lines, (num_lines + 1) * sizeof(char *)); In case it cannot service the reallocation and returns NULL, it will overwrite "lines" with NULL, but the memory that "lines" referred to is still there and needs to be either freed or used. The proper way to call it would be: tmp = realloc(lines, (num_lines + 1) * sizeof(char *)); if (tmp == NULL) { free(lines); lines = NUL…

Author here. Thanks for the flag. As you have probably noticed, I just abort the program a few lines below on realloc failure, so this doesn't leak so much as crash. However, this is a nice example of how fiddly C memory management is.

Taking into account how thoroughly you explain all the intricate details of memory handling it's strange that in the example you haven't clearly commented on the fact of oversimplification of handling unsuccessful allocation (leading to the potentially risky situation).

To say that "this is a nice example of how fiddly C memory management is" in the discussion is a bit too little - perhaps intended readers of the article would prefer an explicit warning there, just to be aware that they shouldn't forget to abort the program as you do.

Re: Understanding Memory Management, Part 1: C

#85

Post by tptacek »

dundarious wrote: post #83
tptacek wrote: post #63

Earlier quoted context omitted.

I was looking for a place to hang this comment and here's as good as any: the right way to handle this problem in most C code is to rig malloc, realloc, and strdup up to explode when they'd return NULL. Proper error handling of a true out-of-memory condition is pretty treacherous, so most of the manual error handling stuff you see on things like realloc and malloc are really just performative. In an application setti…

After using this malloc-auto-abort() style for many many years, I've come to believe that if only for the better error handling properties , manual memory management should primarily be done via explicit up front arena allocation using OS API's like mmap/VirtualAlloc, then a bump allocator within the arena. It helps in the vast amount of cases where sensible memory bounds are known or can be inferred, and it means th…

I feel like the prospect of using arenas and pools is further evidence that malloc and realloc should abort on failure, because you're right: if you're using an arena, you've not only taken application-layer control over allocation, but you've also implicitly segregated out a range of allocations for which you presumably have a strategy for exhaustion. The problem with malloc is that it's effectively the system allocator, which means the whole runtime is compromised when it fails. Yes: if you want to manually manage allocation failures, do it by using a pool or arena allocator on top of malloc.

Re: Understanding Memory Management, Part 1: C

#86

Post by returningfory2 »

dataflow wrote: post #81
returningfory2 wrote: post #80

Earlier quoted context omitted.

The code in the article is not wrong. It is not unsafe. The author explicitly handles the OOM case correctly. It is true that there are more optimal ways to do it if you do have an OOM handling strategy. And no, you're not supposed to teach your students the best way to do things at the start. That's not how teaching works. You start with the simpler (but still correct) way, and then work towards the best way. This i…

> The code in the article is not wrong. It is not unsafe. The author explicitly handles the OOM case correctly. And hence we circle back to what I just wrote above: you're confusing the code with the program that it compiles to. Because the code isn't there solely for the purpose of being compiled into a program, it's also serving as a stepping stone for other things (learning, modification, whatever). https://news.y…

I don't see why the code is not "learning safe". The code presents the simplest safe way to handle an OOM condition. Seems basically perfect for a _beginners guide_ to manual memory management.

Re: Understanding Memory Management, Part 1: C

#87

Post by dataflow »

returningfory2 wrote: post #86
dataflow wrote: post #81

Earlier quoted context omitted.

> The code in the article is not wrong. It is not unsafe. The author explicitly handles the OOM case correctly. And hence we circle back to what I just wrote above: you're confusing the code with the program that it compiles to. Because the code isn't there solely for the purpose of being compiled into a program, it's also serving as a stepping stone for other things (learning, modification, whatever). https://news.y…

I don't see why the code is not "learning safe". The code presents the simplest safe way to handle an OOM condition. Seems basically perfect for a _beginners guide_ to manual memory management.

It's not learning-safe because it teaches said learners to write bad code like this.

Re: Understanding Memory Management, Part 1: C

#88

Post by commandlinefan »

writebetterc wrote: post #54

This post caused me to create an account. This C code is not good. Writing C is absolutely harder than Python, but you're making it so much harder than it has to be. Your program is buggy as heck, has very finicky cleanup code, and so on. Here's a much easier way to write the program: 1. Dump whole file into buffer as one string 2. Find newlines in buffer, replace with NULs. This also let's you find each line and sav…

> Dump whole file into buffer as one string

... unless the file is too big to fit into memory?

Re: Understanding Memory Management, Part 1: C

#89

Post by commandlinefan »

imbnwa wrote: post #66
jll29 wrote: post #3

Great post for intermediary programmers, who started programming in Python, and who should now learn what's under the hood to get to the next level of their education. Sometimes (perhaps most of the time), we should ignore the nitty gritty details, but the moment comes where you need to know the "how": either because you need more performance, sort out an issue, or do something that requires low-level action. There a…

Which is why it sucks the top comments are pedantry over what is proper C code, or other comments are about how to optimize the article's code, all missing the point that we're learning concepts that can be corrected later

> pedantry over what is proper C code

As soon as I clicked on the link and saw there was C code included, I knew how the comment section was going to go...

Re: Understanding Memory Management, Part 1: C

#90

Post by commandlinefan »

sylware wrote: post #12

Avoid as much as you can the C standard lib allocator, go directly to mmap system call with your own allocator if you know you won't use CPU without a MMU. If you write a library, let the user code install its own allocator.

> go directly to mmap system call

TFA said that, too... IIRC (and based on a quick googling), mmap is for memory-mapping files into the virtual address space. I thought sbrk() was used for low-level adjustment of available memory and malloc was responsible for managing an allocation handed to it by the sbrk() call. Or has that fallen out of fashion since I last did low-level C programming?

Post reply on HN