Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory leak proof every C program

flak.tedunangst.com

Post reply on HN
Print view View original

81–90 of 175 posts

Re: Memory leak proof every C program

#81

Post by GuB-42 »

londons_explore wrote: post #39

I've got an idea... We have a counter that goes up by 1 every time you call malloc. And down by one every time you call free. And when the program quits, if the counter isn't zero, an email is fired off and a dollar gets sent from the developers bank account to the users bank account...

That's essentially how all leak detection tools work, minus the money part.

And it is not even always appropriate. It is common to allocate some memory for the entire lifetime of the process. For example, if your app is GUI-based and has a main window, there is no need to free the resources tied to the main window, because closing it means quitting the app which will cause all memory to be reclaimed by the OS. You can properly free your memory but it will only make quitting slower. Usually programmers only do that to satisfy leak detection tools, and if the overhead is significant, it may only be done in debug mode.

Re: Memory leak proof every C program

#82

Post by lgg »

There is a bug here... Clearly the author intended to cache the value of nextmalloc to avoid calling dlsym() on every malloc. The correct code should be:

  static void *(*nextmalloc)(size_t) = NULL;
  if (!nextmalloc) 
    nextmalloc = dlsym(RTLD_NEXT, "malloc");
  }
Somehow the fact that the optimization is incorrectly missed here feels appropriate ;-)

Re: Memory leak proof every C program

#83

Post by hamburglar »

omoikane wrote: post #21
enriquto wrote: post #3

This is the core idea: > It is [...] entirely optional to call free. If you don’t call free, memory usage will increase over time, but technically, it’s not a leak. As an optimization, you may choose to call free to reduce memory, but again, strictly optional. This is beautiful! Unless your program is long-running, there's no point of ever calling free in your C programs. The system will free the memory for you when…

This solution did not override "free" to free the extra struct, so calling free is in fact not optional but probably a bad idea. Need an extra patch: #define free(_) /* no-op */

Best criticism to this tongue-in-cheek solution yet. I’ll add my own more minor criticism: it will explode if the leaksaver struct allocation ever fails because it doesn’t check the result.

That does it. I am not going to use it.

Re: Memory leak proof every C program

#84

Post by shortrounddev2 »

tombert wrote: post #80
rco8786 wrote: post #41

Earlier quoted context omitted.

Reminds me of the HFT shop that built in Java and simply turned the garbage collector off. Then when the market closed they would restart the process for the next day.

I had a friend who worked for one of the big Market Makers, and he told me that they would indeed turn the GC off, but what they'd do is just pre-allocate everything into bigass arrays before-hand, and have incrementers to simulate the "new" keyword. They might do this in something more or less like a threadlocal to avoid having to deal with locks or race conditions or anything like that.

A lot of game engines do this, avoiding the cost of a malloc lookup by just allocating everything as a stack

Re: Memory leak proof every C program

#85

Post by junon »

tombert wrote: post #80
rco8786 wrote: post #41

Earlier quoted context omitted.

Reminds me of the HFT shop that built in Java and simply turned the garbage collector off. Then when the market closed they would restart the process for the next day.

I had a friend who worked for one of the big Market Makers, and he told me that they would indeed turn the GC off, but what they'd do is just pre-allocate everything into bigass arrays before-hand, and have incrementers to simulate the "new" keyword. They might do this in something more or less like a threadlocal to avoid having to deal with locks or race conditions or anything like that.

This is called bump or arena allocation, depending on how you want to define it.

Re: Memory leak proof every C program

#86

Post by shortrounddev2 »

tetromino_ wrote: post #64
HideousKojima wrote: post #51

Earlier quoted context omitted.

Or the missile firmware where the missile is going to explode before they run out of memory: https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98...

Using warhead explosion as garbage collector might seem like a clever hack, but all it takes is an upgrade by a different team (say, adding a new engine, or longer-range sensors, or repurposing a surface-to-air missile for a surface-to-surface role), and suddenly your guidance software runs out of memory before it explodes, and your missile falls onto an an elementary school or hospital.

I assume they would test the missiles before attesting to its range

Re: Memory leak proof every C program

#87

Post by junon »

omoikane wrote: post #21
enriquto wrote: post #3

This is the core idea: > It is [...] entirely optional to call free. If you don’t call free, memory usage will increase over time, but technically, it’s not a leak. As an optimization, you may choose to call free to reduce memory, but again, strictly optional. This is beautiful! Unless your program is long-running, there's no point of ever calling free in your C programs. The system will free the memory for you when…

This solution did not override "free" to free the extra struct, so calling free is in fact not optional but probably a bad idea. Need an extra patch: #define free(_) /* no-op */

Nitpick: for maximum source compatibility, use this instead:

    #define free(_) ((void)0)

Re: Memory leak proof every C program

#88

Post by bluGill »

rco8786 wrote: post #41
enriquto wrote: post #3

This is the core idea: > It is [...] entirely optional to call free. If you don’t call free, memory usage will increase over time, but technically, it’s not a leak. As an optimization, you may choose to call free to reduce memory, but again, strictly optional. This is beautiful! Unless your program is long-running, there's no point of ever calling free in your C programs. The system will free the memory for you when…

Reminds me of the HFT shop that built in Java and simply turned the garbage collector off. Then when the market closed they would restart the process for the next day.

it is somewhat common in garbage collected langages to fight the garrbage collector like that. Sure manual free probably adds up to more CPU time, but it is more spread out and thus not noticable (normally, real time still cannot allocate in the sensitive areas)

Re: Memory leak proof every C program

#89

Post by steveklabnik »

the-smug-one wrote: post #58
avgcorrection wrote: post #46

Earlier quoted context omitted.

Yeah I don’t know how a C library without `_free()` calls would work across FFI (like making bindings).

Why would FFI be an issue?

It’s important to use the same allocator to free as you allocate with. You can’t assume that the same allocator is linked to both sides, so if you are allocating and then giving something to something else over FFI, you want to make sure you give them a way to call back into you to free.

This is true irrespective of language.

Re: Memory leak proof every C program

#90

Post by bluGill »

kbenson wrote: post #54
MrJohz wrote: post #30

Earlier quoted context omitted.

Any large program is composed of libraries. They may not explicitly be described as libraries, or imported from external sources, but there will be abstraction boundaries somewhere. Which means that if you're writing a program, you probably are also writing one or more libraries.

The difference is that if you're writing a program you know the scope of use of all libraries, whether they be externally loaded or internal abstraction boundaries, and also know the scope of use of the program, and can make a call as to whether cleanup during opetation is required.

Not really. In theory you can, but in practice there are parts written by a different team and you don't know the scope of there parts.

I also know someone who maintains some Clinton era encryption. that code is controlled who can know about it as obsecurity was all you were allowed. There are other pathalogical cases where you can't know everything about your program-

Post reply on HN