9cc: A Small C Compiler

Post by sifoobar » Sat, Dec 15, 2018, 2:47 PM UTC

wtracy wrote: post #48

sifoobar wrote: post #13

Earlier quoted context omitted.

It will put more pressure on the allocator when running, doing that a lot will likely have some kind of cumulative consequence down the line. I know from experience [0] that even reusing allocated memory rather than bouncing it back to malloc can have dramatic effects. [0] https://gitlab.com/sifoo/snigl/blob/master/src/snigl/pool.h

Malloc doesn't interact with the kernel at all. The kernel sees pages, not the data structures that malloc manages. The kernel doesn't even know whether you free()'d the memory by the time the process exits. There is exactly zero difference from the operating system's perspective between freeing and not freeing the memory before program termination (except that one might have a higher peak memory usage). The classic…

Post by tptacek » Sat, Dec 15, 2018, 3:49 PM UTC

bakul wrote: post #53

tptacek wrote: post #39

Earlier quoted context omitted.

Wait, doesn't it do the opposite? The big thing that "pressures" an allocator is fragmentation, which you don't get at all if you never free.

With no freeing, memory used is the sum of all allocations. With freeing, it is max of the allocation for active objects at any given time. The former can ask for much more memory from the OS for some programs that generate a lot of temp garbage. Though this probably doesn’t matter for a C compiler - unless it is required to compile megabytes of generated code!

Post by wtracy » Sun, Dec 16, 2018, 3:00 AM UTC

dexen wrote: post #54

wtracy wrote: post #48

Earlier quoted context omitted.

Malloc doesn't interact with the kernel at all. The kernel sees pages, not the data structures that malloc manages. The kernel doesn't even know whether you free()'d the memory by the time the process exits. There is exactly zero difference from the operating system's perspective between freeing and not freeing the memory before program termination (except that one might have a higher peak memory usage). The classic…

> There is exactly zero difference from the operating system's perspective (...) Close, but not quite: in case of larger allocations, malloc() tends to use mmap( , , , | MAP_ANON) rather than brk() to request memory from the OS. For example, the glibc's malloc() uses mmap() when requested size exceeds MMAP_THRESHOLD, which is 128kB by default. The mmap() approach gives large, continuous memory blocks that can also be…

Post by cperciva » Sun, Dec 16, 2018, 11:32 AM UTC

wtracy wrote: post #65

dexen wrote: post #54

Earlier quoted context omitted.

> There is exactly zero difference from the operating system's perspective (...) Close, but not quite: in case of larger allocations, malloc() tends to use mmap( , , , | MAP_ANON) rather than brk() to request memory from the OS. For example, the glibc's malloc() uses mmap() when requested size exceeds MMAP_THRESHOLD, which is 128kB by default. The mmap() approach gives large, continuous memory blocks that can also be…

An implementation certainly could return memory to the OS on calls to free(), but to my knowledge none of the widely used implementations do so. (I would be interested to learn of counterexamples!)

Post by bakul » Sun, Dec 16, 2018, 8:30 PM UTC

tptacek wrote: post #63

bakul wrote: post #53

Earlier quoted context omitted.

With no freeing, memory used is the sum of all allocations. With freeing, it is max of the allocation for active objects at any given time. The former can ask for much more memory from the OS for some programs that generate a lot of temp garbage. Though this probably doesn’t matter for a C compiler - unless it is required to compile megabytes of generated code!

An allocation you don't have to free is, at the margin, a simple pointer increment. It's the need to free memory that makes allocators expensive, not the amount of memory they allocate. This is why performance-sensitive programs use pools, arenas, and other preallocation strategies.