Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

9cc: A Small C Compiler

github.com

Post reply on HN
Print view View original

51–60 of 70 posts

Re: 9cc: A Small C Compiler

#51

Post by Asooka »

mshockwave wrote: post #9

>no memory management is the memory management policy in 9cc. We allocate memory using malloc() but never call free(). >I know that people find the policy odd, but this is actually a reasonable design choice for short-lived programs such as compilers. I'm strongly disagree at this point. Memory management is important even for short-lived programs. It would bring burden to the OS if you invoke this kind of "short-liv…

This is also how the Apache httpd server works, last I checked. Not the entire server, of course - it's a long-lived program with lots of memory to manage, however when servicing a request, it uses the arena pattern to allocate memory and the entire arena is freed at the end of the request. In fact, its entire memory management model is based on a hierarchy of memory arenas that can only grow and get freed once whatever it was that required their liveness ends. I've had the pleasure to write a plugin for it to handle a certain custom upload protocol.

Re: 9cc: A Small C Compiler

#53

Post by bakul »

tptacek wrote: post #39
sifoobar wrote: post #13

Earlier quoted context omitted.

It will put more pressure on the allocator when running, doing that a lot will likely have some kind of cumulative consequence down the line. I know from experience [0] that even reusing allocated memory rather than bouncing it back to malloc can have dramatic effects. [0] https://gitlab.com/sifoo/snigl/blob/master/src/snigl/pool.h

Wait, doesn't it do the opposite? The big thing that "pressures" an allocator is fragmentation, which you don't get at all if you never free.

With no freeing, memory used is the sum of all allocations. With freeing, it is max of the allocation for active objects at any given time. The former can ask for much more memory from the OS for some programs that generate a lot of temp garbage. Though this probably doesn’t matter for a C compiler - unless it is required to compile megabytes of generated code!

Re: 9cc: A Small C Compiler

#54

Post by dexen »

wtracy wrote: post #48
sifoobar wrote: post #13

Earlier quoted context omitted.

It will put more pressure on the allocator when running, doing that a lot will likely have some kind of cumulative consequence down the line. I know from experience [0] that even reusing allocated memory rather than bouncing it back to malloc can have dramatic effects. [0] https://gitlab.com/sifoo/snigl/blob/master/src/snigl/pool.h

Malloc doesn't interact with the kernel at all. The kernel sees pages, not the data structures that malloc manages. The kernel doesn't even know whether you free()'d the memory by the time the process exits. There is exactly zero difference from the operating system's perspective between freeing and not freeing the memory before program termination (except that one might have a higher peak memory usage). The classic…

>There is exactly zero difference from the operating system's perspective (...)

Close, but not quite: in case of larger allocations, malloc() tends to use mmap( , , , | MAP_ANON) rather than brk() to request memory from the OS. For example, the glibc's malloc() uses mmap() when requested size exceeds MMAP_THRESHOLD, which is 128kB by default.

The mmap() approach gives large, continuous memory blocks that can also be easily free()'d via munmap() with little to no bookkeeping needed[1] - not being subject to the same fragmentation woes as memory allocated via brk() - as long as your address space is significantly larger than allocated memory.

That aside, I fully agree with the author of 9cc.

[1] in fact a simplistic libc memory allocator could directly wrap malloc() around mmap(), free() around munmap() and realloc() around mrealloc(), leveraging the in-kernel allocator at the cost of one syscall with at least two context switches at each call - i.e., slowww

Re: 9cc: A Small C Compiler

#55

Post by chubot »

akkartik wrote: post #15

https://github.com/rui314/9cc/blob/882e4b2dd8/main.c#L7 int main(int argc, char **argv) { ... Vector *tokens = tokenize(path, true); Program *prog = parse(tokens); sema(prog); gen_ir(prog); if (dump_ir1) dump_ir(prog->funcs); optimize(prog); liveness(prog); alloc_regs(prog); if (dump_ir2) dump_ir(prog->funcs); gen_x86(prog); return 0; } This is wonderful.

FWIW I started this thread several months ago to advocate that people write their compilers like this :)

https://www.reddit.com/r/ProgrammingLanguages/comments/89n3w...

In many compilers, including some linked on that thread, this clean structure gets lost.

Re: 9cc: A Small C Compiler

#56

Post by emmelaich »

Zelizz wrote: post #23
kazinator wrote: post #6

Regarding your Makefile; you should still pass CFLAGS through to the compiler when linking, not only LDFLAGS. Suppose CFLAGS contains -m32 (supported by an x86-64-targetted GCC to do 32 bit). You compile the .o files with that, but then link without it, which fails trying to make a 64 bit executable out of 32 bit .o's. Some crazy distros pass a --sysroot in CFLAGS; if you don't have that, your build finds the wrong l…

Where would one go to find more of this conventional Makefile wisdom? I've had so many issues trying to use make the "right" way (flexible, clean, terse, etc.). I feel like one of the best ways to acquire this wisdom is to post a project with lots of mistakes and let people tear it apart.

https://nostarch.com/gnumake

Re: 9cc: A Small C Compiler

#58

Post by teddyh »

emmelaich wrote: post #56
Zelizz wrote: post #23

Earlier quoted context omitted.

Where would one go to find more of this conventional Makefile wisdom? I've had so many issues trying to use make the "right" way (flexible, clean, terse, etc.). I feel like one of the best ways to acquire this wisdom is to post a project with lots of mistakes and let people tear it apart.

https://nostarch.com/gnumake

That is not, contrary to what the name might imply, the official GNU Make manual. That one can be read here, for free: https://www.gnu.org/software/make/manual/

Re: 9cc: A Small C Compiler

#60

Post by cperciva »

dexen wrote: post #54
wtracy wrote: post #48

Earlier quoted context omitted.

Malloc doesn't interact with the kernel at all. The kernel sees pages, not the data structures that malloc manages. The kernel doesn't even know whether you free()'d the memory by the time the process exits. There is exactly zero difference from the operating system's perspective between freeing and not freeing the memory before program termination (except that one might have a higher peak memory usage). The classic…

> There is exactly zero difference from the operating system's perspective (...) Close, but not quite: in case of larger allocations, malloc() tends to use mmap( , , , | MAP_ANON) rather than brk() to request memory from the OS. For example, the glibc's malloc() uses mmap() when requested size exceeds MMAP_THRESHOLD, which is 128kB by default. The mmap() approach gives large, continuous memory blocks that can also be…

Using mmap/munmap isn't quite that trivial: You need to remember how large each allocation is in order to know how many pages to unmap.
Post reply on HN