Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

9cc: A Small C Compiler

github.com

Post reply on HN
Print view View original

11–20 of 70 posts

Re: 9cc: A Small C Compiler

#11

Post by ychen306 »

mshockwave wrote: post #9

>no memory management is the memory management policy in 9cc. We allocate memory using malloc() but never call free(). >I know that people find the policy odd, but this is actually a reasonable design choice for short-lived programs such as compilers. I'm strongly disagree at this point. Memory management is important even for short-lived programs. It would bring burden to the OS if you invoke this kind of "short-liv…

Generally speaking a chunk of memory you free is simply marked "free" for reuse but not given back to the OS. So in this regard, it is not a burden to the OS.

Re: 9cc: A Small C Compiler

#12

Post by randyrand »

mshockwave wrote: post #9

>no memory management is the memory management policy in 9cc. We allocate memory using malloc() but never call free(). >I know that people find the policy odd, but this is actually a reasonable design choice for short-lived programs such as compilers. I'm strongly disagree at this point. Memory management is important even for short-lived programs. It would bring burden to the OS if you invoke this kind of "short-liv…

IMO it depends on how much memory is expected to be consumed. If its only at most tens on MBs, then I think not doing memory management is not just smart, but more performant.

Re: 9cc: A Small C Compiler

#13

Post by sifoobar »

cperciva wrote: post #10
mshockwave wrote: post #9

>no memory management is the memory management policy in 9cc. We allocate memory using malloc() but never call free(). >I know that people find the policy odd, but this is actually a reasonable design choice for short-lived programs such as compilers. I'm strongly disagree at this point. Memory management is important even for short-lived programs. It would bring burden to the OS if you invoke this kind of "short-liv…

> Memory management is important even for short-lived programs. It would bring burden to the OS if you invoke this kind of "short-lived, memory-management-free" programs multiple times. All the memory is freed when the process exits. Why does it matter if you run the program multiple times?

It will put more pressure on the allocator when running, doing that a lot will likely have some kind of cumulative consequence down the line. I know from experience [0] that even reusing allocated memory rather than bouncing it back to malloc can have dramatic effects.

[0] https://gitlab.com/sifoo/snigl/blob/master/src/snigl/pool.h

Re: 9cc: A Small C Compiler

#14

Post by snops »

mshockwave wrote: post #9

>no memory management is the memory management policy in 9cc. We allocate memory using malloc() but never call free(). >I know that people find the policy odd, but this is actually a reasonable design choice for short-lived programs such as compilers. I'm strongly disagree at this point. Memory management is important even for short-lived programs. It would bring burden to the OS if you invoke this kind of "short-liv…

I remember from the excellent book "Expert C Programming: Deep C Secrets", that the Sun C compiler also took a similar approach to using malloc, so it isn't unheard of for compilers specifically.

I disagree with your characterisation of this as bad practice and "not facing it", it is an informed decision rather than ignorance, and brings considerable benefits. With a C complier, some reasonable estimates can be made of the input data length/complexity and hence allocation sizes, and having poor performance if someone tries to feed in a million line file is perfectly acceptable, especially in a complier designed to be simple like the OP.

I think recognising the special set of circumstances that justify making an unusual tradeoff that wouldn't normally be acceptable is actually a mark of maturity as a programmer, not a bad attitude. The phrase "Don't let perfect be the enemy of good" comes to mind.

Finally, why would it bring burden to the OS if you invoke this multiple times sequentially? malloc/free is implemented in the C library, not in the kernel, it's a mechanism for sharing bigger chunks fetched/given-back from/to the kernel with sbrk(). The kernel would just reclaim the pages on program exit, as it would have to anyway, as it cannot rely on programs being well behaved enough to call free() (which rarely would give the memory back to the kernel immediately anyway). Since the kernel uses virtual memory, its not actually moving the contents about, just manipulating page tables, so it should be quite fast, and zeroing memory is not as time consuming as you think due to Zero Fill On Demand (ZFOD), and hence the kernel does about the same amount of work regardless of if the program called free() or not.

Sure it's using more peak memory than it could be, but I reckon compared to say, a web browser, it's still a very small fraction of the total.

Re: 9cc: A Small C Compiler

#15

Post by akkartik »

https://github.com/rui314/9cc/blob/882e4b2dd8/main.c#L7

    int main(int argc, char **argv) {
      ...

      Vector *tokens = tokenize(path, true);
      Program *prog = parse(tokens);
      sema(prog);
      gen_ir(prog);

      if (dump_ir1)
        dump_ir(prog->funcs);

      optimize(prog);
      liveness(prog);
      alloc_regs(prog);

      if (dump_ir2)
        dump_ir(prog->funcs);

      gen_x86(prog);
      return 0;
    }
This is wonderful.

Re: 9cc: A Small C Compiler

#16

Post by desertrider12 »

ychen306 wrote: post #11
mshockwave wrote: post #9

>no memory management is the memory management policy in 9cc. We allocate memory using malloc() but never call free(). >I know that people find the policy odd, but this is actually a reasonable design choice for short-lived programs such as compilers. I'm strongly disagree at this point. Memory management is important even for short-lived programs. It would bring burden to the OS if you invoke this kind of "short-liv…

Generally speaking a chunk of memory you free is simply marked "free" for reuse but not given back to the OS. So in this regard, it is not a burden to the OS.

It looks like by default, the GNU libc won't return pages to the OS until 128 KiB are free at the top of the heap segment. The compiler has to make many small allocations, so even if it freed memory at every opportunity I doubt the heap trimming would happen very often.

Source: https://code.woboq.org/userspace/glibc/malloc/malloc.c.html#...

Re: 9cc: A Small C Compiler

#17

Post by userbinator »

Good to see more C/subset-C compilers being written. Besides the immense pedagogical value, they are also useful for preventing the "trusting trust" attack: https://dwheeler.com/trusting-trust/

I will also make a suggestion to use precedence climbing instead of plain recursive descent for the parser; it makes the parser even simpler and table-driven, which is important with a language like C that has many precedence levels.

Re: 9cc: A Small C Compiler

#18

Post by cperciva »

sifoobar wrote: post #13
cperciva wrote: post #10

Earlier quoted context omitted.

> Memory management is important even for short-lived programs. It would bring burden to the OS if you invoke this kind of "short-lived, memory-management-free" programs multiple times. All the memory is freed when the process exits. Why does it matter if you run the program multiple times?

It will put more pressure on the allocator when running, doing that a lot will likely have some kind of cumulative consequence down the line. I know from experience [0] that even reusing allocated memory rather than bouncing it back to malloc can have dramatic effects. [0] https://gitlab.com/sifoo/snigl/blob/master/src/snigl/pool.h

Sure, but if you're not freeing memory you don't need a real malloc. You allocation function can be just "top += allocsize".

Re: 9cc: A Small C Compiler

#20

Post by ksherlock »

mshockwave wrote: post #9

>no memory management is the memory management policy in 9cc. We allocate memory using malloc() but never call free(). >I know that people find the policy odd, but this is actually a reasonable design choice for short-lived programs such as compilers. I'm strongly disagree at this point. Memory management is important even for short-lived programs. It would bring burden to the OS if you invoke this kind of "short-liv…

Not everybody wants to wait around all day for their utilities to free up their memory before exiting.

http://lists.gnu.org/archive/html/coreutils/2014-08/msg00012...

Post reply on HN