Live data from Hacker News

How I wrote a self-hosting C compiler in 40 days

sigbus.info

31–40 of 128 posts

Re: How I wrote a self-hosting C compiler in 40 days

#31
post #30

For anyone interested in compiler writing and looking for a good resource to start, probably one of the best is the "Dragon Book": http://www.amazon.com/Compilers-Principles-Techniques-Tools-... I highly recommend it, but it's heavy stuff. There are probably simpler guides out there that just cover the basics.

Please stop recommending the Dragon Book already. It is not just heavy, it is mostly outdated and irrelevant.

Please could you recommend an alternative?

Re: How I wrote a self-hosting C compiler in 40 days

#32

Long ago UNIX had compiler writing tools like yacc and lex. I wonder if they are useful for exercises like this.

They might, although no production quality C or C++ compiler uses anything other than a hand-rolled recursive descent parser, afaik.

The lex, parse and AST directories in Clangs source tree are ~100,000 LOC combined, and all hand-written.

Re: How I wrote a self-hosting C compiler in 40 days

#33
post #30

Earlier quoted context omitted.

Please stop recommending the Dragon Book already. It is not just heavy, it is mostly outdated and irrelevant.

Please could you recommend an alternative?

Modern Compiler Implementation in C

Re: How I wrote a self-hosting C compiler in 40 days

#34

For anyone interested in compiler writing and looking for a good resource to start, probably one of the best is the "Dragon Book": http://www.amazon.com/Compilers-Principles-Techniques-Tools-... I highly recommend it, but it's heavy stuff. There are probably simpler guides out there that just cover the basics.

Jack Crenshaw, of "Let's Build A Compiler" fame, has an interesting critique of works like these:

http://compilers.iecc.com/crenshaw/tutor8.txt

The main point for our discussion is last:

> Desire for Generality

> We have been concentrating on the use of a recursive-descent parser to parse a deterministic grammar, i.e., a grammar that is not ambiguous and, therefore, can be parsed with one level of lookahead.

> In practice, these issues turn out to be considerably less important. Modern languages tend to be designed to be easy to parse, anyway. That was a key motivation in the design of Pascal. Sure, there are pathological grammars that you would be hard pressed to write unambiguous BNF for, but in the real world the best answer is probably to avoid those grammars!

Here's a bit more of what he said, heavily snipped:

> Limited RAM Forcing Multiple Passes

> All the early compiler writers had to deal with this issue: Break the compiler up into enough parts so that it will fit in memory. When you have multiple passes, you need to add data structures to support the information that each pass leaves behind for the next. That adds complexity, and ends up driving the design.

> Batch Processing

> In a mainframe compiler as well as many micro compilers, considerable effort is expended on error recovery ... it can consume as much as 30-40% of the compiler and completely drive the design. The idea is to avoid halting on the first error, but rather to keep going at all costs, so that you can tell the programmer about as many errors in the whole program as possible.

> Large Programs

[Basically comes down to "This is 1980s Micro Land. We don't have mmap or virtual memory in general. The simple way is to keep everything in RAM and encourage small subroutines."]

> Emphasis on Efficiency

[This is an interesting point. He says that we have fast enough CPUs that compilers can emit sub-optimal code and it doesn't matter.]

> Limited Instruction Sets

[Eh. I don't agree that limited instruction sets make compilers more complicated. Even in his design, code generation was a rather trivial part of the entire program.]

Main link:

http://compilers.iecc.com/crenshaw/

Re: How I wrote a self-hosting C compiler in 40 days

#35

Long ago UNIX had compiler writing tools like yacc and lex. I wonder if they are useful for exercises like this.

For writing a general compiler (or anything similar to that), they're extremely useful because they produce very good lexers and/or parsers. The GNU versions of those two are 'bison' and 'flex'. Really, just about anything that requires parsing text can gain from using both of them. Noting that though, for this specific exercise they're not as useful because the author intended for this compiler to be self-hosting. I…

With the huge caveat that "nobody" uses these tools for production compilers because decent error handling becomes a nightmare.

There are exceptions, but if you dig into most larger compilers, they usually sooner or later end up adopting their own handwritten recursive descent parsers.

Re: How I wrote a self-hosting C compiler in 40 days

#37
post #30

For anyone interested in compiler writing and looking for a good resource to start, probably one of the best is the "Dragon Book": http://www.amazon.com/Compilers-Principles-Techniques-Tools-... I highly recommend it, but it's heavy stuff. There are probably simpler guides out there that just cover the basics.

Please stop recommending the Dragon Book already. It is not just heavy, it is mostly outdated and irrelevant.

It's fairly heavy in lexing and parsing theory, especially around finite automata, pushdown automata, etc. It's the kind of book you might want to read if you're reimplementing yacc.

Modern code generation has moved on a bit, so I wouldn't dig too deeply into the latter third or so of the book.

All in all, for a hobby compiler, it would be a poor choice; heavy on unnecessary theory in the front end and outdated on on back end stuff.

Let's Build a Compiler by Jack Crenshaw is one of my favourite guides for the hobbyist, although it's a bit dated now since it used Turbo Pascal and targeted 16-bit x86.

Re: How I wrote a self-hosting C compiler in 40 days

#39
post #34

For anyone interested in compiler writing and looking for a good resource to start, probably one of the best is the "Dragon Book": http://www.amazon.com/Compilers-Principles-Techniques-Tools-... I highly recommend it, but it's heavy stuff. There are probably simpler guides out there that just cover the basics.

Jack Crenshaw, of "Let's Build A Compiler" fame, has an interesting critique of works like these: http://compilers.iecc.com/crenshaw/tutor8.txt The main point for our discussion is last: > Desire for Generality > We have been concentrating on the use of a recursive-descent parser to parse a deterministic grammar, i.e., a grammar that is not ambiguous and, therefore, can be parsed with one level of lookahead. > In pra…

May of these issues are still relevant to embedded systems with limited resources.

Re: How I wrote a self-hosting C compiler in 40 days

#40
post #30

For anyone interested in compiler writing and looking for a good resource to start, probably one of the best is the "Dragon Book": http://www.amazon.com/Compilers-Principles-Techniques-Tools-... I highly recommend it, but it's heavy stuff. There are probably simpler guides out there that just cover the basics.

Please stop recommending the Dragon Book already. It is not just heavy, it is mostly outdated and irrelevant.

[deleted]
Post reply on HN