Live data from Hacker News

Lessons learned building a toy compiler

jaseemabid.github.io

11–20 of 23 posts

Re: Lessons learned building a toy compiler

#11

Author of the post here. AMA :)

Hello there! Great post :)

I never took a course in writing compilers at University but have always been interested - where would be the best place to start for somebody interested in doing this? To dip their toes into the water if you will?

Re: Lessons learned building a toy compiler

#12

Earlier quoted context omitted.

I think lexing is unnecessary with some methods, e.g. parser combinators and (I think) Might’s “Parsing with Derivatives”. And there are other tactics, like the other commenter mentioned.

There are alternatives, certainly. And lisp style languages are especially easy to parse without a lexer. I was just taking issue to the term "quintessential" being used. In a classical sense, the lexer comes first.

It depends on your mental hierarchy doesn't it? It's easy enough to see lexing as a sub-step of parsing. It's also quite arguable that the classical emphasis of a separation between parsing and lexing is as much an artifact of classical tools than a "natural need".

For instance, from a Chomsky's hierarchy of languages perspective lexing and parsing are simply language analysis at two different levels on the hierarchy with lexing focused on the "regular" language embedded within a larger context-free or context-sensitive language.

Re: Lessons learned building a toy compiler

#13

I love seeing articles like this... my compiler design course in college was one of my favorite. Also, one of the most useful. Parsers and lexers are useful in so many places besides just code compilers. With that said, I think there is one small issue in the article: > The quintessential first step in any compiler is parsing the source string into an Abstract Syntax Tree If there is a quintessential first step in wr…

You are right and calling it quintessential might be too much. Local lexing seems interesting, but the approach I used is called combinatorial parsing and it does parsing and lexing in one go because they are so intertwined anyway. This also lets me get to the AST in just one pass over the input string with very little backtracking.

I'll update the post with this feedback.Thank you.

Re: Lessons learned building a toy compiler

#14
post #2

It's nice to have a small example of how to compile to LLVM, but the compiler is a bit more limited than what the blog post makes it appear. It's not quite `a compiler for simply typed lambda calculus', but only for a small fragment without higher-order functions. One currently cannot write lambda terms that take functions as arguments. I was curious how the compiler represents closures and manages memory, mainly bec…

You are right, the compiler cannot handle much of higher order functions and closures yet. I have hinted how to do that with lambda lifting and closure conversion and I might get it working in the next few weeks. Maybe a part 2 for the blog post.

Re: Lessons learned building a toy compiler

#15
post #11

Author of the post here. AMA :)

Hello there! Great post :) I never took a course in writing compilers at University but have always been interested - where would be the best place to start for somebody interested in doing this? To dip their toes into the water if you will?

Just tinkering with the tools and writing some code helped me a lot more than the university course I had 6 years ago. There are several beginner friendly online resources now including

1. Write a scheme in 48hrs in Haskell

2. Kaleidoscope tutorial by LLVM developers

3. http://www.buildyourownlisp.com

4. http://createyourproglang.com

Re: Lessons learned building a toy compiler

#17
post #11

Earlier quoted context omitted.

Hello there! Great post :) I never took a course in writing compilers at University but have always been interested - where would be the best place to start for somebody interested in doing this? To dip their toes into the water if you will?

Just tinkering with the tools and writing some code helped me a lot more than the university course I had 6 years ago. There are several beginner friendly online resources now including 1. Write a scheme in 48hrs in Haskell 2. Kaleidoscope tutorial by LLVM developers 3. http://www.buildyourownlisp.com 4. http://createyourproglang.com

I would also add 5.http://dev.stephendiehl.com/fun/
Post reply on HN