Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's write a compiler, part 5: A code generator

briancallahan.net

Post reply on HN
Print view View original

11–20 of 32 posts

Re: Let's write a compiler, part 5: A code generator

#11

Post by codr7 »

ofiryanai wrote: post #8

Hi all, I have a bit off topic question but seems related. I'm trying to write sort of a SQL compiler. The current goal is to analyze queries and find similarities, later maybe to translate between sql dialects. I found Uber's QueryParser[1] but it's in haskell, so I started wrapping the python sqlparse[2] library and implement a Visitor to traverse their weird AST. 1. How close is it to implementing a compiler? 2. I…

Depends on the complexity of your queries, but if you have a narrow subset that you're interested in, implementing a recursive descent parser for just those parts of the syntax that fits your problem like a glove could be a better solution.

https://github.com/codr7/swifties

Re: Let's write a compiler, part 5: A code generator

#12

Post by ModernMech »

The ontology of compilation seems to be the only thing people are engaging with regarding this blog post series here on HN, so I don't see why this series is on the front page every day when the discussion is so stale. If you want to talk about how compiling to C is or isn't compiling, save yourself the trouble, everything has been said already in the past week (including this thread):

https://news.ycombinator.com/item?id=28184187

https://news.ycombinator.com/item?id=28199971

https://news.ycombinator.com/item?id=28209950

Alternatively, congrats to the author, for managing to get his blog on the front page 4 out of the last 5 days in a row. Has that ever been done before on HN, I wonder? It's gotta be some sort of record.

Re: Let's write a compiler, part 5: A code generator

#14

Post by refneb »

ofiryanai wrote: post #8

Hi all, I have a bit off topic question but seems related. I'm trying to write sort of a SQL compiler. The current goal is to analyze queries and find similarities, later maybe to translate between sql dialects. I found Uber's QueryParser[1] but it's in haskell, so I started wrapping the python sqlparse[2] library and implement a Visitor to traverse their weird AST. 1. How close is it to implementing a compiler? 2. I…

I’ve been playing with ANTLR[1] and pretty happy with the generated parser. You can find sql grammar on GitHub[2]

[1] https://www.antlr.org/

[2] https://github.com/antlr/grammars-v4

Re: Let's write a compiler, part 5: A code generator

#15

Post by ofiryanai »

codr7 wrote: post #11
ofiryanai wrote: post #8

Hi all, I have a bit off topic question but seems related. I'm trying to write sort of a SQL compiler. The current goal is to analyze queries and find similarities, later maybe to translate between sql dialects. I found Uber's QueryParser[1] but it's in haskell, so I started wrapping the python sqlparse[2] library and implement a Visitor to traverse their weird AST. 1. How close is it to implementing a compiler? 2. I…

Depends on the complexity of your queries, but if you have a narrow subset that you're interested in, implementing a recursive descent parser for just those parts of the syntax that fits your problem like a glove could be a better solution. https://github.com/codr7/swifties

I'm aiming to analyze all the BI queries in my organization, some of them are quite complicated and most of them are hundreds lines of sql. Thanks for the direction, I'll dive into it

Re: Let's write a compiler, part 5: A code generator

#16

Post by chrisseaton »

ModernMech wrote: post #12

The ontology of compilation seems to be the only thing people are engaging with regarding this blog post series here on HN, so I don't see why this series is on the front page every day when the discussion is so stale. If you want to talk about how compiling to C is or isn't compiling, save yourself the trouble, everything has been said already in the past week (including this thread): https://news.ycombinator.com/it…

> so I don't see why this series is on the front page every day when the discussion is so stale

People submit it and upvote it.

Re: Let's write a compiler, part 5: A code generator

#17

Post by ingve »

ModernMech wrote: post #12

The ontology of compilation seems to be the only thing people are engaging with regarding this blog post series here on HN, so I don't see why this series is on the front page every day when the discussion is so stale. If you want to talk about how compiling to C is or isn't compiling, save yourself the trouble, everything has been said already in the past week (including this thread): https://news.ycombinator.com/it…

Crafting Interpreters [0] just came out in print and was widely discussed here [1], there's a Lang Jam [2] event happening this weekend which also spawned a useful HN discussion [3], so I just think a lot of readers here are interested in simple and fun language development tutorials and inspiration. But it is curious, part 2 of Brian Callahan's series [4] failed to get any traction here, maybe people aren't that into lexing?

[0] https://craftinginterpreters.com/

[1] https://news.ycombinator.com/item?id=27997167

[2] https://github.com/langjam/langjam

[3] https://news.ycombinator.com/item?id=28021161

[4] https://briancallahan.net/blog/20210815.html

Re: Let's write a compiler, part 5: A code generator

#18

Post by im3w1l »

junon wrote: post #3
p4bl0 wrote: post #2

This series of blogposts is amusing but it is kind of frustrating too: the end result is merely a compiler. It is a source to source translator from a source language which is very very similar to a subset of the target language. Well I guess you can call that a compiler, but it really doesn't teach the readers what an actual compiler looks like. Exaggerating a little, I would say that it feels like a few calls to se…

It could not. Unless your language is regular class, then regular expressions cannot parse the language accurately.

You can build a parser around regexes though, where most of the code is regexes and then you have a little bit of code to deal with the irregularity. For instance consider arithmethic expressions consisting of constants, -, +, *, /, and parentheses. You could evaluate that using something like (expression to parse a numeral is left as exercise to reader).

   while expression is not a numeral
       replace all "\((NUMERAL)\)" with first group
       if find first "(NUMERAL)([*/])(NUMERAL)"
         replace with result
       else if find first "(NUMERAL)([+-])(NUMERAL)"
         replace with result

Re: Let's write a compiler, part 5: A code generator

#19

Post by ashton314 »

im3w1l wrote: post #18
junon wrote: post #3

Earlier quoted context omitted.

It could not. Unless your language is regular class, then regular expressions cannot parse the language accurately.

You can build a parser around regexes though, where most of the code is regexes and then you have a little bit of code to deal with the irregularity. For instance consider arithmethic expressions consisting of constants, -, +, *, /, and parentheses. You could evaluate that using something like (expression to parse a numeral is left as exercise to reader). while expression is not a numeral replace all "\((NUMERAL)\)"…

What you are doing there is conflating the lexing phase and the parsing phase. Regexes are perfect for recognizing tokens, but for a language with nested parentheses, you must have a push-down automaton to process it. Otherwise, you will not be able to verify that your delimiters are matched.

Re: Let's write a compiler, part 5: A code generator

#20

Post by Joker_vD »

Alright, let's write a comment about the actual implementation described in the post. So: why make the symbol table a linked list that has its head fixed at "main" and grows and shrinks at its tail? It is used in a stack-like fashion, so why not reverse the order: put "main" at its tail and grow and shrink it at its head?

That simplifies adding symbols: you keep checking for duplicates until the depth of the symbols you see drops below your depth, and actual insertion doesn't need the pointer to the list's last element, you insert at the head.

That simplifies destroying symbols: you keep popping the head until you meet a TOK_PROCEDURE at which point you stop.

That simplifies looking symbols up: you search until the first name match and return it immediately, instead of remembering the last seen matching symbol while traversing the full list.

I am not even talking about efficiency (although of course this implementation is also more efficient in all 3 use cases), it's about code simplicity: the less context you need to maintain during a list traversal, the easier it is to understand what this traversal looks for.

Post reply on HN