The Super Tiny Compiler
github.com
The Super Tiny Compiler
1–10 of 44 posts
Re: The Super Tiny Compiler
#2it wasn't clear at all from a 5 second skim of the code and looking at the readme what the source and targets were
Re: The Super Tiny Compiler
#3it wasn't clear at all from a 5 second skim of the code and looking at the readme what the source and targets were
Ten seconds probably would have done it.
Re: The Super Tiny Compiler
#4it wasn't clear at all from a 5 second skim of the code and looking at the readme what the source and targets were
* We're going to compile some lisp-like function calls into some C-like
* function calls.
From the top of the source file.
Re: The Super Tiny Compiler
#5Question to compiler writers : when is it useful, if ever, to tokenize the whole input beforehand (as done here) ?
You wouldn't catch an early syntax error and would go on tokenizing till the end for nothing.
Re: The Super Tiny Compiler
#6Question to compiler writers : when is it useful, if ever, to tokenize the whole input beforehand (as done here) ? You wouldn't catch an early syntax error and would go on tokenizing till the end for nothing.
I think most programming languages tokenize first. Tokenizing is absurdly fast, like 100x faster than printing to a terminal.
Re: The Super Tiny Compiler
#7Question to compiler writers : when is it useful, if ever, to tokenize the whole input beforehand (as done here) ? You wouldn't catch an early syntax error and would go on tokenizing till the end for nothing.
Wrote a C-subset compiler for a compilers course back in university.
In general it's useful for a compiler to not just hit the first error it can find in the source code and immediately error out. Instead, if you keep parsing after encountering an error, you can often encounter more errors, so that you can give the user a list of errors they need to fix, not just the first one. In that sense, a compiler's job isn't over the instant it encounters a syntax error, so the extra tokenization would not be useless.
Re: The Super Tiny Compiler
#8I thought is was about this: https://bellard.org/otcc/otcc.c (a tiny, obfuscated C compiler, winner of the 2001 IOCCC). It has led to TinyCC (https://bellard.org/tcc/), not as tiny, but also more complete, not obfuscated and actually useful.
It turns out the compiler in the article is the opposite of that. It it a simple toy transpiler and the code is very clear and mostly made of comments. The former is a feat of optimization, the latter is a tutorial.
Re: The Super Tiny Compiler
#9Question to compiler writers : when is it useful, if ever, to tokenize the whole input beforehand (as done here) ? You wouldn't catch an early syntax error and would go on tokenizing till the end for nothing.
Back in the day my Intro To Programming course taught the CLU programming language. I remember one cool feature of the compiler; when it hit a syntax error, it would show the error, skip a few lines until it found a stable place to resume parsing, and then continue compiling in order to catch more syntax errors. Pretty neat!
Re: The Super Tiny Compiler
#10Question to compiler writers : when is it useful, if ever, to tokenize the whole input beforehand (as done here) ? You wouldn't catch an early syntax error and would go on tokenizing till the end for nothing.
Some compilers tokenize while parsing, but for a different reason: it's faster and uses less memory to generate the AST (and sometimes even do analyses) while you're reading the input than to allocate and store a giant list of tokens and then parse that.
Most compilers try not to fail when they encounter a syntax error, they try to "recover" and parse the remaining document, usually starting from the next valid statement or declaration. This lets them report more syntax errors and moreover, is very important for good IDE support (if you've ever used an IDE/language combo where you only get completion after you've written the code, you know why). Although if the syntax error messes up tokenization (e.g. missing quote or end comment) it usually screws up the rest of the document anyways.