Writing a compiler in Python using Lex, Yacc and LLVM
wiki.alcidesfonseca.com
Writing a compiler in Python using Lex, Yacc and LLVM
1–10 of 44 posts
Re: Writing a compiler in Python using Lex, Yacc and LLVM
#2Impractical:
1) The compiler is gonna be very slow;
2) If you make compiler for low-level language, such as C
you need precise correspondence between data types used in cpu, datatypes used in compiler's implementation language and datatypes in the target language.
Re: Writing a compiler in Python using Lex, Yacc and LLVM
#3Impractical: 1) The compiler is gonna be very slow; 2) If you make compiler for low-level language, such as C you need precise correspondence between data types used in cpu, datatypes used in compiler's implementation language and datatypes in the target language.
Did you read the post? The guy was writing this for an academic project where speed didn't matter and secondly he was using LLVM for code generation. low level details don't really come into play in compiler construction until the code generation step most of the work up to that point is text parsing, syntax checking, and AST creation. In other words tasks suited perfectly to a language like Python.
Re: Writing a compiler in Python using Lex, Yacc and LLVM
#4Impractical: 1) The compiler is gonna be very slow; 2) If you make compiler for low-level language, such as C you need precise correspondence between data types used in cpu, datatypes used in compiler's implementation language and datatypes in the target language.
Did you read the post? The guy was writing this for an academic project where speed didn't matter and secondly he was using LLVM for code generation. low level details don't really come into play in compiler construction until the code generation step most of the work up to that point is text parsing, syntax checking, and AST creation. In other words tasks suited perfectly to a language like Python.
> low level details don't really come into play in compiler construction until the code generation
-- Not true. modern compilers do a lot of optimization long before code generation, and you really want to have for example floating point to be behaving exactly like in target language - if you collapse operations on constants such as 1.0D + 4.0. And aside from optimization, lexer - it has to distinguish single-precision and double precision floats.
Re: Writing a compiler in Python using Lex, Yacc and LLVM
#5Impractical: 1) The compiler is gonna be very slow; 2) If you make compiler for low-level language, such as C you need precise correspondence between data types used in cpu, datatypes used in compiler's implementation language and datatypes in the target language.
On the contrary. Compilers written in a high-level language are at a distinct advantage: they're easier to optimize (significantly so), easier to debug, and support forms that are difficult to deal with in a low(er)-level language like C or C++.
I do compiler development every day, and I no longer touch low-level compilers. Everything I write is Boo (for my OS), Ruby (for my startup -- by far my favorite language for compiler dev, despite not liking it in general), or Python. As a result, I have far more maintainable code than the majority, and the code I output is incredibly well optimized.
Edit: To clarify, by 'support forms that are difficult to deal with ...' I mean things like using a pure S-exp structure. Rather than a traditional intermediary form, you can represent your compiler state as an S-exp and iteratively optimize and compile it. Standardizing around a form like that, when it's easy to deal with, greatly simplifies code. (This is actually the reason Ruby is my compiler language of choice these days.)
Re: Writing a compiler in Python using Lex, Yacc and LLVM
#6Impractical: 1) The compiler is gonna be very slow; 2) If you make compiler for low-level language, such as C you need precise correspondence between data types used in cpu, datatypes used in compiler's implementation language and datatypes in the target language.
Because as everybody knows, just as the correct way to learn about weightlifting is to jump straight to benching 350lbs, the correct way to learn compilers is to write a complete gcc replacement as your first project.
If you can't manage that, then, geeze, what kind of crappy programmer are you?
Re: Writing a compiler in Python using Lex, Yacc and LLVM
#7Impractical: 1) The compiler is gonna be very slow; 2) If you make compiler for low-level language, such as C you need precise correspondence between data types used in cpu, datatypes used in compiler's implementation language and datatypes in the target language.
On the contrary. Compilers written in a high-level language are at a distinct advantage: they're easier to optimize (significantly so), easier to debug, and support forms that are difficult to deal with in a low(er)-level language like C or C++. I do compiler development every day, and I no longer touch low-level compilers. Everything I write is Boo (for my OS), Ruby (for my startup -- by far my favorite language for…
if you read carefully what I said, i never said that hig-level languages are bad - I said only this:
1) Compiler written in python, ruby and other "slow" (do not beleive that pyruby is slow?) languages going to take eternity to compile linux kernel. 2) You need to have _precise_ mapping between types in compiler and the language you implement. It is so important that gcc use a software library for floating point computation, otherwise you are locked-in wit you cpu's FP implementation and can not write cross-compilers.
Re: Writing a compiler in Python using Lex, Yacc and LLVM
#8Earlier quoted context omitted.
On the contrary. Compilers written in a high-level language are at a distinct advantage: they're easier to optimize (significantly so), easier to debug, and support forms that are difficult to deal with in a low(er)-level language like C or C++. I do compiler development every day, and I no longer touch low-level compilers. Everything I write is Boo (for my OS), Ruby (for my startup -- by far my favorite language for…
if you read carefully what I said, i never said that hig-level languages are bad - I said only this: 1) Compiler written in python, ruby and other "slow" (do not beleive that pyruby is slow?) languages going to take eternity to compile linux kernel. 2) You need to have _precise_ mapping between types in compiler and the language you implement. It is so important that gcc use a software library for floating point comp…
You said they were impractical, then went on to explain why. I then explained why they are not impractical. The speed I'll give you, but the type issue is nonsensical. GCC's use of a software FP implementation is in support of this. Regardless of the language, you'll need to have an FP implementation that matches the target, which is nearly always done in software.
High-level languages for compilation are, 9 times out of 10, a better choice.
Re: Writing a compiler in Python using Lex, Yacc and LLVM
#9Earlier quoted context omitted.
if you read carefully what I said, i never said that hig-level languages are bad - I said only this: 1) Compiler written in python, ruby and other "slow" (do not beleive that pyruby is slow?) languages going to take eternity to compile linux kernel. 2) You need to have _precise_ mapping between types in compiler and the language you implement. It is so important that gcc use a software library for floating point comp…
You said they were impractical, then went on to explain why. I then explained why they are not impractical. The speed I'll give you, but the type issue is nonsensical. GCC's use of a software FP implementation is in support of this. Regardless of the language, you'll need to have an FP implementation that matches the target, which is nearly always done in software. High-level languages for compilation are, 9 times ou…
1. FP is almost never done in software, it hurts performance big time, The only reason gcc uses it - it wants to be cross-platform, otherwise it might have perfectly used native FP.
Also a lot more convenient to use different floating point point formats whene they are supported by language and hardware you running on, than messing with libraries.
2. Another example : OCaml. It has no "single float" type, and that is a reason I am not gonna use it for making a compiler; I'd choose haskell or F# - both of them have float32.
3. Depends what kind of HLL we are tslking about - Python and Ruby are defintely very poor choice for a compiler : lack of strong typing, only one type of FP is used, needs explicit watch for integer operations overflow, poor performance, lack of mature, bug free implementation (compared to C++ or ML for example).
ML derived languages, however are very good for making compilers.
Re: Writing a compiler in Python using Lex, Yacc and LLVM
#10Earlier quoted context omitted.
On the contrary. Compilers written in a high-level language are at a distinct advantage: they're easier to optimize (significantly so), easier to debug, and support forms that are difficult to deal with in a low(er)-level language like C or C++. I do compiler development every day, and I no longer touch low-level compilers. Everything I write is Boo (for my OS), Ruby (for my startup -- by far my favorite language for…
if you read carefully what I said, i never said that hig-level languages are bad - I said only this: 1) Compiler written in python, ruby and other "slow" (do not beleive that pyruby is slow?) languages going to take eternity to compile linux kernel. 2) You need to have _precise_ mapping between types in compiler and the language you implement. It is so important that gcc use a software library for floating point comp…
> You need to have _precise_ mapping between types in compiler and the language you implement.
No. You don't need that. You can treat your data however you want until you place it into the final binary. Before that, you can even treat all your numbers as strings if you really want to. If you need some constant expression evaluation, you just have to replicate the target machine's math operations in software - no magic involved here. As long as you get the right result, noone cares what you do internally.
If your compilation machine == target machine, you can even construct a function that calculates the expression and actually run it to get the result.