Becoming a compiler engineer
rona.substack.com
Becoming a compiler engineer
1–10 of 160 posts
Re: Becoming a compiler engineer
#2Not many companies are willing to maintain a compiler... but LLMs will change that. An LLM can find bugs in the code if the "compiler guru" is out on vacation that day. And yes, you will still need a "compiler guru" who will use the LLM but do so at a much higher level.
Re: Becoming a compiler engineer
#3Not many companies are willing to maintain a compiler... but LLMs will change that. An LLM can find bugs in the code if the "compiler guru" is out on vacation that day. And yes, you will still need a "compiler guru" who will use the LLM but do so at a much higher level.
LLMs (or LLM assisted coding), if successful, will more likely make the number of compilers go down, as LLMs are better with mainstream languages compared to niche ones. Same effect as with frameworks. Less languages, less compilers needed.
Re: Becoming a compiler engineer
#4Made an account to say thank you for sharing this post (and to Rona Wang for writing it)! I stumbled into having an interview for a Compiler Engineer position coming up and I wasn't sure how to prepare for it (the fact that I got this interview just goes to show how little people really know about Compilers if they're willing to take a chance on a normal C++ dev like me hah) and I had absolutely NO idea where to even begin (I was just working through Crafting Interpreters[1] that I picked up at the end of my contractorship last week but that's to make an Interpreter, not to make a Compiler)
...And honestly it seems that I'm screwed. And I need about 6 months of study to learn all this stuff. What I'd do right now is finish Crafting Interpreters, then grab that other book on Interpreters that was recommended here recently[2] and written in Go because I remember it had a followup book on Compilers, and THEN start going through the technical stuff that Rona suggested in the article.
And my interview is on Monday so that's not happening. I have other more general interviews that should pay better so I'm not too upset. If only I wasn't too lazy during my last position and kept learning while working. If the stars align and somehow I get that Compiler Engineer position, then I will certainly reach out to Rona and thank you again lalitkale for sharing this post with HN!
Re: Becoming a compiler engineer
#5Not many companies are willing to maintain a compiler... but LLMs will change that. An LLM can find bugs in the code if the "compiler guru" is out on vacation that day. And yes, you will still need a "compiler guru" who will use the LLM but do so at a much higher level.
LLMs (or LLM assisted coding), if successful, will more likely make the number of compilers go down, as LLMs are better with mainstream languages compared to niche ones. Same effect as with frameworks. Less languages, less compilers needed.
I mostly disagree.
First, LLMs should be happy to use made up languages described in a couple thousand tokens without issues (you just have to have a good llm-friendly description, some examples). That and having a compiler it can iterate with / get feedback from.
Second, LLMs heavily reduce ecosystem advantage. Before LLMs, presence of libraries for common use cases to save myself time was one of the main deciding factors for language choice.
Now? The LLM will be happy to implement any utility / api client library I want given the API I want. May even be more thoroughly tested than the average open-source library.
Re: Becoming a compiler engineer
#6Most (all?) of compiler engineering jobs I've seen were about writing glue code for LLVM.
Re: Becoming a compiler engineer
#7Step one: no engineering education, just get a job that a company calls engineering.
>In 2023, I graduated from MIT with a double major in math and computer science.
Re: Becoming a compiler engineer
#8Not many companies are willing to maintain a compiler... but LLMs will change that. An LLM can find bugs in the code if the "compiler guru" is out on vacation that day. And yes, you will still need a "compiler guru" who will use the LLM but do so at a much higher level.
I'm desperately looking forward to, like, 5-10 years from now when all the "LLMs are going to change everything!!1!" comments have all but completely abated (not unlike the blockchain stuff of ~10 years ago).
No, LLMs are not going to replace compiler engineers. Compilers are probably one of the least likely areas to profit from extensive LLM usage in the way that you are thinking, because they are principally concerned with correctness, and LLMs cannot reason about whether something is correct — they only can predict whether their training data would be likely to claim that it is correct.
Additionally, each compiler differs significantly in the minute details. I simply wouldn't trust the output of an LLM to be correct, and the time wasted on determining whether it's correct is just not worth it.
Stop eating pre-chewed food. Think for yourself, and write your own code.
Re: Becoming a compiler engineer
#9Very interesting and informative!
I'm a bit shocked that it would take significant effort/creativity for an MIT grad with relevant course/project work to get a job in the niche
I would have thought the recruiting pipeline is kinda smooth
Although maybe it's a smaller niche than I think -- I imagine compiler engineers skew more senior. Maybe it's not a common first or second job
I graduated at the bottom of bear market (2001), and it was hard to get a job. But this seems a bit different
Re: Becoming a compiler engineer
#10Made an account to say thank you for sharing this post (and to Rona Wang for writing it)! I stumbled into having an interview for a Compiler Engineer position coming up and I wasn't sure how to prepare for it (the fact that I got this interview just goes to show how little people really know about Compilers if they're willing to take a chance on a normal C++ dev like me hah) and I had absolutely NO idea where to even…
In my dabbling with compilers I’ve found Andrew Appel’s books [0] to be invaluable for understanding backend (after parsing) compiler algorithms. It’s a bit dated but covers SSA and other still-relevant optimizations and is pretty readable.
There are three versions (C, ML, and Java). The language isn’t all that important; the algorithms are described in pseudo-code.
I also find the traditional Dragon Book to be somewhat helpful, but you can mostly skip the parsing/frontend sections.