Live data from Hacker News

How to learn compilers: LLVM Edition

lowlevelbits.org

11–20 of 28 posts

Re: How to learn compilers: LLVM Edition

#11

1. Getting Started with LLVM Core Libraries It's a bit dated (covers DAGISel rather than GlobalISel) but it gives a thorough introduction. 2. LLVM Developer Meeting tutorials These are really good although you'll have to put them in order yourself. They will be out of date, a little. LLVM is a moving target. Also, you don't have to go through every tutorial. For example, MLIR is not for me. 3. LLVM documentation I sp…

> 4. Discord, LLVM email list, git blame

and don't forget IRC!

Re: How to learn compilers: LLVM Edition

#12

1. Getting Started with LLVM Core Libraries It's a bit dated (covers DAGISel rather than GlobalISel) but it gives a thorough introduction. 2. LLVM Developer Meeting tutorials These are really good although you'll have to put them in order yourself. They will be out of date, a little. LLVM is a moving target. Also, you don't have to go through every tutorial. For example, MLIR is not for me. 3. LLVM documentation I sp…

> LLVM Developer Meeting tutorials

Are these all in one place or scattered about?

Re: How to learn compilers: LLVM Edition

#14
post #9
post #4

since here's many compiler hackers then I'd want to ask question: How do you distribute your frontend with LLVM? Let's say that I have lexer, parser and emitter written in e.g Haskell (random example) I emit LLVM IR and then I use LLVM to generate something other but the problem is, that I need to have LLVM binaries and I'd rather avoid telling people that want to contribute to my OSS project to install LLVM because…

When you have an LLVM frontend, what you generally do is have your driver run the optimization and code generation steps itself using the LLVM APIs rather than using opt/llc binaries to drive this step. That way, you don't need the LLVM binaries, just the libraries that you statically link into your executable. For example, all of the code in clang to do this is located in https://github.com/llvm/llvm-project/blob/ma…

What if my frontend is written in non cpp? e.g haskell, js, java, c#, etc.

Re: How to learn compilers: LLVM Edition

#15

1. Getting Started with LLVM Core Libraries It's a bit dated (covers DAGISel rather than GlobalISel) but it gives a thorough introduction. 2. LLVM Developer Meeting tutorials These are really good although you'll have to put them in order yourself. They will be out of date, a little. LLVM is a moving target. Also, you don't have to go through every tutorial. For example, MLIR is not for me. 3. LLVM documentation I sp…

> LLVM Developer Meeting tutorials Are these all in one place or scattered about?

Either llvm.org under Developer Meetings or the LLVM Youtube channel. The advantage of llvm.org is that it has a lot of the PDFs for the presentations as well as some old, pre-Youtube tutorials.

Re: How to learn compilers: LLVM Edition

#16
post #9

Earlier quoted context omitted.

When you have an LLVM frontend, what you generally do is have your driver run the optimization and code generation steps itself using the LLVM APIs rather than using opt/llc binaries to drive this step. That way, you don't need the LLVM binaries, just the libraries that you statically link into your executable. For example, all of the code in clang to do this is located in https://github.com/llvm/llvm-project/blob/ma…

What if my frontend is written in non cpp? e.g haskell, js, java, c#, etc.

You use the LLVM-C bindings via your favorite FFI mechanism to generate the code then, usually.

Re: How to learn compilers: LLVM Edition

#17
post #4

since here's many compiler hackers then I'd want to ask question: How do you distribute your frontend with LLVM? Let's say that I have lexer, parser and emitter written in e.g Haskell (random example) I emit LLVM IR and then I use LLVM to generate something other but the problem is, that I need to have LLVM binaries and I'd rather avoid telling people that want to contribute to my OSS project to install LLVM because…

In the Julia world, we make redistributable binaries for all sorts of things; you can find lots of packages here [0], and for LLVM in particular (which Julia uses to do its codegen) you can find _just_ libLLVM.so (plus a few supporting files) here [1]. If you want a more fully-featured, batteries-included build of LLVM, check out this package [2].

When using these JLL packages from Julia, it will automatically download and load in dependencies, but if you're using it from some other system, you'll probably need to manually check out the `Project.toml` file and see what other JLL packages are listed as dependencies. As an example, `LLVM_full_jll` requires `Zlib_jll` [3], since we build with support for compressed ELF sections. As you may have guessed, you can get `Zlib_jll` from [4], and it thankfully does not have any transitive dependencies.

In the Julia world, we're typically concerned with dynamic linking, (we `dlopen()` and `dlsym()` our way into all our binary dependencies) so this may not meet all your needs, but I figured I'd give it a shout out as it is one of the easier ways to get some binaries; just `curl -L $url | tar -zxv` and you're done. Some larger packages like GTK need to have environment variables set to get them to work from strange locations like the user's home directory. We set those in Julia code when the package is loaded [5], so if you try to use a dependency like one of those, you're on your own to set whatever environment variables/configuration options are needed in order to make something work at an unusual location on disk. Luckily, LLVM (at least the way we use it, via `libLLVM.so`) doesn't require any such shenanigans.

[0] https://github.com/JuliaBinaryWrappers/ [1] https://github.com/JuliaBinaryWrappers/libLLVM_jll.jl/releas... [2] https://github.com/JuliaBinaryWrappers/LLVM_full_jll.jl/rele... [3] https://github.com/JuliaBinaryWrappers/LLVM_full_jll.jl/blob... [4] https://github.com/JuliaBinaryWrappers/Zlib_jll.jl/releases [5] https://github.com/JuliaGraphics/Gtk.jl/blob/0ff744723c32c3f...

Re: How to learn compilers: LLVM Edition

#18

I have to say personally I find general program analysis (e.g. for security) a much more interesting topic than most vanilla compiler courses. For example I recently came across this course by the maintainers of soot: https://youtube.com/playlist?list=PLamk8lFsMyPXrUIQm5naAQ08a... Any pointers to similar courses much appreciated!

Anders Moeller and Michael Schwartzbach's book [1] on static program analysis is a fantastic resource, with (I think) a great balance of theory and practice. If you want to get really deep into the theory of program analysis, Patrick Cousot just published an incredibly thorough book on abstract interpretation (just got my copy this week, so haven't fully explored enough to have much of an opinion on it as a pedagogical resource)

[1] cs.au.dk/~amoeller/spa

Re: How to learn compilers: LLVM Edition

#20
post #4

since here's many compiler hackers then I'd want to ask question: How do you distribute your frontend with LLVM? Let's say that I have lexer, parser and emitter written in e.g Haskell (random example) I emit LLVM IR and then I use LLVM to generate something other but the problem is, that I need to have LLVM binaries and I'd rather avoid telling people that want to contribute to my OSS project to install LLVM because…

In the Julia world, we make redistributable binaries for all sorts of things; you can find lots of packages here [0], and for LLVM in particular (which Julia uses to do its codegen) you can find _just_ libLLVM.so (plus a few supporting files) here [1]. If you want a more fully-featured, batteries-included build of LLVM, check out this package [2]. When using these JLL packages from Julia, it will automatically downlo…

I’ll take advantage of this comment to ask the tangential question: Where can i learn how llvm “compilation” works in Julia?

I know code is only supposed to be JIT’ed and then executed by the runtime (that’s why PackageCompiler exists), but still I’d like to know more about how it works..

Like, if i write a simple pure function in Julia and call code_llvm on it… How “standalone” is the llvm code (if that is even a thing)? When does GC get called? How exactly does the generated code depend on the runtime?

Is there any good explanation of this?

Post reply on HN