I've been wondering lately if the modern compilers should all be using C as the intermediate language (or some language-specific code optimization opportunities could be lost if they do that).
Doesn't Nim do that?
A possible new back end for Rust
41–50 of 224 posts
Re: A possible new back end for Rust
#42This is really great. The world needs more diverse compiler tech. The llvm monoculture is constraining what kind of compiler research folks do to just the things that are practical to do in llvm. I particularly suspect that if something like Cranelift gets evolved more then it will eventually reach throughput parity with llvm, likely without actually implementing all of the optimizations that llvm has. It shouldn’t b…
There are non-llvm compilers.
FreePascal for example has its own x86, arm, mips, sparc and powerpc backends
Re: A possible new back end for Rust
#43Thanks for nice article!
Hoping the author reads the comments, I would like to leave an, hopefully useful, feedback.
It would greatly improve the reading experience of your blog if you could make clickable the footnotes/references.
For example when you say:
> I’ve taken the chart from the 2016 MIR blog post[3]
I have to scroll to the end of the page to find the blog post (and then scroll back to resume reading). If [3] were clickable it would be great. It would be even better if [MIR blog post] were an actual link itself.
Re: A possible new back end for Rust
#44Novel compiler backends are a super cool idea, but I don't think it's going to help Rust compile speeds as much as this posts suggests. The complexity of Rust's type system puts a pretty high lower bound on compile times because of work the front end needs to do. Plain C compiles quickly even with an LLVM backend, for example.
While the type system does add to compile times, profiling generally doesn't show that it's the current limiting factor for compile times. Additionally, tools like rust-analyzer will give you type errors pretty much instantaneously, though of course that work is not finished.
Also of note, this blog post isn't speculation; they posted numbers from actually doing it.
Re: A possible new back end for Rust
#45Earlier quoted context omitted.
It's unfair to rms to say that. He would be happy for corporations to use and contribute to any project associated with the GNU project (like GCC), if everyone wanted to play along in GPL land (which of course isn't reality).
Unfortunately not. For years people wanted gcc to output a nice parse tree for C++, which would have been plenty useful for open source text editors, but was banned by RMS as it would also be useful for closed source systems.
Not sure, but I think his concern was more about the introduction of opaque steps being introduced in the compiler and becoming something people depend on. A weak analogy might be nVidia drivers on linux, imagine a new arch where part of the toolchain is a closed blob.
It turns out that hasn't happened yet with LLVM and allowing such things under LGPL may have worked.
Re: A possible new back end for Rust
#46I've been wondering lately if the modern compilers should all be using C as the intermediate language (or some language-specific code optimization opportunities could be lost if they do that).
C is a pretty lousy intermediate language:
* It's missing several useful operators, such as classic bit manipulation (count trailing zero, byteswap), or even 8- and 16-bit arithmetic. Checked arithmetic is another useful one that's not present (or even really possible in C's ABI).
* Signed integer overflow is UB.
* Utterly no support for SIMD types.
* Proper IEEE 754 floating-point control is kind of spotty, although it tends to be as bad or worse in most other languages.
* ABI control is poor. You can't come up with any way to return multiple register values, for example.
* Anything that's not a vanilla function isn't supported. No uparg function support (required for Pascal), multiple entry points (required for Fortran), or zero-cost exception handling (required for C++). Hell, even computed goto isn't actually supported.
And all of this is assuming you have strong control over how you expect implementation-defined behavior (e.g., sizeof(int)) to work.
Re: A possible new back end for Rust
#47Novel compiler backends are a super cool idea, but I don't think it's going to help Rust compile speeds as much as this posts suggests. The complexity of Rust's type system puts a pretty high lower bound on compile times because of work the front end needs to do. Plain C compiles quickly even with an LLVM backend, for example.
Rustc normally spends way more time in LLVM than in the frontend. Rust parsing and type checking are very fast in comparison to LLVM's codegen.
Here is a chart from last September showing where the time goes in compiling a large Rust codebase (rustc itself):
https://gistpreview.github.io/?74d799739504232991c49607d5ce7...
(Scroll down to the large horizontal bars once dependencies have been built.) (Sorry if GitHub is down at the moment; try later if it doesn't load.)
The blue part of each bar is time in the frontend, the purple part is time in LLVM. The largest bar (rustc) spans 105 seconds in LLVM out of 140 total, or 75% in LLVM. Many of the subcrates are even more dominated by LLVM time, for example look at rustc_metadata or rustc_traits where >95% of compile time is spent in LLVM.
Re: A possible new back end for Rust
#48Novel compiler backends are a super cool idea, but I don't think it's going to help Rust compile speeds as much as this posts suggests. The complexity of Rust's type system puts a pretty high lower bound on compile times because of work the front end needs to do. Plain C compiles quickly even with an LLVM backend, for example.
I think you might be surprised just how much time is spent in codegen- and optimization-related code.
For example, a bit over 75% of the time needed to compile the regex crate can be attributed to codegen- and optimization-related events, with a bit over 64% of that time spent in LLVM-related events specifically [0]. Granted, I'm not certain whether this is a release or debug build, but it does show that there is room for significant wins by switching backends.
As for why C can compile quickly with an LLVM backend while Rust can't, I'm not sure. I've read in the past that rustc generates pretty bad LLVM IR to pass to the backend, and it takes time for LLVM to sort through that, but there's probably some other factors in there too.
[0]: https://blog.rust-lang.org/inside-rust/2020/02/25/intro-rust...
Re: A possible new back end for Rust
#49Novel compiler backends are a super cool idea, but I don't think it's going to help Rust compile speeds as much as this posts suggests. The complexity of Rust's type system puts a pretty high lower bound on compile times because of work the front end needs to do. Plain C compiles quickly even with an LLVM backend, for example.
While the type system does add to compile times, profiling generally doesn't show that it's the current limiting factor for compile times. Additionally, tools like rust-analyzer will give you type errors pretty much instantaneously, though of course that work is not finished. Also of note, this blog post isn't speculation; they posted numbers from actually doing it.
A 30% speedup is nothing to sneeze at, but it's not putting Rust within spitting distance of Go or C for similar amounts of code.
Re: A possible new back end for Rust
#50This is really great. The world needs more diverse compiler tech. The llvm monoculture is constraining what kind of compiler research folks do to just the things that are practical to do in llvm. I particularly suspect that if something like Cranelift gets evolved more then it will eventually reach throughput parity with llvm, likely without actually implementing all of the optimizations that llvm has. It shouldn’t b…
Now it's called monoculture? Rather strange. But anyway: in your terms you're just replacing LLVM monoculture by Rust monoculture, isn't it?
Yes, it's a monoculture when the majority of all compiler work/research is happening on one compiler chain. (I feel like GCC is still competitive enough to keep up some competition, but Clang does have a lot of backing.) And yes, if we made a rust replacement and that somehow eclipsed all other compiler suites it would be a monoculture and be bad, but that's unlikely and creating an alternative to the most popular option reduces monoculture issues by adding more options.