Live data from Hacker News

A possible new back end for Rust

jason-williams.co.uk

101–110 of 224 posts

Re: A possible new back end for Rust

#101
post #6

Earlier quoted context omitted.

This is also why I think it was great that Maxime eventually graduated into GraalVM. Another tool for compiler research using modern approaches with type safe languages.

Isn't GraalVM completely tied to LLVM bitcode, and therefore has all the same problems that LLVM has ?

Isn’t that just the (née sulong) llvm frontend? IIUC GraalVM is deeply dependent on OpenJDK internals.

Re: A possible new back end for Rust

#102
post #34

Earlier quoted context omitted.

Why? Other than to prove it can be done what is the point. If rust was a huge community okay, but face it, they are not. It is better therefore to focus their efforts where they can make a difference. A new x where the existing ones are just fine (this includes well maintained) is a waste of resources. There are many possible good answers to the above question. However I'm not sure they apply, and worse I believe the…

Cranelift - the compiler toolchain being discussed in this post (previously known as Cretonne) - is actually completely written in Rust, being developed (obviously) by Rust programmers, that are members of the Rust community. Its development started at Mozilla, which still employs some of its developers to work on it full-time. So.. the claim that the Rust community is not big enough to achieve this is wrong, since t…

DMD inherited the backend from DMC++, which was the end of a long line of optimizing C and C++ compilers going back over a decade before the earliest D alphas.

Re: A possible new back end for Rust

#103
post #47

Earlier quoted context omitted.

Rustc normally spends way more time in LLVM than in the frontend. Rust parsing and type checking are very fast in comparison to LLVM's codegen. Here is a chart from last September showing where the time goes in compiling a large Rust codebase (rustc itself): https://gistpreview.github.io/?74d799739504232991c49607d5ce7... (Scroll down to the large horizontal bars once dependencies have been built.) (Sorry if GitHub is…

Rust is famous for throwing garbage IR at LLVM and hoping it cleans it all up. They've made a lot of progress but comparing the timing is very misleading when the work is intentionally offloaded to LLVM.

Isn't that kind of the point of having a relatively high-level backend - to avoid the need for every front-end to do the same tedious optimizations?

Re: A possible new back end for Rust

#104
post #93

This is really great. The world needs more diverse compiler tech. The llvm monoculture is constraining what kind of compiler research folks do to just the things that are practical to do in llvm. I particularly suspect that if something like Cranelift gets evolved more then it will eventually reach throughput parity with llvm, likely without actually implementing all of the optimizations that llvm has. It shouldn’t b…

I still remember when Clang bringing LLVM along was seen as SO OUT THERE and I'm just mentioning it because I find it weird to be old enough to see fads in system languages come and start to go. Just curious, do you have any examples of this "limitations" you speak of? Sounds like a very interesting read.

LLVM's MCJIT library is 17MB. If you have a language that you want to JIT and you thought you could embed your language like lua (Also if you want to use llvm as a backend for your project and expect to build llvm as part of a vendored package, the llvm libraries with debug symbols on my machine was about 3GB. Also not ideal.

Re: A possible new back end for Rust

#105
post #41
post #5

I've been wondering lately if the modern compilers should all be using C as the intermediate language (or some language-specific code optimization opportunities could be lost if they do that).

Doesn't Nim do that?

Yes, Nim uses C and GCC and this gave it very fast compile times and similar performance. It also runs on most devices supported by GCC.

Re: A possible new back end for Rust

#106
post #90
post #38

Novel compiler backends are a super cool idea, but I don't think it's going to help Rust compile speeds as much as this posts suggests. The complexity of Rust's type system puts a pretty high lower bound on compile times because of work the front end needs to do. Plain C compiles quickly even with an LLVM backend, for example.

Haskell, OCaml, SML, Idris also compile quite fast, with complex type systems. Their secret? Multiple backends with different kinds of optimizations. You don't need to compile for the ultimate release performance when in the middle of compile-debug-edit cycle.

From my (limited) experience, Haskell does not compile fast, especially if you’re doing something that needs lenses.

Re: A possible new back end for Rust

#107
post #93

This is really great. The world needs more diverse compiler tech. The llvm monoculture is constraining what kind of compiler research folks do to just the things that are practical to do in llvm. I particularly suspect that if something like Cranelift gets evolved more then it will eventually reach throughput parity with llvm, likely without actually implementing all of the optimizations that llvm has. It shouldn’t b…

I still remember when Clang bringing LLVM along was seen as SO OUT THERE and I'm just mentioning it because I find it weird to be old enough to see fads in system languages come and start to go. Just curious, do you have any examples of this "limitations" you speak of? Sounds like a very interesting read.

As an example, WebKit had an LLVM-based JavaScript optimizer in 2014 (https://webkit.org/blog/3362/introducing-the-webkit-ftl-jit/), but dropped it for another one in 2016 (https://webkit.org/blog/5852/introducing-the-b3-jit-compiler...)

In broad strokes, LLVM chooses to optimize for generating good code for statically compiled code more than for, for example, memory usage, compilation speed, or ability to dynamically change compiled code. That doesn’t make it optimal for JavaScript, a language that’s highly dynamic and often is used in cases where compilation time can easily dwarf execution time.

Re: A possible new back end for Rust

#108

This is really great. The world needs more diverse compiler tech. The llvm monoculture is constraining what kind of compiler research folks do to just the things that are practical to do in llvm. I particularly suspect that if something like Cranelift gets evolved more then it will eventually reach throughput parity with llvm, likely without actually implementing all of the optimizations that llvm has. It shouldn’t b…

Devil's advocate: more diverse compiler tech will mean a more fragmented community and a larger probability of divergence across implementations. People think the C compiler community is dominated by GCC and Clang, and it is, but there are literally 1000s of implementations out there in the wild. Most are necessary, because we need code generated for some obscure processor architecture that's completely proprietary,…

What about the usecase provided in the blog post, where one is used for fast debugging/dev builds but you use LLVM for production releases? Basically: why not both?

I'm guessing this could create some divergence in terms of what is supported by the compiler but I'm curious how much that would matter in reality - for day-to-day serious project development. I'm not familiar with language dev at the compiler level, so I'm curious to hear if that's practical or sane.

Re: A possible new back end for Rust

#109

Earlier quoted context omitted.

Of ones I'm a familiar with, LLVM IR is probably the best, although it has other issues of its own (in particular, floating point is done even worse than C). I'm not aware of any language which is going to beat a retargeting compiler's processor-agnostic IR. But even the "better C" languages tend to not really attempt to expand C structurally. The changes amount to fixing the egregious semantics (fixed-size types, no…

> in particular, floating point is done even worse than C Do you mind expanding on this or pointing me to places where I can read more?

There is a hidden floating-point environment that affects, and is affected by, every single floating-point instruction. Predominantly, this is rounding mode control, sticky bits, and exception control (does overflow cause a SIGFPE?), although most processors have some form of flushing denormals or treating them as 0s, which isn't in IEEE 754.

LLVM's floating point instructions assume that there is no floating point environment [1]. And there's no real facility to indicate that floating point instructions might be affected. To remedy this, they've been working on adding constrained floating point intrinsics.

[1] More specifically, that the environment is set up to the default rounding mode (round-nearest), all exceptions are masked, and no one will ever care about sticky bits.

Re: A possible new back end for Rust

#110
One cool advantage of having multiple compilers for a language is that you can use one as a check on the other.

For example, if you're worried that one of the compilers might be malicious, you can use the other compiler to check on it: https://dwheeler.com/trusting-trust

Even if you're not worried about malicious compilers, you can generate code, compiled it against multiple compilers, and sending inputs and see when they differ in the outputs. This has been used as a fuzzing technique to detect subtle errors in compilers.

Post reply on HN