Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A possible new back end for Rust

jason-williams.co.uk

Post reply on HN
Print view View original

111–120 of 224 posts

Re: A possible new back end for Rust

#111

Post by jfkebwjsbx »

fluffything wrote: post #77
bluGill wrote: post #34

Earlier quoted context omitted.

Why? Other than to prove it can be done what is the point. If rust was a huge community okay, but face it, they are not. It is better therefore to focus their efforts where they can make a difference. A new x where the existing ones are just fine (this includes well maintained) is a waste of resources. There are many possible good answers to the above question. However I'm not sure they apply, and worse I believe the…

Cranelift - the compiler toolchain being discussed in this post (previously known as Cretonne) - is actually completely written in Rust, being developed (obviously) by Rust programmers, that are members of the Rust community. Its development started at Mozilla, which still employs some of its developers to work on it full-time. So.. the claim that the Rust community is not big enough to achieve this is wrong, since t…

> LLVM is not fine: it is super _super_ slow

Source? LLVM is fast for what it does.

What people usually complain about is rustc being slow overall, not the LLVM passes.

Re: A possible new back end for Rust

#112

Post by nitwit005 »

int_19h wrote: post #103
tick_tock_tick wrote: post #85

Earlier quoted context omitted.

Rust is famous for throwing garbage IR at LLVM and hoping it cleans it all up. They've made a lot of progress but comparing the timing is very misleading when the work is intentionally offloaded to LLVM.

Isn't that kind of the point of having a relatively high-level backend - to avoid the need for every front-end to do the same tedious optimizations?

You do have to have some sort of balance. With a large enough input, any program will become slow.

Re: A possible new back end for Rust

#113

Post by dtolnay »

tick_tock_tick wrote: post #85
dtolnay wrote: post #47

Earlier quoted context omitted.

Rustc normally spends way more time in LLVM than in the frontend. Rust parsing and type checking are very fast in comparison to LLVM's codegen. Here is a chart from last September showing where the time goes in compiling a large Rust codebase (rustc itself): https://gistpreview.github.io/?74d799739504232991c49607d5ce7... (Scroll down to the large horizontal bars once dependencies have been built.) (Sorry if GitHub is…

Rust is famous for throwing garbage IR at LLVM and hoping it cleans it all up. They've made a lot of progress but comparing the timing is very misleading when the work is intentionally offloaded to LLVM.

My comment is in response to "super cool idea, but I don't think it's going to help Rust compile speeds". Even a compiler that emits garbage IR from the frontend would get 20x faster with a magic instant backend if 95% of time is currently spent in the backend.

A magic instant backend is unrealistic, so Rust will need to move some of the current backend work to frontend work for things that can be done more efficiently on the frontend. But the fact remains that there is an opportunity for big improvement from a much faster backend.

Re: A possible new back end for Rust

#114

Post by steveklabnik »

dwheeler wrote: post #110

One cool advantage of having multiple compilers for a language is that you can use one as a check on the other. For example, if you're worried that one of the compilers might be malicious, you can use the other compiler to check on it: https://dwheeler.com/trusting-trust Even if you're not worried about malicious compilers, you can generate code, compiled it against multiple compilers, and sending inputs and see when…

Yep! This is a very good property, and part of why mrustc is a big deal.

Re: A possible new back end for Rust

#115

Post by [deleted] »

j88439h84 wrote: post #97
steveklabnik wrote: post #66

Earlier quoted context omitted.

First of all, deciding features and behaviors is not simple. :) There are a number of technical implementation challenges in the compiler. It is a large project, and Rust's got a really intense stability policy. The compiler was bootstrapped very early, when the rate of change of the language itself was still "multiple things per day." This introduced significant architectural debt. There have been multiple projects…

> Rust's got a really intense stability policy. I know the code won't stop running, but I wonder how soon it stops being idiomatic. If it's not idiomatic, it's harder to maintain due to unfamiliar style and structure. Does Rust have measures to deal with this issue?

[deleted]

Re: A possible new back end for Rust

#116

Post by steveklabnik »

j88439h84 wrote: post #97
steveklabnik wrote: post #66

Earlier quoted context omitted.

First of all, deciding features and behaviors is not simple. :) There are a number of technical implementation challenges in the compiler. It is a large project, and Rust's got a really intense stability policy. The compiler was bootstrapped very early, when the rate of change of the language itself was still "multiple things per day." This introduced significant architectural debt. There have been multiple projects…

> Rust's got a really intense stability policy. I know the code won't stop running, but I wonder how soon it stops being idiomatic. If it's not idiomatic, it's harder to maintain due to unfamiliar style and structure. Does Rust have measures to deal with this issue?

I think the closest thing is enforced rustfmt. I don't hack on the compiler though, so maybe there's some stuff that the team does that they don't broadcast super widely.

Re: A possible new back end for Rust

#117

Post by stephencanon »

raphlinus wrote: post #98
stephencanon wrote: post #36

Earlier quoted context omitted.

If you're trying to implement a language with substantially different semantics from C (e.g. a substantially different memory model, or without UB) the semantics of C make it really unsuitable as an IR. You can't use C's casts (undef for out of range float -> int conversions, for example), arithmetic (undef for signed overflow), or shift operators (implementation-defined behavior for signed right shifts, undefined be…

I agree 99.44%. The behavior of shift operations on signed integers will be fixed in C++20 and C2x, as part of the effort to require twos complement representation. It is a massive potential source of UB in currently standardized C and C++. All the other problems listed remain. [1]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p090... [2]: http://eel.is/c++draft/expr.shift#2

Even after C2x is finalized, people will be using C compilers that don't conform to C++20 and C2x for at least another decade, so you'll forgive me if I don't hold my breath =)

Re: A possible new back end for Rust

#118

Post by eximius »

Waterluvian wrote: post #64
tadfisher wrote: post #61

Earlier quoted context omitted.

A language like Rust aims for "zero-cost abstractions", which means the features and behaviors of the language must be evaluated in the context of the implementation.

Let me attempt to unpack this into language I think I understand: It's up to Rust's compiler to verify all the contracts made, because in order for the binary to be "zero-cost", none of those checks are being done at runtime. If you looked at the output assembly, you would see what looks like very carefully written code that shows no explicit signs of protecting itself. Ie. there's no swaths of boilerplate assembly d…

That might be a stronger way of putting it than I would.

The canonical example is iterator chains with complex logic compiling down into vectorised and unrolled loops. Powerful logical abstractions are used by the compiler to generate code that does what it says to do without the runtime cost of closures or whatever other logical but not mechanically necessary things you have.

Iterators can't go out of bounds, so the compiler can elide those checks. There are still some runtime costs at the intersection of safety, ergonomics, and performance. Bounds checking, overflow checking. But they have escape hatches and are relatively rare in the language. Most things do compile out.

Re: A possible new back end for Rust

#119

Post by gbrown_ »

dwheeler wrote: post #110

One cool advantage of having multiple compilers for a language is that you can use one as a check on the other. For example, if you're worried that one of the compilers might be malicious, you can use the other compiler to check on it: https://dwheeler.com/trusting-trust Even if you're not worried about malicious compilers, you can generate code, compiled it against multiple compilers, and sending inputs and see when…

> For example, if you're worried that one of the compilers might be malicious, you can use the other compiler to check on it: https://dwheeler.com/trusting-trust

This still requires the use of a use of trusted compiler though. Comparing two compilers arbitrarily shows if there is consensus, it does not give guarantees about correctness.

From the link.

    In the DDC technique, source code is compiled twice: once with a second
    (trusted) compiler (using the source code of the compiler’s parent), and then
    the compiler source code is compiled using the result of the first
    compilation. If the result is bit-for-bit identical with the untrusted
    executable, then the source code accurately represents the executable.

Re: A possible new back end for Rust

#120

Post by xscott »

kevinmgranger wrote: post #60
Waterluvian wrote: post #57

When writing a language like Rust, is the biggest challenge simply deciding what Rust's features and behaviors should be? And implementing the syntax and Rust -> LLVM compiler is really just a chore for the individuals who are super familiar with the implementation of these languages? Or is the technical implementation also genuinely challenging and non-obvious?

The concept of lifetime management is relatively novel and uncharted territory, if I understand correctly. There's only some prior art. So implementing that must have been an adventure and a half. And while I'm sure the folks who work on these languages are wonderfully intelligent people, let's dispel this notion that you need to be a super genius to implement a compiler or something like that! It seems magical, like…

Do you have any links for the prior art? I'm sincerely interested, thank you if you do. I'd also be interested in any articles (or even blog posts) that describe Rust's process for that from a compiler writer's point of view.
Post reply on HN