Live data from Hacker News

A possible new back end for Rust

jason-williams.co.uk

181–190 of 224 posts

Re: A possible new back end for Rust

#181
post #164

Earlier quoted context omitted.

I think we can all agree that LLVM IR is a more powerful compilation target than C. However, what Pizlo was saying is that generating C can be simpler than generating LLVM IR. A bunch of printfs can get you very far.

There are very few things that you can say in LLVM IR that you can't say in C.

Except for _e.g._ memory alignment, or types of a given width (until C99).

Re: A possible new back end for Rust

#182

Earlier quoted context omitted.

There are very few things that you can say in LLVM IR that you can't say in C.

Except for _e.g._ memory alignment, or types of a given width (until C99).

For lots of us the extensions that gcc and clang support are part of C. So, a backend that emits C would get access to those things if it wanted.

Re: A possible new back end for Rust

#183

There wouldn't be any surprises, or cognitive dissonance, from using very different paths for debug versus release builds? On a small project, personally I use --release sometimes during development because the compile time doesn't matter that much and the resulting executable is much faster: if I don't use --release I can get a misleading sense of UX during development.

This already happens a bunch, even with the current setups. It's very natural if you come from a compiled language, and not if you don't. The first step of someone saying "hey why is Rust slow?" is five people replying "did you use --release".

Can confirm.

I had a graph traversal program written in Python. I ported it to Rust, and the runtime was identical -- 68.4 seconds, down to the tenth of a second. (Kinda blew my mind -- I had to triple check that I was running and timing what I thought I was!) I had a bit of a crisis of faith.

I poked at it a few times over the next week, then finally got on the IRC channel and quickly received the advice mentioned above. Same input, with --release: 6.2 seconds.

Re: A possible new back end for Rust

#184

Earlier quoted context omitted.

> it's harder to have a function return more than one value in C than it is in most architectures Biggest issue is the cultural aversion to returning structs and tagged unions.

And it's not even hard , just ugly. Which is much less of a problem for a compiler IR.

That’s true. If you’re doing a lot of that then llvm ir might be nicer. Matter of taste of course.

Re: A possible new back end for Rust

#185

Earlier quoted context omitted.

Historically writing a compiler in the language that you’re promoting is a good way to really understand the limitations of your language. I think this works so well because language designers tend to understand compilers better than they understand other software.

I heard Niklaus Wirth would only allow new compiler optimizations (in his compilers for Pascal, Oberson, Modula-2) that proved themselves by speeding up the compiler itself.

The specific metric, IIRC, is self-compilation of the compiler. Adding optimizations to the compiler needed to speed up compilation of the compiler more than their added complexity slowed down compilation of the compiler.

Re: A possible new back end for Rust

#186

Earlier quoted context omitted.

> it's harder to have a function return more than one value in C than it is in most architectures Biggest issue is the cultural aversion to returning structs and tagged unions.

And it's not even hard , just ugly. Which is much less of a problem for a compiler IR.

Some syntactic sugar using anonymous structs and type inference would probably fix the ugly.

Re: A possible new back end for Rust

#187
post #93

Earlier quoted context omitted.

I still remember when Clang bringing LLVM along was seen as SO OUT THERE and I'm just mentioning it because I find it weird to be old enough to see fads in system languages come and start to go. Just curious, do you have any examples of this "limitations" you speak of? Sounds like a very interesting read.

Llvm makes some questionable choices about how to do SSA, alias analysis, register allocation, and instruction selection. Also it goes all in on UB optimizations even when experience from other compilers shows that it’s not really needed. Maybe those choices are really fundamental and there is no escaping them to get peak perf - but you’re not going to know for sure until folks try alternatives. Those alternatives li…

> Llvm makes some questionable choices about how to do SSA, alias analysis, register allocation, and instruction selection.

Do you mind expanding more on these points or directing me to some places where I can learn more about them? Compilers are a fairly new field for me, so anything I can learn about their design decisions and tradeoffs are worth their weight in gold.

Re: A possible new back end for Rust

#188
post #40

Earlier quoted context omitted.

Why phrase it as "other than to prove it can be done" if you already know there are good answers? I think the following obviously do apply: 1) much easier for Rust community to contribute to the compiler from end-to-end. 2) lower coordination cost with LLVM giving complete, Rust-focussed control over code generation/optimisation. Think about e.g. fixing noalias. 3) lower maintenance cost for LLVM integration/fork. It…

Because I don't think the possible good answers apply. Sure it is harder to contribute to the backend, but does it matter? I've been doing c++ for years and never looked at the backend. I'll grant lower coordination costs. However I believe they are not outweighed by the advantages of the other llvm contributions. If they need to fork llvm that is a problem. Either merge it back in and be done (with some tests so wha…

Lower coordination cost is a big deal. Having your own backend means you can do frontend-backend codesign. You can implement language specific optimizations in the backend. Those things are not in the cards if you’re using llvm. (I mean they might be, but unless you fork, the time it’ll take for the changes to make it into llvm will be comparable to the time it takes to write your own backend.)

Re: A possible new back end for Rust

#189
post #144

Earlier quoted context omitted.

Part of it may be code style, but another part comes from just how much LLVM IR the frontend generates for just about any code style. Part of the reason LLVM runs so much faster on C than on Rust is that Clang is smarter about generating less/better IR from the start, so LLVM's optimizer has less of a hole to dig itself out of.

I think it's actually mostly the language. Rust idioms just generate a lot of code. Clang is in a worse spot than rustc is in terms of emitting good LLVM IR, since it has no IR aside from the AST. By contrast, rustc has MIR which is more amenable to optimizations. At this point I'm fairly sure the problem is just that C code naturally generates less IR than Rust code does. All those function calls that go into iterat…

Modern C++ is in a similar situation with iterators, indexing, etc. and it's still often faster for LLVM to get through.

MIR optimization can help close the gap but that still involved generating "garbage" that takes extra time to codegen (in Debug) or optimize out (in Release).

Being smarter about generating IR, whether MIR or LLVM IR, is still an area rustc has a lot of room to improve, even given Rust's idioms. E.g. stuff like this: https://github.com/rust-lang/rust/issues/69715

Re: A possible new back end for Rust

#190
post #90
post #38

Novel compiler backends are a super cool idea, but I don't think it's going to help Rust compile speeds as much as this posts suggests. The complexity of Rust's type system puts a pretty high lower bound on compile times because of work the front end needs to do. Plain C compiles quickly even with an LLVM backend, for example.

Haskell, OCaml, SML, Idris also compile quite fast, with complex type systems. Their secret? Multiple backends with different kinds of optimizations. You don't need to compile for the ultimate release performance when in the middle of compile-debug-edit cycle.

Haskell and Idris both have very slow compilation time.
Post reply on HN