Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A possible new back end for Rust

jason-williams.co.uk

Post reply on HN
Print view View original

171–180 of 224 posts

Re: A possible new back end for Rust

#171

Post by kohtatsu »

dwheeler wrote: post #138
gbrown_ wrote: post #119

Earlier quoted context omitted.

> For example, if you're worried that one of the compilers might be malicious, you can use the other compiler to check on it: https://dwheeler.com/trusting-trust This still requires the use of a use of trusted compiler though. Comparing two compilers arbitrarily shows if there is consensus , it does not give guarantees about correctness . From the link. In the DDC technique, source code is compiled twice: once with a…

First, I forgot to disclose: I am the author of https://dwheeler.com/trusting-trust . As discussed in detail in that dissertation, if you are using diverse double compiling to look for malicious compilers, the trusted compiler does not have to be perfect or even non-malicious. The trusted compiler could be malicious itself. The only thing you're trusting is that the trusted compiler does not have the same triggers or…

Neat. Reduces attacks to conspiracies.

Re: A possible new back end for Rust

#172

Post by dbcurtis »

ddevault wrote: post #37
hawski wrote: post #35

Earlier quoted context omitted.

I see that cproc is under quite heavy development, but qbe had last commit at the end of November. Is it considered feature complete? I heard about it some months ago and was quite interested in QBE, but it did not enjoy high tempo of changes. It may be considered advantage, I know too little to judge.

It's complete enough to compile C11 programs - to me, that's as good of a benchmark as anything. The main thing qbe is missing for cproc's purposes is inline assembly and VLAs. DWARF support would also be nice, but no one seems to care enough to do the work yet.

Are you saying that qbe does not generate any debug information, or just not DWARF format?

Re: A possible new back end for Rust

#173

Post by gsnedders »

pizlonator wrote: post #125
Someone wrote: post #107

Earlier quoted context omitted.

As an example, WebKit had an LLVM-based JavaScript optimizer in 2014 ( https://webkit.org/blog/3362/introducing-the-webkit-ftl-jit/ ), but dropped it for another one in 2016 ( https://webkit.org/blog/5852/introducing-the-b3-jit-compiler... ) In broad strokes, LLVM chooses to optimize for generating good code for statically compiled code more than for, for example, memory usage, compilation speed, or ability to dynami…

Worth noting that B3’s biggest win was higher peak throughput. It generated better code than llvm. It achieved that by having an IR that lets us be a lot more precise about things that were important to our front end compiler. It’s not even about what language you’re compiling. It’s about the IR that goes into llvm or whatever you would use instead of llvm. If that IR generally does C-like things and can only describ…

> It achieved that by having an IR that lets us be a lot more precise about things that were important to our front end compiler.

Do you have any examples off-hand? I presume caring about patchpoints and OSR is as fair gain to start with?

Re: A possible new back end for Rust

#174

Post by pizlonator »

ufo wrote: post #164
msla wrote: post #151

Earlier quoted context omitted.

When you emit C, you're limited by C, at least if you want to emit C as opposed to inline assembly wrapped in C. For example, it's harder to have a function return more than one value in C than it is in most architectures, you can't do things with processor flags (on architectures which have them), you're at the mercy of the C compiler's optimizer as to vectorization and loop unrolling, you can't always preserve sema…

I think we can all agree that LLVM IR is a more powerful compilation target than C. However, what Pizlo was saying is that generating C can be simpler than generating LLVM IR. A bunch of printfs can get you very far.

There are very few things that you can say in LLVM IR that you can't say in C.

Re: A possible new back end for Rust

#175

Post by pizlonator »

gsnedders wrote: post #173
pizlonator wrote: post #125

Earlier quoted context omitted.

Worth noting that B3’s biggest win was higher peak throughput. It generated better code than llvm. It achieved that by having an IR that lets us be a lot more precise about things that were important to our front end compiler. It’s not even about what language you’re compiling. It’s about the IR that goes into llvm or whatever you would use instead of llvm. If that IR generally does C-like things and can only describ…

> It achieved that by having an IR that lets us be a lot more precise about things that were important to our front end compiler. Do you have any examples off-hand? I presume caring about patchpoints and OSR is as fair gain to start with?

And aliasing. The aliasing story in B3 is so wonderful. That was one of the biggest wins - being able to say for example that something can side-exit (and can do weird shit after exit) but doesn't write any state if it falls through.

Re: A possible new back end for Rust

#176

Post by Rochus »

yjftsjthsd-h wrote: post #167
Rochus wrote: post #73

Earlier quoted context omitted.

So if we join forces and create a reusable compiler backend so not every compiler writer has to implement the same optimizers and code generators over and over again, then this is bad because it's a monoculture? How strange is that? To me, it sounds more like political propaganda from a few idealists who want to justify why - instead of participating in a joint project - they want to develop everything themselves fro…

> So if we join forces and create a reusable compiler backend so not every compiler writer has to implement the same optimizers and code generators over and over again, then this is bad because it's a monoculture? How strange is that? Why is that strange? You now have a diverse set of frontends and a monoculture on the backend. A world with Chrome, Chromium, Edge, Brave, and the Yandex browser is still a browser engi…

The word "monoculture" means something completely different. LLVM is by definition no "monoculture" because it integrates a diverse set of function and is developed and maintained by a diverse set of people, and there is a plethora of versions and applications. With your definition each country or company would be a monoculture; and mathematics would be a monoculture because it always applies the same rules; and all fuel driven cars would be a monoculture; and electrical engineering would be a monoculture because everyone applies Maxwells laws; you name it. This misuse of terms is done with the intention of creating a negative connotation where none is justified.

Re: A possible new back end for Rust

#177

Post by nerdponx »

tick_tock_tick wrote: post #85
dtolnay wrote: post #47

Earlier quoted context omitted.

Rustc normally spends way more time in LLVM than in the frontend. Rust parsing and type checking are very fast in comparison to LLVM's codegen. Here is a chart from last September showing where the time goes in compiling a large Rust codebase (rustc itself): https://gistpreview.github.io/?74d799739504232991c49607d5ce7... (Scroll down to the large horizontal bars once dependencies have been built.) (Sorry if GitHub is…

Rust is famous for throwing garbage IR at LLVM and hoping it cleans it all up. They've made a lot of progress but comparing the timing is very misleading when the work is intentionally offloaded to LLVM.

It sounds like this is addressed in the post, no? LLVM IR doesn't always map cleanly to Rust representations, so a backend that supports a cleaner mapping should obviate the need for more frontend optimization.

I'm basically a layperson when it comes to this topic, but that's my understanding of one of the potential benefits of a different backend.

Re: A possible new back end for Rust

#178

Post by pizlonator »

temac wrote: post #148
pizlonator wrote: post #124

Earlier quoted context omitted.

Llvm makes some questionable choices about how to do SSA, alias analysis, register allocation, and instruction selection. Also it goes all in on UB optimizations even when experience from other compilers shows that it’s not really needed. Maybe those choices are really fundamental and there is no escaping them to get peak perf - but you’re not going to know for sure until folks try alternatives. Those alternatives li…

I dislike UB, but I do at language level. When LLVM is reached, UB can only have and only be continued to be removed, never added (from a global point of view, applying general as-if rules a compiler can always generate its own boilerplate in which it knows something can not happen, then maybe latter leverage "UB" to e.g. trim impossible paths, that are really impossible in this case -- at least barring other languag…

It is true that compilers sometimes have to have operations that have a semantics that are defined only if some conditions hold. But LLVM's and C's interpretation of what happens when the conditions don't hold is extraordinarily liberal and I'm not sure that is either beneficial or sane.

Like, LLVM tries not to add UB, but design choices it made to support optimization with UB do sometimes result in new UB being introduced, like the horror show that happens with `undef` and code versioning.

So, I think that optimizing with UB internally is fine but only if it's some kind of bounded UB where you promise something stronger than nasal demons.

Re: A possible new back end for Rust

#179

Post by pcwalton »

Rusky wrote: post #144
edflsafoiewq wrote: post #59

Earlier quoted context omitted.

I wonder how much of it is just code style. A simple for-loop in C is probably going to be an iterator blob in Rust with an order of magnitude more code for the backend to chew through.

Part of it may be code style, but another part comes from just how much LLVM IR the frontend generates for just about any code style. Part of the reason LLVM runs so much faster on C than on Rust is that Clang is smarter about generating less/better IR from the start, so LLVM's optimizer has less of a hole to dig itself out of.

I think it's actually mostly the language. Rust idioms just generate a lot of code.

Clang is in a worse spot than rustc is in terms of emitting good LLVM IR, since it has no IR aside from the AST. By contrast, rustc has MIR which is more amenable to optimizations. At this point I'm fairly sure the problem is just that C code naturally generates less IR than Rust code does. All those function calls that go into iterators, array indexing, containers, etc. etc. add up quick.

Post reply on HN