Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A possible new back end for Rust

jason-williams.co.uk

Post reply on HN
Print view View original

141–150 of 224 posts

Re: A possible new back end for Rust

#141

Post by pizlonator »

bluGill wrote: post #134
pizlonator wrote: post #121

Earlier quoted context omitted.

Llvm is absolutely not the least effort for generating machine code. In many settings, it takes a fraction of the effort of integrating llvm to create a template compiler that goes straight to machine code. In many other cases, your best bet is to have your compiler emit C and then feed that to a C compiler of your choice. It’s good to have divergence. Competition is good. Otherwise people stop trying new things.

Depends on your goals. Writing a front end, optimizer and backend quickly gets to more work. I can write a c++ compiler in a few months. It won't be good and to make it good would be many many years of work. If I write a llvm backend it might take a little longer (I doubt it), but I automatically get all the optimizations llvm has plus a good front end that doesn't have bugs in obscure corner cases. (not claiming llv…

Still not as good as emitting C code in most cases? C code gets optimized using either llvm or any other optimizer so it’s a more portable compile target.

Re: A possible new back end for Rust

#142

Post by et2o »

dwheeler wrote: post #138
gbrown_ wrote: post #119

Earlier quoted context omitted.

> For example, if you're worried that one of the compilers might be malicious, you can use the other compiler to check on it: https://dwheeler.com/trusting-trust This still requires the use of a use of trusted compiler though. Comparing two compilers arbitrarily shows if there is consensus , it does not give guarantees about correctness . From the link. In the DDC technique, source code is compiled twice: once with a…

First, I forgot to disclose: I am the author of https://dwheeler.com/trusting-trust . As discussed in detail in that dissertation, if you are using diverse double compiling to look for malicious compilers, the trusted compiler does not have to be perfect or even non-malicious. The trusted compiler could be malicious itself. The only thing you're trusting is that the trusted compiler does not have the same triggers or…

Sounds fascinating. Are there real-world examples of malicious compilers?

Re: A possible new back end for Rust

#143

Post by gbrown_ »

dwheeler wrote: post #138
gbrown_ wrote: post #119

Earlier quoted context omitted.

> For example, if you're worried that one of the compilers might be malicious, you can use the other compiler to check on it: https://dwheeler.com/trusting-trust This still requires the use of a use of trusted compiler though. Comparing two compilers arbitrarily shows if there is consensus , it does not give guarantees about correctness . From the link. In the DDC technique, source code is compiled twice: once with a…

First, I forgot to disclose: I am the author of https://dwheeler.com/trusting-trust . As discussed in detail in that dissertation, if you are using diverse double compiling to look for malicious compilers, the trusted compiler does not have to be perfect or even non-malicious. The trusted compiler could be malicious itself. The only thing you're trusting is that the trusted compiler does not have the same triggers or…

Ha, I didn't even notice the username! I agree consensus (or lack thereof) is an useful property to demonstrate. I think I may have been a bit of a pedant in my prior comment.

Re: A possible new back end for Rust

#144

Post by Rusky »

edflsafoiewq wrote: post #59
aw1621107 wrote: post #48

Earlier quoted context omitted.

I think you might be surprised just how much time is spent in codegen- and optimization-related code. For example, a bit over 75% of the time needed to compile the regex crate can be attributed to codegen- and optimization-related events, with a bit over 64% of that time spent in LLVM-related events specifically [0]. Granted, I'm not certain whether this is a release or debug build, but it does show that there is roo…

I wonder how much of it is just code style. A simple for-loop in C is probably going to be an iterator blob in Rust with an order of magnitude more code for the backend to chew through.

Part of it may be code style, but another part comes from just how much LLVM IR the frontend generates for just about any code style.

Part of the reason LLVM runs so much faster on C than on Rust is that Clang is smarter about generating less/better IR from the start, so LLVM's optimizer has less of a hole to dig itself out of.

Re: A possible new back end for Rust

#145

Post by 411111111111111 »

et2o wrote: post #142
dwheeler wrote: post #138

Earlier quoted context omitted.

First, I forgot to disclose: I am the author of https://dwheeler.com/trusting-trust . As discussed in detail in that dissertation, if you are using diverse double compiling to look for malicious compilers, the trusted compiler does not have to be perfect or even non-malicious. The trusted compiler could be malicious itself. The only thing you're trusting is that the trusted compiler does not have the same triggers or…

Sounds fascinating. Are there real-world examples of malicious compilers?

i read a story about a compiler adding malware to the compiled binary once.

they kept getting owned until they supposedly found a pretty dump hack which just appended the backdoor to the final compilation on the build server...

no clue if it was just a story though, as i personally havent experienced anything like that before.

Re: A possible new back end for Rust

#146

Post by gryfft »

411111111111111 wrote: post #145
et2o wrote: post #142

Earlier quoted context omitted.

Sounds fascinating. Are there real-world examples of malicious compilers?

i read a story about a compiler adding malware to the compiled binary once. they kept getting owned until they supposedly found a pretty dump hack which just appended the backdoor to the final compilation on the build server... no clue if it was just a story though, as i personally havent experienced anything like that before.

I don't think this is what you're looking for, but Coding Machines[1] is a great little story in which the Ken Thompson hack[2] plays a role.

[1]https://www.teamten.com/lawrence/writings/coding-machines/

[2]https://www.win.tue.nl/~aeb/linux/hh/thompson/trust.html

Re: A possible new back end for Rust

#147

Post by kibwen »

jfkebwjsbx wrote: post #111
fluffything wrote: post #77

Earlier quoted context omitted.

Cranelift - the compiler toolchain being discussed in this post (previously known as Cretonne) - is actually completely written in Rust, being developed (obviously) by Rust programmers, that are members of the Rust community. Its development started at Mozilla, which still employs some of its developers to work on it full-time. So.. the claim that the Rust community is not big enough to achieve this is wrong, since t…

> LLVM is not fine: it is super _super_ slow Source? LLVM is fast for what it does. What people usually complain about is rustc being slow overall, not the LLVM passes.

> What people usually complain about is rustc being slow overall, not the LLVM passes.

The LLVM phases are usually the dominating factor in Rust compile times (the other big single contender is the linking phase). However, when the Rust developers point this out, they are also careful to mention that this may be due to the rustc frontend generating suboptimal IR as input to LLVM; we can both acknowledge that LLVM is often the bottleneck for Rust compilation while also not framing it as a failure on LLVM's part (though at the same time it is uncontroversial to state that LLVM does err on the side of superior codegen versus minimal compilation time, hence the niche that alternative compilers like Cranelift seek to fill).

Re: A possible new back end for Rust

#148

Post by temac »

pizlonator wrote: post #124
cfv wrote: post #93

Earlier quoted context omitted.

I still remember when Clang bringing LLVM along was seen as SO OUT THERE and I'm just mentioning it because I find it weird to be old enough to see fads in system languages come and start to go. Just curious, do you have any examples of this "limitations" you speak of? Sounds like a very interesting read.

Llvm makes some questionable choices about how to do SSA, alias analysis, register allocation, and instruction selection. Also it goes all in on UB optimizations even when experience from other compilers shows that it’s not really needed. Maybe those choices are really fundamental and there is no escaping them to get peak perf - but you’re not going to know for sure until folks try alternatives. Those alternatives li…

I dislike UB, but I do at language level. When LLVM is reached, UB can only have and only be continued to be removed, never added (from a global point of view, applying general as-if rules a compiler can always generate its own boilerplate in which it knows something can not happen, then maybe latter leverage "UB" to e.g. trim impossible paths, that are really impossible in this case -- at least barring other language level "real" UB). So are there really any drawback to internal exploitation of "UB" (maybe we should call it otherwise then) if for example the source language had none?

Re: A possible new back end for Rust

#149

Post by pizlonator »

cpeterso wrote: post #140
pizlonator wrote: post #126

Earlier quoted context omitted.

Historically writing a compiler in the language that you’re promoting is a good way to really understand the limitations of your language. I think this works so well because language designers tend to understand compilers better than they understand other software.

I heard Niklaus Wirth would only allow new compiler optimizations (in his compilers for Pascal, Oberson, Modula-2) that proved themselves by speeding up the compiler itself.

Hahaha that sounds excessive!

JavaScriptCore does it differently: many of our benchmarks are either interpreters or compilers written in JavaScript.

One of those benchmarks, Air, is just the stack slot coloring phase of JSC’s FTL JIT (that JIT has >90 phases) rewritten in JavaScript instead of C++. It runs like 50x slower in JS than C++ even in JSC, which wins on that test. So, probably it won’t be possible to write a JS VM in JS anytime soon. I mean, surely it’ll be possible, but it’ll also be hilariously shitty.

Re: A possible new back end for Rust

#150

Post by Leherenn »

Myrmornis wrote: post #131

There wouldn't be any surprises, or cognitive dissonance, from using very different paths for debug versus release builds? On a small project, personally I use --release sometimes during development because the compile time doesn't matter that much and the resulting executable is much faster: if I don't use --release I can get a misleading sense of UX during development.

It's funny, because I do the exact opposite.

As a developer I usually have a pretty powerful machine, and I've found that debug mode is a good way to approximate slow computers, and something that is unbearably slow in debug will bother some users later on.

Post reply on HN