Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A possible new back end for Rust

jason-williams.co.uk

Post reply on HN
Print view View original

61–70 of 224 posts

Re: A possible new back end for Rust

#61

Post by tadfisher »

Waterluvian wrote: post #57

When writing a language like Rust, is the biggest challenge simply deciding what Rust's features and behaviors should be? And implementing the syntax and Rust -> LLVM compiler is really just a chore for the individuals who are super familiar with the implementation of these languages? Or is the technical implementation also genuinely challenging and non-obvious?

A language like Rust aims for "zero-cost abstractions", which means the features and behaviors of the language must be evaluated in the context of the implementation.

Re: A possible new back end for Rust

#62

Post by Waterluvian »

kevinmgranger wrote: post #60
Waterluvian wrote: post #57

When writing a language like Rust, is the biggest challenge simply deciding what Rust's features and behaviors should be? And implementing the syntax and Rust -> LLVM compiler is really just a chore for the individuals who are super familiar with the implementation of these languages? Or is the technical implementation also genuinely challenging and non-obvious?

The concept of lifetime management is relatively novel and uncharted territory, if I understand correctly. There's only some prior art. So implementing that must have been an adventure and a half. And while I'm sure the folks who work on these languages are wonderfully intelligent people, let's dispel this notion that you need to be a super genius to implement a compiler or something like that! It seems magical, like…

I walked through the Java portion of Crafting Interpreters and indeed, they can be much simpler than you might imagine in your early years. I was more just paying a compliment than suggesting that maintainers are superhuman. Everyone who ships productive code is a wizard and you can be too!

I modified my original question to avoid a potential distraction from what I want to talk about. Thanks!

Re: A possible new back end for Rust

#63

Post by mratsim »

Ididntdothis wrote: post #29
Koshkin wrote: post #5

I've been wondering lately if the modern compilers should all be using C as the intermediate language (or some language-specific code optimization opportunities could be lost if they do that).

That’s how C++ started out but as far as I know this had lots of limitations in terms of optimization so they started writing native C++ compilers.

I would be surprised if optimization was the actual goal. C++ is not faster than C.

Re: A possible new back end for Rust

#64

Post by Waterluvian »

tadfisher wrote: post #61
Waterluvian wrote: post #57

When writing a language like Rust, is the biggest challenge simply deciding what Rust's features and behaviors should be? And implementing the syntax and Rust -> LLVM compiler is really just a chore for the individuals who are super familiar with the implementation of these languages? Or is the technical implementation also genuinely challenging and non-obvious?

A language like Rust aims for "zero-cost abstractions", which means the features and behaviors of the language must be evaluated in the context of the implementation.

Let me attempt to unpack this into language I think I understand:

It's up to Rust's compiler to verify all the contracts made, because in order for the binary to be "zero-cost", none of those checks are being done at runtime. If you looked at the output assembly, you would see what looks like very carefully written code that shows no explicit signs of protecting itself. Ie. there's no swaths of boilerplate assembly doing borrow checking, out of bounds checking, etc.

Re: A possible new back end for Rust

#65

Post by mratsim »

steveklabnik wrote: post #18
Koshkin wrote: post #5

I've been wondering lately if the modern compilers should all be using C as the intermediate language (or some language-specific code optimization opportunities could be lost if they do that).

It's a sad thing that you've been downvoted for posting a thought. Others have already said the drawbacks of this idea, but there are also pros.

Indeed. Time-to-market is an obvious one and esoteric platform support, and maybe also debuggability.

Re: A possible new back end for Rust

#66

Post by steveklabnik »

Waterluvian wrote: post #57

When writing a language like Rust, is the biggest challenge simply deciding what Rust's features and behaviors should be? And implementing the syntax and Rust -> LLVM compiler is really just a chore for the individuals who are super familiar with the implementation of these languages? Or is the technical implementation also genuinely challenging and non-obvious?

First of all, deciding features and behaviors is not simple. :)

There are a number of technical implementation challenges in the compiler.

It is a large project, and Rust's got a really intense stability policy.

The compiler was bootstrapped very early, when the rate of change of the language itself was still "multiple things per day." This introduced significant architectural debt.

There have been multiple projects that have re-written massive parts of the compiler, and more ongoing. For example, non-lexical lifetimes required inventing an entire additional intermediate language, re-writing the compiler to use it, and making sure that everything kept working while doing so.

More recently, the compiler has been being re-done from a classic, multiple-pass architecture to a more Roslyn, "query-based" one. Again, this is being done entirely "in-flight", while keeping a project that's used by a lot of folks stable. The rust-analyzer has made this project even more interesting; a "librarification" strategy is being undergone to make the compiler more modular.

For some numbers on this kind of thing, https://twitter.com/steveklabnik/status/1211667962379276288/... and https://twitter.com/steveklabnik/status/1211717308143587334/...

Re: A possible new back end for Rust

#67

Post by Waterluvian »

steveklabnik wrote: post #66
Waterluvian wrote: post #57

When writing a language like Rust, is the biggest challenge simply deciding what Rust's features and behaviors should be? And implementing the syntax and Rust -> LLVM compiler is really just a chore for the individuals who are super familiar with the implementation of these languages? Or is the technical implementation also genuinely challenging and non-obvious?

First of all, deciding features and behaviors is not simple. :) There are a number of technical implementation challenges in the compiler. It is a large project, and Rust's got a really intense stability policy. The compiler was bootstrapped very early, when the rate of change of the language itself was still "multiple things per day." This introduced significant architectural debt. There have been multiple projects…

Using the word "simply" is a quirk of mine. I'm very much trying to express that I think it's by far the hardest part. Thanks for your response!

Re: A possible new back end for Rust

#68

Post by Ididntdothis »

mratsim wrote: post #63
Ididntdothis wrote: post #29

Earlier quoted context omitted.

That’s how C++ started out but as far as I know this had lots of limitations in terms of optimization so they started writing native C++ compilers.

I would be surprised if optimization was the actual goal. C++ is not faster than C.

It’s not faster but it’s also not much slower. I think some features of C++ Would be hard to optimize by a compiler that doesn’t understand them so a C++ to C compiler may produce slow or bloated code.

Re: A possible new back end for Rust

#69

Post by JoshTriplett »

CJefferson wrote: post #39
ddavis wrote: post #19

Earlier quoted context omitted.

It's unfair to rms to say that. He would be happy for corporations to use and contribute to any project associated with the GNU project (like GCC), if everyone wanted to play along in GPL land (which of course isn't reality).

Unfortunately not. For years people wanted gcc to output a nice parse tree for C++, which would have been plenty useful for open source text editors, but was banned by RMS as it would also be useful for closed source systems.

This was a legitimate concern, which helped in the days GCC originated in, and hurt later on. Parsing C and C++ well and doing intermediate code generation and optimization was the hard part; taking the result and generating code for a target architecture might well have been proprietary for many architectures if it had been allowed to be, in the era when UNIX and other OS vendors were fighting against each other for every scrap of differentiation. And frontends for some languages would have been proprietary, as well.

LLVM's extensible architecture was its most critical property; its permissive license is an unfortunate side effect of a rewrite.

If GCC had come up with the "GCC Runtime Library Exception" way back in the day, and provided a modular architecture, half the innovation happening around LLVM might have happened around GCC instead. (Might, not "would have"; we can only speculate on what alternate history might have occurred.)

Re: A possible new back end for Rust

#70

Post by steveklabnik »

edflsafoiewq wrote: post #59
aw1621107 wrote: post #48

Earlier quoted context omitted.

I think you might be surprised just how much time is spent in codegen- and optimization-related code. For example, a bit over 75% of the time needed to compile the regex crate can be attributed to codegen- and optimization-related events, with a bit over 64% of that time spent in LLVM-related events specifically [0]. Granted, I'm not certain whether this is a release or debug build, but it does show that there is roo…

I wonder how much of it is just code style. A simple for-loop in C is probably going to be an iterator blob in Rust with an order of magnitude more code for the backend to chew through.

We can compare them! https://godbolt.org/z/-Fzuqs

(My screen is small so it's tough for me to read these results, to be honest...)

Post reply on HN