Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust compiler performance

kobzol.github.io

Post reply on HN
Print view View original

101–110 of 264 posts

Re: Rust compiler performance

#101

Post by mrec »

c-cube wrote: post #93
jtrueb wrote: post #38

A true champion > when I started contributing to Rust back in 2021, my primary interest was compiler performance. So I started doing some optimization work. Then I noticed that the compiler benchmark suite could use some maintenance, so I started working on that. Then I noticed that we don’t compile the compiler itself with as many optimizations as we could, so I started working on adding support for LTO/PGO/BOLT, wh…

I'm worried this person is going to experience a Yak overflow, honestly.

Coincidentally, I discovered this glorious page literally five minutes ago:

https://github.com/SerenityOS/yaksplained?tab=readme-ov-file...

Re: Rust compiler performance

#102

Post by zozbot234 »

kibwen wrote: post #70
panstromek wrote: post #57

Earlier quoted context omitted.

There are many more mundane examples of language design choices in rust that are problematic for compile time. Polymorphization (which has big potential to speed up compile time) has been blocked on pretty obscure problems with TypeId. Procedural macros require double parsing. Ability to define items in function bodies prevents skipping parsing bodies. Those things are not essential, they could pretty easily be tweak…

This is an oversimplification. Automatic polymorphization is blocked on several concerns, e.g. dyn safety (and redesigning the language to make it possible to paper over the difference between dyn and non-dyn safe traits imposes costs on the static use case), and/or obscure LLVM implementation deficiencies (which was the blocker for the last time I proposed a Swift-style ABI to address this). Procedural macros don't…

AIUI, "Swift-style" ABI mechanisms are heavily dependent on alloca (dynamically-sized allocations on the stack) which the Rust devs have just proposed backing out of a RFC for (i.e. give up on it as an approved feature for upcoming versions of Rust) because it's too complex to implement, even with existing LLVM support for it.

Re: Rust compiler performance

#103

Post by FridgeSeal »

kalaksi wrote: post #91
johnfn wrote: post #86

Earlier quoted context omitted.

A minute is pretty bad. I understand it may work for your use case, but there are plenty of use cases out there where errors typically don't fail the compile and a minute iteration time is a deal killer. For instance: UI work - good luck catching an incorrect color with a compile error. Vite can compile 40,000 loc and display it on your screen in probably a couple of milliseconds.

Compile what language?

Probably js given they mentioned Vite; not exactly sure I’d call it “compiling” in nearly t he r same order of magnitude of complexity though…

Re: Rust compiler performance

#104

Post by CrendKing »

kibwen wrote: post #97
WhyNotHugo wrote: post #95

Earlier quoted context omitted.

One of the issue why compile times are so awful is that all dependencies must be compiled for each project. 20 different projects use the same dependency? They each need to recompile it. This is an effect of the language not having a proper ABI for compiling libraries as dynamically loadable modules, which in itself presents many other issues, including making distribution of software a complete nightmare.

> This is an effect of the language not having a proper ABI for compiling libraries as dynamically loadable modules No, this is a design decision of Cargo to default to using project-local cached artifacts rather than caching them at the user or system level. You can configure Cargo to do so if you'd like. The reason it doesn't do this by default is because Cargo gives crates great latitude to configure themselves vi…

Why can't Cargo have a system like PyPI where library author uploads compiled binary (even with their specific flags) for each rust version/platform combination, and if said binary is missing for certain combination, fallback to local compile? Imagine `cargo publish` handle the compile+upload task, and crates.io be changed to also host binaries.

Re: Rust compiler performance

#105

Post by maccard »

sapiogram wrote: post #82
maccard wrote: post #60

Earlier quoted context omitted.

I work on large c++ code bases day in day out - think 30 minute compiles on an i9 with 128GB ram and NVMe drives. Rusts compile times are still ungodly slow. I contributed to a “small to medium” open source project [0] a while back, fixing a few issues that we came across when using it. Given that the project is approximately 3 orders of magnitude smaller than my day to day project, a clean build of a few thousand li…

Thanks for actually including the slow repo in your comment. My results on a Ryzen 5900X: * Clean debug build: 1m 22s * Incremental debug build: 13s * Clean release build: 1m 51s * Incremental release build: 24s Incremental builds were done by changing one line in creates/symbolicator/src/cli.rs. It's not great, but it sounds like your experience was much worse for some reason.

Sorry - my clean build was actually including the dependency fetching, which is a large part of it. My experience was in 2023 which if we go by article roughly scales with compiler performance to 5 minutes or so

Re: Rust compiler performance

#106

Post by maccard »

afdbcreid wrote: post #75
maccard wrote: post #60

Earlier quoted context omitted.

I work on large c++ code bases day in day out - think 30 minute compiles on an i9 with 128GB ram and NVMe drives. Rusts compile times are still ungodly slow. I contributed to a “small to medium” open source project [0] a while back, fixing a few issues that we came across when using it. Given that the project is approximately 3 orders of magnitude smaller than my day to day project, a clean build of a few thousand li…

What are incremental compile times with the C++ codebase? Also, does the line of code you count include dependencies (admitting, dependencies in Rust are a problem, but it's not related to compiler performance)?

About 15 seconds, because we carve it up into 100 or so dlls specifically for this case

Re: Rust compiler performance

#107

Post by fngjdflmdflg »

norir wrote: post #27

Compiler performance must be considered up front in language design. It is nearly impossible to fix once the language reaches a certain size without it being a priority. I recently saw here the observation that one can often get a 2x performance improvement through optimization, but 10x requires redesigning the architecture. Rust can likely never be rearchitected without causing a disastrous schism in the community,…

This was a big reason for dart canceling its previous macros attempt (as I understand it). Fast compilation is integral for Flutter development - which accounts for a late percentage of dart usage - so after IIRC more than two years of developing it they still ended up not going through with that iteration of macros because it would make hot reload too slow. That degree of level-headedness and consideration is worthy of respect IMO.

Re: Rust compiler performance

#108

Post by cesarb »

CrendKing wrote: post #104
kibwen wrote: post #97

Earlier quoted context omitted.

> This is an effect of the language not having a proper ABI for compiling libraries as dynamically loadable modules No, this is a design decision of Cargo to default to using project-local cached artifacts rather than caching them at the user or system level. You can configure Cargo to do so if you'd like. The reason it doesn't do this by default is because Cargo gives crates great latitude to configure themselves vi…

Why can't Cargo have a system like PyPI where library author uploads compiled binary (even with their specific flags) for each rust version/platform combination, and if said binary is missing for certain combination, fallback to local compile? Imagine `cargo publish` handle the compile+upload task, and crates.io be changed to also host binaries.

> Why can't Cargo have a system like PyPI where library author uploads compiled binary

Unless you have perfect reproducible builds, this is a security nightmare. Source code can be reviewed (and there are even projects to share databases of already reviewed Rust crates; IIRC, both Mozilla and Google have public repositories with their lists), but it's much harder to review a binary, unless you can reproducibly recreate it from the corresponding source code.

Re: Rust compiler performance

#109

Post by dvt »

codr7 wrote: post #53
bobbylarrybobby wrote: post #51

Earlier quoted context omitted.

Not to be that guy who comes to Rust’s defense whenever Go is mentioned, but... Rust protects from a much larger class of errors than just memory safety. For instance, it is impossible to invalidate an iterator while iterating over it, refer to an unset or invalid value, inadvertently merely shallow copy a variable, or forget to lock/unlock a mutex.

If only these were common problems that were difficult to otherwise avoid.

I like Rust, but I think this post is unfairly downvoted. Rustaceans often annoyingly point out that "you can't use super-common-footgun X with Rust!" which, while true, they also omit the compromises made are immense (frankly, compiler performance is one of them).

Re: Rust compiler performance

#110

Post by adrian17 »

kibwen wrote: post #98
WhyNotHugo wrote: post #96

Earlier quoted context omitted.

Macros themselves are a terrible hack to work around support for proper reflection. The entire Rust ecosystem would be reshaped in such fascinating ways if we had support for reflection. I'd love to see this happen one day.

> Macros themselves are a terrible hack to work around support for proper reflection. No, I'm not sure where you got this idea. Macros are a disjoint feature from reflection. Macros exist to let you implement DSLs and abstract over syntax.

They are disjoint, but the things you can use them for overlap a lot. In particular, I'd dare say a majority of existing `#[derive]`-style macros might be easier to implement in a hypothetical reflection layer instead.

Instead of taking a raw token stream of a struct, parsing it with `syn` (duplicating the work the compiler does later), generating the proper methods and carefully generating trait checks for the compiler to check in a later phase (for example, `#[derive(Eq)] struct S(u16)` creates an invisible never-called method just to do `let _: ::core::cmp::AssertParamIsEq;` so the compiler can show an error 20s after an incorrectly used macro finished), just directly iterate fields and check `field.type.implements_trait(Eq)` inside the derive macro itself.

That said, that's just wishful thinking - with how complex trait solving is, supporting injecting custom code in the middle of it (checking existing traits and adding new trait impls) might make compile time even worse, assuming it's even possible at all. It’s also not a clear perf win if a reflection function were to run on each instantiation of a generic type.

Post reply on HN