Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust compiler performance

kobzol.github.io

Post reply on HN
Print view View original

91–100 of 264 posts

Re: Rust compiler performance

#91

Post by kalaksi »

johnfn wrote: post #86
Animats wrote: post #56

This isn't a huge problem. My big Rust project compiles in about a minute in release mode. Failed compiles with errors only take a few seconds. That's where most of the debugging takes place. Once it compiles, it usually works the first time.

A minute is pretty bad. I understand it may work for your use case, but there are plenty of use cases out there where errors typically don't fail the compile and a minute iteration time is a deal killer. For instance: UI work - good luck catching an incorrect color with a compile error. Vite can compile 40,000 loc and display it on your screen in probably a couple of milliseconds.

Compile what language?

Re: Rust compiler performance

#92

Post by nicoburns »

pornel wrote: post #89
swsieber wrote: post #80

Earlier quoted context omitted.

What about crates as the unit of compilation? I am genuinely curious because it's not clear to me what trade-offs there are around that decision.

It's a "unit" in the sense of calling `rustc` once, but it's not a minimal unit of work. It's not directly comparable to what C does. Rust has incremental compilation within a crate. It also splits optimization work into many parallel codegen units. The compiler front-end is also becoming parallel within crates. The advantage is that there can be common shared state (equivalent of parsing C headers) in RAM, used for…

> Rust has incremental compilation within a crate. It also splits optimization work into many parallel codegen units.

Eh, it does, but it's not currently very good at this in my experience. Nothing unfixable AFAIK (and the parallel frontend can help (but is currently a significant regression on small crates)), but currently splitting things into smaller crates can often lead to much faster compiles.

Re: Rust compiler performance

#93

Post by c-cube »

jtrueb wrote: post #38

A true champion > when I started contributing to Rust back in 2021, my primary interest was compiler performance. So I started doing some optimization work. Then I noticed that the compiler benchmark suite could use some maintenance, so I started working on that. Then I noticed that we don’t compile the compiler itself with as many optimizations as we could, so I started working on adding support for LTO/PGO/BOLT, wh…

I'm worried this person is going to experience a Yak overflow, honestly.

Re: Rust compiler performance

#94

Post by cozzyd »

1718627440 wrote: post #69
jmyeet wrote: post #28

I'm a big fan of Rust but there are definitely warts that are going to be difficult to cure [1]. This is 5 years old now but I believe it's still largely relevant. It is a weird hill to die on for C/C++ devs though, given header files and templates creating massive compile-time issues that really can't be solved. Google is known for having infrastructure for compiling large projects. They use Blaze (open-sourced at B…

I think there is a massive difference in compile times between idiomatic C and C++, so its problematic to be lumping them together. But there is also some selection bias since large projects tend to migrate from C to C++.

C++ compile times are why I often stick to C, which compiles nearly instantly.

Re: Rust compiler performance

#95

Post by WhyNotHugo »

norir wrote: post #27

Compiler performance must be considered up front in language design. It is nearly impossible to fix once the language reaches a certain size without it being a priority. I recently saw here the observation that one can often get a 2x performance improvement through optimization, but 10x requires redesigning the architecture. Rust can likely never be rearchitected without causing a disastrous schism in the community,…

One of the issue why compile times are so awful is that all dependencies must be compiled for each project.

20 different projects use the same dependency? They each need to recompile it.

This is an effect of the language not having a proper ABI for compiling libraries as dynamically loadable modules, which in itself presents many other issues, including making distribution of software a complete nightmare.

Re: Rust compiler performance

#96

Post by WhyNotHugo »

panstromek wrote: post #57
kibwen wrote: post #45

Earlier quoted context omitted.

It's certainly possible to think of language features that would preclude trivially-achievable high-performance compilation. None of those language features that are present in Rust (specifically, monomorphized generics) would have ever been considered for omission, regardless of their compile-time cost, because that would have compromised Rust's other goals.

There are many more mundane examples of language design choices in rust that are problematic for compile time. Polymorphization (which has big potential to speed up compile time) has been blocked on pretty obscure problems with TypeId. Procedural macros require double parsing. Ability to define items in function bodies prevents skipping parsing bodies. Those things are not essential, they could pretty easily be tweak…

Macros themselves are a terrible hack to work around support for proper reflection.

The entire Rust ecosystem would be reshaped in such fascinating ways if we had support for reflection. I'd love to see this happen one day.

Re: Rust compiler performance

#97

Post by kibwen »

WhyNotHugo wrote: post #95
norir wrote: post #27

Compiler performance must be considered up front in language design. It is nearly impossible to fix once the language reaches a certain size without it being a priority. I recently saw here the observation that one can often get a 2x performance improvement through optimization, but 10x requires redesigning the architecture. Rust can likely never be rearchitected without causing a disastrous schism in the community,…

One of the issue why compile times are so awful is that all dependencies must be compiled for each project. 20 different projects use the same dependency? They each need to recompile it. This is an effect of the language not having a proper ABI for compiling libraries as dynamically loadable modules, which in itself presents many other issues, including making distribution of software a complete nightmare.

> This is an effect of the language not having a proper ABI for compiling libraries as dynamically loadable modules

No, this is a design decision of Cargo to default to using project-local cached artifacts rather than caching them at the user or system level. You can configure Cargo to do so if you'd like. The reason it doesn't do this by default is because Cargo gives crates great latitude to configure themselves via compile-time flags, and any difference in flags means you get a different compiled artifact anyway. On top of that, there's the question of what `cargo clean` should do when you have a global cache rather than a local one.

Re: Rust compiler performance

#98

Post by kibwen »

WhyNotHugo wrote: post #96
panstromek wrote: post #57

Earlier quoted context omitted.

There are many more mundane examples of language design choices in rust that are problematic for compile time. Polymorphization (which has big potential to speed up compile time) has been blocked on pretty obscure problems with TypeId. Procedural macros require double parsing. Ability to define items in function bodies prevents skipping parsing bodies. Those things are not essential, they could pretty easily be tweak…

Macros themselves are a terrible hack to work around support for proper reflection. The entire Rust ecosystem would be reshaped in such fascinating ways if we had support for reflection. I'd love to see this happen one day.

> Macros themselves are a terrible hack to work around support for proper reflection.

No, I'm not sure where you got this idea. Macros are a disjoint feature from reflection. Macros exist to let you implement DSLs and abstract over syntax.

Re: Rust compiler performance

#99

Post by IshKebab »

adrian17 wrote: post #50

> On this benchmark, the compiler is almost twice as fast than it was three years ago. I think the cause of the public perception issue could be the variant of Wirth's law: the size of an average codebase (and its dependencies) might be growing faster than the compiler's improvements in compiling it?

Yeah definitely when you include dependencies. Also I've noticed that when your dependency tree gets above a certain size you end up pulling in every alternative crate for a certain task, because e.g. one of your dependencies uses miniz_oxide and another uses zlib-rs (or whatever).

On the other hand the compile to for most dependencies doesn't matter hugely because they are easy to do in parallel. It's always the last few crates and linking that take half the time.

Re: Rust compiler performance

#100

Post by kibwen »

swsieber wrote: post #80
kibwen wrote: post #45

Earlier quoted context omitted.

It's certainly possible to think of language features that would preclude trivially-achievable high-performance compilation. None of those language features that are present in Rust (specifically, monomorphized generics) would have ever been considered for omission, regardless of their compile-time cost, because that would have compromised Rust's other goals.

What about crates as the unit of compilation? I am genuinely curious because it's not clear to me what trade-offs there are around that decision.

All compilers have compilation units, there's not actually much interesting about Rust here other than using the word "crate" as a friendlier term for "compilation unit".

What you may be referring to instead is Cargo's decision to re-use the notion of a crate as the unit of package distribution. I don't think this was necessarily a bad idea (it certainly made things simpler, which matters when you're bootstrapping an ecosystem), but it's true that prevailing best practices since then have led to Rust's ecosystem having comparatively larger compilation units (which itself isn't necessarily a bad thing either; larger compilation units do tend to produce faster code). I would personally like to see Cargo provide a way to decouple the unit of distribution from the unit of compilation, which would give us free parallelism (which currently today rustc needs to tease out via parallel codegen units (and the forthcoming parallel frontend)) and also assuage some of the perpetual hand-wringing about how many crates are in a dependency tree (which is exactly the wrong measure as getting upset about how many source files are in your C program). This would be a fully backwards-compatible change.

Post reply on HN