Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is my Rust build so slow?

fasterthanli.me

Post reply on HN
Print view View original

181–190 of 217 posts

Re: Why is my Rust build so slow?

#181

Post by echelon »

pjmlp wrote: post #56
echelon wrote: post #20

Earlier quoted context omitted.

Does anyone have an example of a Rust Bazel build project? Or even better, a massive Rust monorepo orchestrated with Bazel build?

Not what you asked for, but maybe still interesting to look into. Android uses Soong for Rust, https://source.android.com/setup/build/rust/building-rust-mo... While Fucshia uses GN with Rust, https://fuchsia.dev/fuchsia-src/development/build/concepts/b... https://fuchsia.dev/fuchsia-src/development/languages/rust

Fantastic resources, thanks!

Re: Why is my Rust build so slow?

#182

Post by misnome »

MuffinFlavored wrote: post #169
misnome wrote: post #75

Earlier quoted context omitted.

This is more of a step-by-step deep-dive into the ways to profile, diagnose and work through the timing and performance problems. Anyone who doesn't know about many of these tools will probably find them useful (or at least useful to know that they exist) in the future. Besides, for iteration people are probably in debug, not release mode, which the article mentions is initially 19s vs 2m+. As far as I can see the ma…

> This is more of a step-by-step deep-dive into the ways to profile, diagnose and work through the timing and performance problems. I'd argue that when it comes to doing work, I only want to deep-dive into my code. Not third party libraries that "vow" to have been tested + performant, let alone the language/build tools themselves.

I'd be interested to hear what system/field/language you use with compilers that come with guarantees that they will never, ever introduce performance regressions for compile times under any circumstances (presumably with a hefty SLA to match), and where the addition of e.g. an extra innocuous header by some other part of your team will never never ever introduce unexpected nonlinear compile times that require investigations of what the compiler is spending times on.

I suspect this only applies to an extremely small portion of real world code and projects (direct hand optimised asm, perhaps something like forth?)

As far as I am aware the only "guarantees" (interpreting "vow" to mean this) with rust are that edition-bound code will continue to compile correctly, modulo bugs which may be introduced/need fixing.

Of course, it feels like rust could do better, the reputation it has hasn't come from nowhere, but, personally, it's a far cry from e.g. the projects I work on, where we are mandated (by inter-company politics) to use a custom SCons atrocity on C++ that takes two minutes to run a no-op build.

Re: Why is my Rust build so slow?

#183

Post by misnome »

estebank wrote: post #98
dwheeler wrote: post #94

Earlier quoted context omitted.

Compiler authors can often make compilation faster, but they have to work at it and think carefully through the problem. It has to be a focus not an afterthought. Years ago the GNAT Ada compiler authors intentionally avoided implementing precompiled headers. They instead focused on very fast lexical analysis (using a small handwritten lexer optimized for lower case letters because that was the usual case). By careful…

We track performance: https://perf.rust-lang.org/ I assure you it's not an afterthought. We track performance changes every week , going as far as reverting highly desired features to avoid regressions. The only reason we accept (small) regressions is for correctness fixes. This regression was only partly exercised by our perf test suite. That's how we grow it: see bad behaviour in the wild, add to the suite. The sou…

Thanks for reposting this. I recalled reading here a while ago some posts on why and how this was put into place, but I couldn't find the link.

Re: Why is my Rust build so slow?

#184

Post by pdimitar »

mplanchard wrote: post #132
lpapez wrote: post #30

Earlier quoted context omitted.

That is because the languages are different and choose different tradeoffs. The tradeoffs might make a language inapplicable in your domain. Rust has always been marketed as a "systems programming language". C++ is Rust's closes competitor in this domain space and it suffers from terribly slow compiling as well. I would say that due to headers, massive portability baggage and a lack of a standard build system, compil…

I work on a large rust project professionally, and we have tried to make build times a priority for CI. I’m just noting a couple of things that helped us a lot, in case they’re useful. - sccache is pretty much essential - beefiness: GitLab’s default runners for example are way too underpowered and made our builds take like 30 minutes (as opposed to 5 on a beefy machine in CircleCI). We’re working on switching to self…

> caching the cargo home can help. It wasn’t a massive gain relative to sccache, though. Similar story with caching the target directory

Can you please clarify this point? Caching how?

Re: Why is my Rust build so slow?

#185

Post by pdimitar »

Ar-Curunir wrote: post #175
saurik wrote: post #36

Earlier quoted context omitted.

The biggest issue cited in this article is a regression in the type checker, which is an open bug that apparently is just lower priority than whatever else the Rust team is working on instead of ensuring compilation time is fast. Only once that was worked around did they article really start having to pay attention to the code generation and linking stages, and the issues there seem to mostly be about lack of paralle…

Contrary to what you say, rust compilation speed is a top priority for rustc developers. Rust compiler times have gone down YoY every year for the past few years. See, eg, https://nnethercote.github.io/2021/11/12/the-rust-compiler-h...

Then this means that the developer ecosystem continuously finds ways to swallow the improvements in speed. :(

Re: Why is my Rust build so slow?

#186

Post by pdimitar »

yawaramin wrote: post #159
couchand wrote: post #156

Earlier quoted context omitted.

Go is also known for treating developers like cattle, so, I'll take Rust's trade-off, thanks.

OCaml is a powerful, expressive language with generics from day one, and Go-like build speeds.

All that remains is for it to gain a build tool like mix or cargo. Futzing around with esy works but takes more time than I'm willing to regularly invest in it.

Super excited about 5.00 and multicore!

Re: Why is my Rust build so slow?

#187

Post by howinteresting »

yawaramin wrote: post #174
howinteresting wrote: post #170

Earlier quoted context omitted.

Because you can have circular dependencies within a compilation unit, but not between units.

That actually sounds more appealing to me. In fact it's exactly how OCaml works. Of course in OCaml compilation units are just files already.

I don't think there is a moral justification for circular dependency scopes to always be individual files, especially given that import statements count as circular. The same issue in other languages where circular scopes are individual files leads to some horrible workarounds:

* Python TYPE_CHECKING https://adamj.eu/tech/2021/05/13/python-type-hints-how-to-fi...

* C/C++ header files

But regardless, there is too much Rust code in existence to change this now.

Re: Why is my Rust build so slow?

#188

Post by yawaramin »

howinteresting wrote: post #187
yawaramin wrote: post #174

Earlier quoted context omitted.

That actually sounds more appealing to me. In fact it's exactly how OCaml works. Of course in OCaml compilation units are just files already.

I don't think there is a moral justification for circular dependency scopes to always be individual files, especially given that import statements count as circular. The same issue in other languages where circular scopes are individual files leads to some horrible workarounds: * Python TYPE_CHECKING https://adamj.eu/tech/2021/05/13/python-type-hints-how-to-fi... * C/C++ header files But regardless, there is too much…

A 'moral' justification is not needed. We are talking about potentially massive build speed improvement here. Sounds like it could just be a build setting.

Re: Why is my Rust build so slow?

#189

Post by yawaramin »

pdimitar wrote: post #186
yawaramin wrote: post #159

Earlier quoted context omitted.

OCaml is a powerful, expressive language with generics from day one, and Go-like build speeds.

All that remains is for it to gain a build tool like mix or cargo. Futzing around with esy works but takes more time than I'm willing to regularly invest in it. Super excited about 5.00 and multicore!

I really go and forth on this. By separating the package manager and build tool, they leave the door open for either to be swapped out (which is what esy does, incidentally). The flexibility comes at a price of complexity, but it usually works well after climbing the learning curve.

Re: Why is my Rust build so slow?

#190

Post by pdimitar »

JoshTriplett wrote: post #26
howdydoo wrote: post #12

I've always wondered why useful flags like `-Z timings` are not available on stable. I guess they're worried about the output changing in a future stable release, but I don't think anyone would expect the reports to look exactly the same and list exactly the same compilation phases, etc, for all eternity. I think that for diagnostic features, people would be just fine with a loose stability guarantee that allows the…

I've been working on stabilizing some options like this, with exactly that approach: the functionality is stable but the exact details aren't. For instance, instrument-coverage will work like that, generating coverage data that requires the current version of LLVM coverage tools to work with. timings doesn't seem especially hard to stabilize; I'll take a look at it.

It will be hugely appreciated, thank you. It's honestly annoying periodically checking the flags docs and wondering "these things don't seem to change much, when will we have them in stable?".

As another poster down-thread said, human output can change at any time and this shouldn't be viewed as a stability problem. And if the machine output is still in the air then maybe it pays off to branch it out in a separate flag?

Post reply on HN