Why is my Rust build so slow? • Hacker News Board

Re: Why is my Rust build so slow?

#51

Post by spockz » Fri, Dec 31, 2021, 9:13 AM UTC

saurik wrote: post #34

lpapez wrote: post #30

Earlier quoted context omitted.

That is because the languages are different and choose different tradeoffs. The tradeoffs might make a language inapplicable in your domain. Rust has always been marketed as a "systems programming language". C++ is Rust's closes competitor in this domain space and it suffers from terribly slow compiling as well. I would say that due to headers, massive portability baggage and a lack of a standard build system, compil…

C++ projects tend to support massively parallel builds, despite what people tend to complain about with header files. The issue here in this article was partly that Rust only compiles crates in parallel, but while people tend to have lots and lots of C++ translation units, they don't have lots and lots of Rust crates. Languages seem to be moving in the wrong direction here :/.

I rarely touch C(++) but every time I have to compile something a bit more complicated than the simplest of projects it is noticeably slow. Other languages I use, scala and go and rust, have higher initial cost but seem to suffer less when the sources grow.

A recent example, linkerd proxy compiled in a few minutes, while it took me hours to compile envoy proxy.

Then there are the stories in how ridiculously long it takes to build Chrome or the V8 engine. I’m really not sold that real world c(++) projects are fast to build.

Re: Why is my Rust build so slow?

#52

Post by codeflo » Fri, Dec 31, 2021, 9:14 AM UTC

nmfisher wrote: post #43

codeflo wrote: post #40

Earlier quoted context omitted.

That matches my experience as well. It takes a nontrivial amount of discipline, but it’s possible to structure a C++ project for fast builds. In theory, having a module system that isn’t based on text substitution should give Rust an advantage, but on the minus side, the crate structure gives you fewer knobs to optimize.

Care to elaborate further on how to best structure a C++ project for fast builds?

It’s a large topic, but the most important goal is to minimize the amount of code that’s transitively included in headers.

That can mean avoiding templates, or (if you’re willing to do the work) using explicit instantiations in one source file only.

Use the PImpl pattern so that users of a class don’t need to know the declarations of its private members.

And whenever reasonable, design your method signatures so that objects are passed by pointer/reference — this means you can work with forward declarations instead of always having to include the full class declaration.

Do all of that only where reasonable. Small headers usually don’t matter much, and avoiding dependencies on headers that users are likely to include anyway (like ) doesn’t win anything either.

Re: Why is my Rust build so slow?

#53

Post by pjmlp » Fri, Dec 31, 2021, 9:14 AM UTC

charcircuit wrote: post #21

the8472 wrote: post #19

Earlier quoted context omitted.

> Like why does it take tens of seconds to recompile when I am just changing a single number in a file? Impossible to tell without knowing more. Steps to reproduce the issue would help. Just consider how TFA went through many different things to investigate. > Does it really need to waste so much of my time to change a single byte in the output binary? Does it actually only change a single byte in the binary? Changin…

>Impossible to tell without knowing more. It was a rhetorical question based off my experience with rust. I was running into this issue with building a site using warp along with some other dependencies. Those projects have been deleted off my system. I am not interested in chasing down performance issues with the compiler. Remaking the site using C++ did not have me run into slow compile times. I just want to be abl…

I feel the same pain, specially since when using C++ I tend to just use binary libraries for all the code I don't own, so even cold builds are quite fast even when talking about C++.

Just by having cargo support binary libraries would be an improvement for cold builds, but I understand it isn't a priority.

The Rust/WinRT folks have a big binary crate that they use to workaround Rust's compilation times, https://github.com/microsoft/windows-rs/tree/master/.windows

Re: Why is my Rust build so slow?

#54

Post by dataflow » Fri, Dec 31, 2021, 9:15 AM UTC

nmfisher wrote: post #43

codeflo wrote: post #40

Earlier quoted context omitted.

That matches my experience as well. It takes a nontrivial amount of discipline, but it’s possible to structure a C++ project for fast builds. In theory, having a module system that isn’t based on text substitution should give Rust an advantage, but on the minus side, the crate structure gives you fewer knobs to optimize.

Care to elaborate further on how to best structure a C++ project for fast builds?

Avoid over-templating, don't inline expensive functions, put expensive #includes inside .cpp files instead of headers, use internal linkage whenever possible, etc. Doable in theory, painful to get there in practice if you've already dug yourself into a hole, and you can't get out of said hole without convincing the rest of your team/organization to change their coding practices accordingly.

Re: Why is my Rust build so slow?

#55

Post by menaerus » Fri, Dec 31, 2021, 9:19 AM UTC

Whenever I read articles like this one, it leaves me wondering in what parallel reality do I live in.

Or I could put it differently and say that authors making blogposts complaining about full rebuild cycle taking staggering 2 minutes live in a pink bubble.

Everyday reality in somewhat complex industry-placed product is 10x that if you're lucky. More often than not it's few hours before you're able to build the whole thing.

Re: Why is my Rust build so slow?

#56

Post by pjmlp » Fri, Dec 31, 2021, 9:19 AM UTC

echelon wrote: post #20

sjakwjsjsisj wrote: post #6

Earlier quoted context omitted.

Use bazel? Individual files won't be faster but it's been a wonder for whole project compilations

Does anyone have an example of a Rust Bazel build project? Or even better, a massive Rust monorepo orchestrated with Bazel build?

Not what you asked for, but maybe still interesting to look into.

Android uses Soong for Rust,

https://source.android.com/setup/build/rust/building-rust-mo...

While Fucshia uses GN with Rust,

https://fuchsia.dev/fuchsia-src/development/build/concepts/b...

https://fuchsia.dev/fuchsia-src/development/languages/rust

Re: Why is my Rust build so slow?

#57

Post by maxwell86 » Fri, Dec 31, 2021, 9:20 AM UTC

twblalock wrote: post #28

This much complexity and the tribal knowledge required to bypass it, so early in the the life of a programming language, is a really bad sign. It means the complexity of the problem space is not solved by the language, and the responsibility of solving it is being passed on to the users. You might react to this by saying that slow build times are not a big deal. That's a mistake -- iteration is key to productivity an…

Rust 1.58 has a bug that introduced a compile time regression, and the knowledge required to bypass it is to just switch to the preview of the next release, which is trivial to set up.

So I really don’t know what you are talking about in your comment. What’s hard about this?

Re: Why is my Rust build so slow?

#58

Post by Uther » Fri, Dec 31, 2021, 9:22 AM UTC

izacus wrote: post #46

twblalock wrote: post #28

This much complexity and the tribal knowledge required to bypass it, so early in the the life of a programming language, is a really bad sign. It means the complexity of the problem space is not solved by the language, and the responsibility of solving it is being passed on to the users. You might react to this by saying that slow build times are not a big deal. That's a mistake -- iteration is key to productivity an…

Rust seems to have inherited two favorite C++ development process features: long compilation times and incomprehensible error messages that aren't really related to the actual issue (e.g. just how C++ barfed out a bunch of template errors, borrow checker really loves to barf out very obscure error messages when there's an issue with lifetimes).

While compilation time is a well known Rust pain point, Rust error messages are really one of the best part of the language.

Complicated error messages are clearly no a common occurrence unless you rely heavily on meta-programming tricks, and that is not common in Rust, unlike C++.

Re: Why is my Rust build so slow?

#59

Post by pjmlp » Fri, Dec 31, 2021, 9:22 AM UTC

OtomotO wrote: post #48

_ma_rit_231 wrote: post #18

The long compile times are actually one of Rust's problems I could live with. Having to depend on so many third party libs is a much bigger issue for long term maintainability and security. Also the time choosing the libs is not neglectible. At work we currently developing a mid size project (CLI and HTTP API) with Go and the standard lib is simply amazing.

The difference between unmaintained code in a stdlib and unmaintained code in a third party lib is artificial. I want the stdlib to be as lean as possible. Also, as you just read, crates are compilation modules, hence generally this means more crates, lead to faster compile times.

Until cargo supports binary dependencies this is only true for incremental builds.

Microsoft has to ship binary libraries with build scripts to work around it,

https://github.com/microsoft/windows-rs/tree/master/.windows

Re: Why is my Rust build so slow?

#60

Post by jokethrowaway » Fri, Dec 31, 2021, 9:26 AM UTC

menaerus wrote: post #50

lpapez wrote: post #30

Earlier quoted context omitted.

That is because the languages are different and choose different tradeoffs. The tradeoffs might make a language inapplicable in your domain. Rust has always been marketed as a "systems programming language". C++ is Rust's closes competitor in this domain space and it suffers from terribly slow compiling as well. I would say that due to headers, massive portability baggage and a lack of a standard build system, compil…

To be more precise, rust is a competitor to C++ and not the other way around. Also, compiling C++ is _not_ worse than compiling rust projects but again the other way around. Although compilation times are a hog in both of these languages, it is well known that this is actually one of the biggest rust pain points. In general, long compilation times are mostly attributed by the complexity of things (algorithms) that co…

I disagree that it's a pain point in C++ or in Rust. You're generally just doing a lot more stuff and shifting the pain to incremental debug builds (so that the slow path doesn't need recompilation) is a perfectly viable solution. Template metaprogramming has never been the pain point in C++ for me because nobody wanted to use templates (after a certain points), huge deps are.

Anecdotally, the slow builds I've experienced in Rust were case sof overzealous Serialize implementation (aka, people adding Serialize to everything, just in case) or deps on large external projects (often in C++).

Coming from C++, where you either have someone who knows how to debug build times, I think the comparison with rust is unfair. cargo-timings is pretty nice and gives you a hint on what to look for.

This article actually shows debugging build times is not that hard. I did that for a personal project (which compiled part of OpenCV) and I'm certainly no ninja. I remember attempting similar tasks in C++ and getting completely lost.