Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is my Rust build so slow?

fasterthanli.me

Post reply on HN
Print view View original

71–80 of 217 posts

Re: Why is my Rust build so slow?

#71

Post by LowLevelMahn »

menaerus wrote: post #50
lpapez wrote: post #30

Earlier quoted context omitted.

That is because the languages are different and choose different tradeoffs. The tradeoffs might make a language inapplicable in your domain. Rust has always been marketed as a "systems programming language". C++ is Rust's closes competitor in this domain space and it suffers from terribly slow compiling as well. I would say that due to headers, massive portability baggage and a lack of a standard build system, compil…

To be more precise, rust is a competitor to C++ and not the other way around. Also, compiling C++ is _not_ worse than compiling rust projects but again the other way around. Although compilation times are a hog in both of these languages, it is well known that this is actually one of the biggest rust pain points. In general, long compilation times are mostly attributed by the complexity of things (algorithms) that co…

  In general, long compilation times are mostly attributed by 
  the complexity of things (algorithms) that compilers are 
  trying to do for you in order to produce machine code and 
  not because there's a lack of standardized build system (a 
  bad idea) nor because of existence of headers.
that is not true - most of the project i worked for suffered from header parsing a lot - sometimes more then 50%, in 100k-1MioK lines of code projects many C++ developer tend to think that the code generation is the time consuming part but that is less true - you will see if you start benchmarking the compilation with todays tools available in Clang and VS2019+

Re: Why is my Rust build so slow?

#72

Post by vladvasiliu »

codeflo wrote: post #35
twblalock wrote: post #28

This much complexity and the tribal knowledge required to bypass it, so early in the the life of a programming language, is a really bad sign. It means the complexity of the problem space is not solved by the language, and the responsibility of solving it is being passed on to the users. You might react to this by saying that slow build times are not a big deal. That's a mistake -- iteration is key to productivity an…

I agree, and even as someone who holds Rust in high regard, I’m becoming increasingly frustrated by this. There doesn’t seem to be one clear root of this problem. Some of it is the language not being designed with ease of compilation in mind, unlike let’s say, to pick an extreme example, Go. Some of it is the tooling and default settings, especially around incremental compilation, linkers etc. People are working on t…

> But unfortunately, a lot of the problem is with the ecosystem, as hinted at in the article. There seems to be no limit to the amount of code bloat and compile time complexity that people are willing to accept to win some microbenchmarks. This includes some very popular crates with lots of dependents, like Tokio.

But isn't this the right trade off for something like Tokio which is at the base of applications which expect to be fast? Or else those applications wouldn't bother with async runtimes and what-have-you. There's also the fact that during the dev cycle, you'll compile tokio & friends once and be done with it. And for CI pipelines, there's caching.

I'm however not a professional developer, so I'm curious if I'm missing something.

Re: Why is my Rust build so slow?

#73

Post by space_rock »

izacus wrote: post #46
twblalock wrote: post #28

This much complexity and the tribal knowledge required to bypass it, so early in the the life of a programming language, is a really bad sign. It means the complexity of the problem space is not solved by the language, and the responsibility of solving it is being passed on to the users. You might react to this by saying that slow build times are not a big deal. That's a mistake -- iteration is key to productivity an…

Rust seems to have inherited two favorite C++ development process features: long compilation times and incomprehensible error messages that aren't really related to the actual issue (e.g. just how C++ barfed out a bunch of template errors, borrow checker really loves to barf out very obscure error messages when there's an issue with lifetimes).

Someone has never used Rust

Re: Why is my Rust build so slow?

#74

Post by jcelerier »

nmfisher wrote: post #43
codeflo wrote: post #40

Earlier quoted context omitted.

That matches my experience as well. It takes a nontrivial amount of discipline, but it’s possible to structure a C++ project for fast builds. In theory, having a module system that isn’t based on text substitution should give Rust an advantage, but on the minus side, the crate structure gives you fewer knobs to optimize.

Care to elaborate further on how to best structure a C++ project for fast builds?

Here's my edit-compile-run cycle in dev configuration: https://streamable.com/az397y

I use Qt, boost, lots of templates.

- clang as a compiler

- mold as a linker

- PCH (easy with cmake)

- plugin architecture for the software ; plugins are dynamic libraries when developing (everything is linked statically for releases which take of course much longer to build with lto, etc.)

- building on Linux (it's seriously slower on Windows, when using the exact same compiler on the exact same SSD)

Re: Why is my Rust build so slow?

#75

Post by misnome »

twblalock wrote: post #28

This much complexity and the tribal knowledge required to bypass it, so early in the the life of a programming language, is a really bad sign. It means the complexity of the problem space is not solved by the language, and the responsibility of solving it is being passed on to the users. You might react to this by saying that slow build times are not a big deal. That's a mistake -- iteration is key to productivity an…

This is more of a step-by-step deep-dive into the ways to profile, diagnose and work through the timing and performance problems. Anyone who doesn't know about many of these tools will probably find them useful (or at least useful to know that they exist) in the future.

Besides, for iteration people are probably in debug, not release mode, which the article mentions is initially 19s vs 2m+. As far as I can see the main conclusion seems to be that there was a compiler regression causing the issues, which is fixed in nightly.

I don't think many users would need to go through these steps to diagnose the problem beyond "wait for a newer compiler release".

In general I don't think being slightly slower in a systems language in cases where you've explicitly asked for full optimisations is that much of a problem when there are simple ways to mitigate the feedback loop (e.g. debug compilations). Would I like these things to be faster? Sure. But with the size of the rust projects I use, it's not noticeable slower than the large established C++ projects I have to deal with (that aren't tuned for compilation speed).

Re: Why is my Rust build so slow?

#76

Post by masklinn »

maxwell86 wrote: post #57
twblalock wrote: post #28

This much complexity and the tribal knowledge required to bypass it, so early in the the life of a programming language, is a really bad sign. It means the complexity of the problem space is not solved by the language, and the responsibility of solving it is being passed on to the users. You might react to this by saying that slow build times are not a big deal. That's a mistake -- iteration is key to productivity an…

Rust 1.58 has a bug that introduced a compile time regression, and the knowledge required to bypass it is to just switch to the preview of the next release, which is trivial to set up. So I really don’t know what you are talking about in your comment. What’s hard about this?

> Rust 1.58

Do you mean 1.57?

As soon as I saw the final binary taking an ungodly amount of time I knew what it was going to be, because I hit it myself, in 1.57, using warp.

Though there are also a few separate older regressions (at least one in 1.56, though I gather somewhat less critical / troublesome).

> the knowledge required to bypass it is to just switch to the preview of the next release

Switching to the previous tends to be more reliable and a better idea, really.

Re: Why is my Rust build so slow?

#77

Post by aliceryhl »

codeflo wrote: post #35
twblalock wrote: post #28

This much complexity and the tribal knowledge required to bypass it, so early in the the life of a programming language, is a really bad sign. It means the complexity of the problem space is not solved by the language, and the responsibility of solving it is being passed on to the users. You might react to this by saying that slow build times are not a big deal. That's a mistake -- iteration is key to productivity an…

I agree, and even as someone who holds Rust in high regard, I’m becoming increasingly frustrated by this. There doesn’t seem to be one clear root of this problem. Some of it is the language not being designed with ease of compilation in mind, unlike let’s say, to pick an extreme example, Go. Some of it is the tooling and default settings, especially around incremental compilation, linkers etc. People are working on t…

Tokio maintainer here. I've actually been spending a bunch of time recently looking in to how we can reduce Tokio's compile-times. I haven't really been able to find any big wins yet, but one thing has been pretty clear from the benchmarks: The number of dependencies of Tokio is not the problem. They all compile pretty fast and can all compile in parallel. Tokio itself takes a lot longer than the dependencies.

(Except for tokio-macros which takes a long time because it depends on syn and quote. Consider disabling it if you don't need it.)

Re: Why is my Rust build so slow?

#78

Post by Ygg2 »

brabel wrote: post #62
Ygg2 wrote: post #49

Earlier quoted context omitted.

What the hell are you on about? Saying in it inherited them, just proves you probably didn't use either. While compilation times are longer than say Go, they definitely never felt too long, compared to say Java. But your comment around errors takes the cake. Like maybe, if you use some combination of macros expansion and traits it could get confused. But Rust errors are on par with Elm. They show the line, they show…

> they definitely never felt too long, compared to say Java. Woah... now I am the one who have to ask: what the hell are you on about?? I work on Rust and Java projects, if you exclude running tests, Java compiles very very fast compared to pretty much any language... Rust is a lot slower to compile even on the much smaller projects I've worked on... in this post, a very small project (I think it's like 16,000 LoC, i…

And I've worked on large Java projects.

Things that have like 420+ dependencies (spring boot + some other requirements). I timed the maven clean compile. And it takes around 2min.

Re: Why is my Rust build so slow?

#79

Post by masklinn »

menaerus wrote: post #55

Whenever I read articles like this one, it leaves me wondering in what parallel reality do I live in. Or I could put it differently and say that authors making blogposts complaining about full rebuild cycle taking staggering 2 minutes live in a pink bubble. Everyday reality in somewhat complex industry-placed product is 10x that if you're lucky. More often than not it's few hours before you're able to build the whole…

I think it helps to have some of the not-exactly-stated background: their build did not originally take 2mn, it suddenly shot up to this from around 20s seemingly overnight, so they investigated what happened, and as that's lime they also produced a nice article out of it.

That's my assumption anyway because that's about what I went through, minus the blog post: had a project I was working on, at one point I noticed the clean builds were getting really long, we're talking >10mn. Initially I figured it might be Serde (as that project is really serde-heavy) and the compile time had crept up on me unnoticed, so I moved all the serialized and deserialized types in a sub-crate in case that did anything... and it did not.

Then I busted out the -Ztimings getting about the same result Lime did (ungodly amount of time spent producing the binary for no clear reason), tried fiddling with some compile-time options (went nowhere near as deep there, I looked at -Z self-profile fairly quickly IIRC), finally found the evaluate_obligation stuff. From there I didn't bust out the profiler to see what was what, instead through I don't quite remember which chain got me to issue 91598 (https://github.com/rust-lang/rust/issues/91598).

Added a follow to the issue and various PRs proposed for it, and locked the project to 1.56.

Re: Why is my Rust build so slow?

#80

Post by masklinn »

brabel wrote: post #65
howdydoo wrote: post #11

Earlier quoted context omitted.

I once had a similar project to OP that took 30 seconds to compile a one-line change (albeit on a 10-year-old CPU). I split it into about 7 crates, and got compiles down to 3 seconds. Since then I'm always vigilant about keeping crates nice and small. I think as long as you're keeping an eye on your crate sizes, compile times won't get away from you. This is different from C++ where each individual source file can be…

That's a pretty bad incentive though and you end up with very large dependency trees similar to JS. And we know how that went. OTOH there might be a benefit too as more code becomes re-usable in independent crates.

> That's a pretty bad incentive though and you end up with very large dependency trees similar to JS. And we know how that went.

That's... not the same thing at all.

Rust lets you have multiple crate in a single workspace, which allows parallelising the build (because crates are the concurrency unit of building rust).

That's got nothing whatsoever to do with pulling random crates, which is a separate issue.

Post reply on HN