Why is my Rust build so slow? • Hacker News Board

Re: Why is my Rust build so slow?

#121

Post by ModernMech » Fri, Dec 31, 2021, 2:00 PM UTC

gameswithgo wrote: post #103

vladvasiliu wrote: post #72

Earlier quoted context omitted.

> But unfortunately, a lot of the problem is with the ecosystem, as hinted at in the article. There seems to be no limit to the amount of code bloat and compile time complexity that people are willing to accept to win some microbenchmarks. This includes some very popular crates with lots of dependents, like Tokio. But isn't this the right trade off for something like Tokio which is at the base of applications which e…

Yes, a slow compiletime of a dependency is less painful since you don’t do full recompiles that often, but things can spiral out of control if you have say 50 dependencies and many of them are real slow and you are on your old dual core laptop and find it takes hours, and so on.

Hold up, this is some severe goalpost moving. I realize these comments are coming from different people, but this thread started at:

“tribal knowledge required to bypass it, so early in the the life of a programming language, is a really bad sign.”

and within a few replies we ended at (paraphrasing):

“Granted compilation time is incremental, libraries are only compiled once, debug mode is much faster, but it’s really going to be an issue with a huge project on absurdly obsolete hardware.”

As the author of a project that relies on 50+ projects with some massive dependencies (wgpu, Tokio, rayon), yes it can take a good long while to compile from scratch, but it’s not “hours”, and after first compile that’s all paid for going forward.

Honestly this all seems like grasping at one of Rust’s perceived weak points, but really there are so many better criticisms, I don’t know why compile time gets so much attention.

Re: Why is my Rust build so slow?

#122

Post by ATsch » Fri, Dec 31, 2021, 2:11 PM UTC

brabel wrote: post #65

howdydoo wrote: post #11

Earlier quoted context omitted.

I once had a similar project to OP that took 30 seconds to compile a one-line change (albeit on a 10-year-old CPU). I split it into about 7 crates, and got compiles down to 3 seconds. Since then I'm always vigilant about keeping crates nice and small. I think as long as you're keeping an eye on your crate sizes, compile times won't get away from you. This is different from C++ where each individual source file can be…

That's a pretty bad incentive though and you end up with very large dependency trees similar to JS. And we know how that went. OTOH there might be a benefit too as more code becomes re-usable in independent crates.

> you end up with very large dependency trees similar to JS. And we know how that went.

The problem in the JS ecosystem is not the number of dependencies, but the number of entities you have to trust. There is no real difference between having one big crate in a repo managed by one entity or fifteen small crates in a repo managed by one entity.

Re: Why is my Rust build so slow?

#123

Post by swsieber » Fri, Dec 31, 2021, 2:12 PM UTC

Another article about rust diagnosing rust compile times: https://matklad.github.io/2021/09/04/fast-rust-builds.html

Re: Why is my Rust build so slow?

#124

Post by ModernMech » Fri, Dec 31, 2021, 2:15 PM UTC

mickeyp wrote: post #45

geewee wrote: post #32

Earlier quoted context omitted.

Often when building compilers you trade the speed of compilation against the speed of execution[1], and Rust has (traditionally) chosen the speed of executiong over making the compiler fast. This won't necessarily always be so, there's e.g. a new compiler backend called cranelift in the works, that makes compilations 30%ish faster, and in return the code isn't as optimized - which is good for a faster inner cycle. ht…

I'm not disagreeing, but I could cold compile a million lines of Delphi code in a few minutes on a crappy laptop more than a decade ago. Delphi managed to be both fast to compile and a fast language; of course, it's not as sophisticated or complex a language as C++ or Rust. But still...

The third axis in compiler development is safety. The compiler writer’s dilemma can be framed as:

fast execution, safe execution, fast compilation — choose 2.

Re: Why is my Rust build so slow?

#125

Post by Cu3PO42 » Fri, Dec 31, 2021, 2:16 PM UTC

formerly_proven wrote: post #101

aliceryhl wrote: post #77

Earlier quoted context omitted.

Tokio maintainer here. I've actually been spending a bunch of time recently looking in to how we can reduce Tokio's compile-times. I haven't really been able to find any big wins yet, but one thing has been pretty clear from the benchmarks: The number of dependencies of Tokio is not the problem. They all compile pretty fast and can all compile in parallel. Tokio itself takes a lot longer than the dependencies. (Excep…

So I've read "syn" and "slow rust compiles" in pretty much every discussion of "slow rust compiles" and finally googled it and... syn is a Rust parser? For use in macros? Because procedural macros operate on tokens and not an AST? Hm. I'm sure there are reasons for how it ended up like this, but it does smell kinda funny.

If I recall the reason is "rustc's representation of the AST is an implementation detail and unstable, thus macro code may not rely on it (and it will not be exposed)". Hence "syn" came along and provided a stable API and format. "quote" is the inverse function, turning an AST back into tokens.

Re: Why is my Rust build so slow?

#126

Post by Klasiaster » Fri, Dec 31, 2021, 2:16 PM UTC

Instead of using sccache and turning off incremental compilation it could help to change the approach and retain the build folder from the main branch CI builds for all other builds, since both sccache and incremental compilation are kind of overlapping with incremental compilation being the integrated solution while sccache seems to be more like a workaround?

Re: Why is my Rust build so slow?

#127

Post by estebank » Fri, Dec 31, 2021, 2:18 PM UTC

tialaramex wrote: post #115

Ygg2 wrote: post #49

Earlier quoted context omitted.

What the hell are you on about? Saying in it inherited them, just proves you probably didn't use either. While compilation times are longer than say Go, they definitely never felt too long, compared to say Java. But your comment around errors takes the cake. Like maybe, if you use some combination of macros expansion and traits it could get confused. But Rust errors are on par with Elm. They show the line, they show…

Rust's errors are definitely the best of any language I've used. That's one reason it's nice to write Rust after a day fighting with SQL Server for example which seems to still think in 2021 that "Syntax error" is a reasonable diagnostic message. As to "How to fix it" however if you have lifetime confusion the error reported by the borrow checker probably isn't going to suggest refactoring your code with a design tha…

> It would be cute if the Rust compiler said "I think you want a Boxed collection of strings here to use the algorithm you're attempting, change how this is represented in all the related structs too, here's how:" but that's unrealistic

I think we could provide such messages in some cases. :)

For example, if you have an impl Trait return type and have multiple returned values of different types, we suggest using Box when valid. It's just a lot of work to get those in place. You're effectively implementing a language that does those things automatically, plus all the work to print out the actual diagnostic.

Re: Why is my Rust build so slow?

#128

Post by ModernMech » Fri, Dec 31, 2021, 2:21 PM UTC

izacus wrote: post #46

twblalock wrote: post #28

This much complexity and the tribal knowledge required to bypass it, so early in the the life of a programming language, is a really bad sign. It means the complexity of the problem space is not solved by the language, and the responsibility of solving it is being passed on to the users. You might react to this by saying that slow build times are not a big deal. That's a mistake -- iteration is key to productivity an…

Rust seems to have inherited two favorite C++ development process features: long compilation times and incomprehensible error messages that aren't really related to the actual issue (e.g. just how C++ barfed out a bunch of template errors, borrow checker really loves to barf out very obscure error messages when there's an issue with lifetimes).

I teach C++ and Rust to college students, and their number one praise of Rust over C++ is how much more helpful the error messages are. It’s really night and day, so I can’t even fathom how you would equate the two.

Re: Why is my Rust build so slow?

#129

Post by pjmlp » Fri, Dec 31, 2021, 2:25 PM UTC

ModernMech wrote: post #121

gameswithgo wrote: post #103

Earlier quoted context omitted.

Yes, a slow compiletime of a dependency is less painful since you don’t do full recompiles that often, but things can spiral out of control if you have say 50 dependencies and many of them are real slow and you are on your old dual core laptop and find it takes hours, and so on.

Hold up, this is some severe goalpost moving. I realize these comments are coming from different people, but this thread started at: “tribal knowledge required to bypass it, so early in the the life of a programming language, is a really bad sign.” and within a few replies we ended at (paraphrasing): “Granted compilation time is incremental, libraries are only compiled once, debug mode is much faster, but it’s really…

Mainly because not everyone is willing to buy a last generation threadripper to have the same workflow as apt/yum/rpm/NuGet install libXXX-dev + make.

Re: Why is my Rust build so slow?

#130

Post by zenlot » Fri, Dec 31, 2021, 2:27 PM UTC

"In another language, say, C or C++, we might write a Makefile by hand.

What? No. Nobody does that anymore."

Oh these Rustaceans or how they call themselves. Look at the gaming industry, if you're playing any AAA/AA game it's most likely C++, there's some market share on C# (where Unity is used), but still the whole gaming industry is C++, and it's bigger than movie/music industries combined.

So yes, people "do that", and it's not going to change anytime soon.