Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is my Rust build so slow?

fasterthanli.me

Post reply on HN
Print view View original

131–140 of 217 posts

Re: Why is my Rust build so slow?

#131

Post by pjmlp »

lpapez wrote: post #30
twblalock wrote: post #28

This much complexity and the tribal knowledge required to bypass it, so early in the the life of a programming language, is a really bad sign. It means the complexity of the problem space is not solved by the language, and the responsibility of solving it is being passed on to the users. You might react to this by saying that slow build times are not a big deal. That's a mistake -- iteration is key to productivity an…

That is because the languages are different and choose different tradeoffs. The tradeoffs might make a language inapplicable in your domain. Rust has always been marketed as a "systems programming language". C++ is Rust's closes competitor in this domain space and it suffers from terribly slow compiling as well. I would say that due to headers, massive portability baggage and a lack of a standard build system, compil…

On most corporations C++ third party libraries are consumed as binary dependencies, hence the large compile times aren't as serious as in Rust.

Re: Why is my Rust build so slow?

#132

Post by mplanchard »

lpapez wrote: post #30
twblalock wrote: post #28

This much complexity and the tribal knowledge required to bypass it, so early in the the life of a programming language, is a really bad sign. It means the complexity of the problem space is not solved by the language, and the responsibility of solving it is being passed on to the users. You might react to this by saying that slow build times are not a big deal. That's a mistake -- iteration is key to productivity an…

That is because the languages are different and choose different tradeoffs. The tradeoffs might make a language inapplicable in your domain. Rust has always been marketed as a "systems programming language". C++ is Rust's closes competitor in this domain space and it suffers from terribly slow compiling as well. I would say that due to headers, massive portability baggage and a lack of a standard build system, compil…

I work on a large rust project professionally, and we have tried to make build times a priority for CI. I’m just noting a couple of things that helped us a lot, in case they’re useful.

- sccache is pretty much essential

- beefiness: GitLab’s default runners for example are way too underpowered and made our builds take like 30 minutes (as opposed to 5 on a beefy machine in CircleCI). We’re working on switching to self-hosted runners for GL

- replacing diesel with sqlx saved us a fair bit of time

- caching the cargo home can help. It wasn’t a massive gain relative to sccache, though. Similar story with caching the target directory

- lld linking was another relatively small win, but a win nonetheless

- splitting things up into smaller libraries and binaries (all in one workspace) helped us a fair bit, since building/testing some targets can now totally skip some libraries, but I imagine YMMV

Re: Why is my Rust build so slow?

#133

Post by ithkuil »

spockz wrote: post #51
saurik wrote: post #34

Earlier quoted context omitted.

C++ projects tend to support massively parallel builds, despite what people tend to complain about with header files. The issue here in this article was partly that Rust only compiles crates in parallel, but while people tend to have lots and lots of C++ translation units, they don't have lots and lots of Rust crates. Languages seem to be moving in the wrong direction here :/.

I rarely touch C(++) but every time I have to compile something a bit more complicated than the simplest of projects it is noticeably slow. Other languages I use, scala and go and rust, have higher initial cost but seem to suffer less when the sources grow. A recent example, linkerd proxy compiled in a few minutes, while it took me hours to compile envoy proxy. Then there are the stories in how ridiculously long it t…

I have a vague memory that at a previous job I was building envoy proxy in a build automation pipeline and it regularly OOMed. While troubleshooting I noticed that most of the time and memory was spent compiling one or two generated C++ files; they contained many many many type parameters. When you have to generate templates something is wrong.

Re: Why is my Rust build so slow?

#134

Post by ModernMech »

pjmlp wrote: post #129
ModernMech wrote: post #121

Earlier quoted context omitted.

Hold up, this is some severe goalpost moving. I realize these comments are coming from different people, but this thread started at: “tribal knowledge required to bypass it, so early in the the life of a programming language, is a really bad sign.” and within a few replies we ended at (paraphrasing): “Granted compilation time is incremental, libraries are only compiled once, debug mode is much faster, but it’s really…

Mainly because not everyone is willing to buy a last generation threadripper to have the same workflow as apt/yum/rpm/NuGet install libXXX-dev + make.

Including me! There’s a lot of daylight between “an old dual core laptop” and a last gen thread ripper. For example, I’m running on a quad core i7 from 2015.

Re: Why is my Rust build so slow?

#135

Post by pornel »

lordnacho wrote: post #66

Very long but interesting article. What's the reason that boxing a few things shortens the compile time?

It's not about boxing itself, but about using dynamic dispatch that is enabled by boxing.

1. Type erasure — compiler works with a single abstract interface rather than big complex concrete types.

2. Replaces monomorphisation (a fancy word for big copy'n'paste of code for every type it's used with) with dynamic dispatch that has a single implementation for all users.

Re: Why is my Rust build so slow?

#136

Post by alexchamberlain »

zenlot wrote: post #130

"In another language, say, C or C++, we might write a Makefile by hand. What? No. Nobody does that anymore." Oh these Rustaceans or how they call themselves. Look at the gaming industry, if you're playing any AAA/AA game it's most likely C++, there's some market share on C# (where Unity is used), but still the whole gaming industry is C++, and it's bigger than movie/music industries combined. So yes, people "do that"…

I think the OP meant no one writes Makefiles manually; you might use CMake or ninja, for example.

Re: Why is my Rust build so slow?

#137

Post by howdydoo »

amelius wrote: post #41
howdydoo wrote: post #11

Earlier quoted context omitted.

I once had a similar project to OP that took 30 seconds to compile a one-line change (albeit on a 10-year-old CPU). I split it into about 7 crates, and got compiles down to 3 seconds. Since then I'm always vigilant about keeping crates nice and small. I think as long as you're keeping an eye on your crate sizes, compile times won't get away from you. This is different from C++ where each individual source file can be…

Can this be automated? Could a script divide code into small crates upon compilation?

I did it by hand. It's a mostly mechanical transformation, so in theory it could be automated, but I've never seen an automated refactor for any language that can e.g. take a giant file and decide how to split it up into 5 smaller files. But that's something I'd love to see exist

Re: Why is my Rust build so slow?

#138

Post by pornel »

sanxiyn wrote: post #16
charcircuit wrote: post #3

Rust's slow compiles are such a turn off for me. Like why does it take tens of seconds to recompile when I am just changing a single number in a file? Does it really need to waste so much of my time to change a single byte in the output binary?

You are right. There is no technical reason it should be as slow as it is. It's just that not enough resource was spent to make it fast. I mean, if this were a higher priority, Rust issue #26600 would have been fixed years ago. https://github.com/rust-lang/rust/issues/26600

Looks like it's blocked by lack of required LLVM support.

Re: Why is my Rust build so slow?

#139

Post by pjmlp »

ModernMech wrote: post #134
pjmlp wrote: post #129

Earlier quoted context omitted.

Mainly because not everyone is willing to buy a last generation threadripper to have the same workflow as apt/yum/rpm/NuGet install libXXX-dev + make.

Including me! There’s a lot of daylight between “an old dual core laptop” and a last gen thread ripper. For example, I’m running on a quad core i7 from 2015.

My old dual core laptop from 2009 works just fine with Visual Studio 2022, even though I wasn't the OP.

And long compile times kill the battery on the go, and cargo check hardly helps when writing GUI code.

Re: Why is my Rust build so slow?

#140

Post by howdydoo »

heavyset_go wrote: post #14
howdydoo wrote: post #11

Earlier quoted context omitted.

I once had a similar project to OP that took 30 seconds to compile a one-line change (albeit on a 10-year-old CPU). I split it into about 7 crates, and got compiles down to 3 seconds. Since then I'm always vigilant about keeping crates nice and small. I think as long as you're keeping an eye on your crate sizes, compile times won't get away from you. This is different from C++ where each individual source file can be…

What was your heuristic for deciding if a crate is too big or not?

Going by feel, same as when I have a giant function I decide to split up. IIRC the crates ended up between 1k-10k LOC
Post reply on HN