Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is my Rust build so slow?

fasterthanli.me

Post reply on HN
Print view View original

81–90 of 217 posts

Re: Why is my Rust build so slow?

#81

Post by masklinn »

marmada wrote: post #2

I loved this article! Super informative. Very interesting that the author managed to hit a pathological case in rustc.

> Very interesting that the author managed to hit a pathological case in rustc.

FWIW lots of people hit this one, because entire pretty-base libraries got impacted by the regression: it hit everything which makes significant use of deeply nested "decorator" types, like iterator, future, and stream combinators.

Lime hit it because Warp is a big, big user of large nested decorator types (its entire routing and response system is based on that), pretty much every user of Warp with more than a handful of routes in their project hit this issue.

Re: Why is my Rust build so slow?

#82

Post by forrestthewoods »

twblalock wrote: post #28

This much complexity and the tribal knowledge required to bypass it, so early in the the life of a programming language, is a really bad sign. It means the complexity of the problem space is not solved by the language, and the responsibility of solving it is being passed on to the users. You might react to this by saying that slow build times are not a big deal. That's a mistake -- iteration is key to productivity an…

> It means the complexity of the problem space is not solved by the language, and the responsibility of solving it is being passed on to the users.

What languages do it best?

JavaScript is a dystopia of stacked bundlers. Python dependencies and deployment is a toxic superfund wasteland.

You’re not wrong that it’s a problem. But as far as I can tell build+deployment is an unsolved problem for non-hobby projects in most, if not all, languages.

Re: Why is my Rust build so slow?

#83

Post by holoduke »

I came from java and compared to that even our slowest rust build time is faster than I have ever experienced in java. I remember we had a Java springboot application who had a buildtime of 8 minutes on my 2015 MacBook. I am so happy I don't have to deal with this anymore.

Re: Why is my Rust build so slow?

#84

Post by alexchamberlain »

lordnacho wrote: post #66

Very long but interesting article. What's the reason that boxing a few things shortens the compile time?

Warp uses a lot of generics, which can produce extremely deep and long types. In turn, these cause a particular compile stage to take a long time (> 40 seconds).

Boxing disguises/splits the chain by taking a pointer to a trait object, which doesn't require the same depth of checks.

(This is especially important if the algorithm is superlinear, which is implied but not stated in the article.)

Re: Why is my Rust build so slow?

#85

Post by Ygg2 »

holoduke wrote: post #83

I came from java and compared to that even our slowest rust build time is faster than I have ever experienced in java. I remember we had a Java springboot application who had a buildtime of 8 minutes on my 2015 MacBook. I am so happy I don't have to deal with this anymore.

Damn. Are you me? I had the same experience.

Re: Why is my Rust build so slow?

#86

Post by Smaug123 »

alexchamberlain wrote: post #84
lordnacho wrote: post #66

Very long but interesting article. What's the reason that boxing a few things shortens the compile time?

Warp uses a lot of generics, which can produce extremely deep and long types. In turn, these cause a particular compile stage to take a long time (> 40 seconds). Boxing disguises/splits the chain by taking a pointer to a trait object, which doesn't require the same depth of checks. (This is especially important if the algorithm is superlinear, which is implied but not stated in the article.)

(The underlying issue is https://github.com/rust-lang/rust/issues/91598.)

Re: Why is my Rust build so slow?

#87

Post by xpressvideoz »

holoduke wrote: post #83

I came from java and compared to that even our slowest rust build time is faster than I have ever experienced in java. I remember we had a Java springboot application who had a buildtime of 8 minutes on my 2015 MacBook. I am so happy I don't have to deal with this anymore.

That's more to do with Spring, not Java. The compilation time of Java is infinitely faster than that of Rust.

Re: Why is my Rust build so slow?

#88

Post by estebank »

alexchamberlain wrote: post #84
lordnacho wrote: post #66

Very long but interesting article. What's the reason that boxing a few things shortens the compile time?

Warp uses a lot of generics, which can produce extremely deep and long types. In turn, these cause a particular compile stage to take a long time (> 40 seconds). Boxing disguises/splits the chain by taking a pointer to a trait object, which doesn't require the same depth of checks. (This is especially important if the algorithm is superlinear, which is implied but not stated in the article.)

The problem is not the length of the types, but rather the number of "obligations" introduced by each type. If you have Foo>, and each one of those types has a single trait bound, the resulting type now has >3 obligations to evaluate (> because you also have Sized and WF checks for them as well). The deeper the type depth, the more potential obligations you end up with, which have in some cases can have exponential evaluation. We have caching in place to only evaluate a given obligation once, but it can't always kick in. I'm sure there are still things we can improve there, though.

Re: Why is my Rust build so slow?

#89

Post by mtzet »

saurik wrote: post #34
lpapez wrote: post #30

Earlier quoted context omitted.

That is because the languages are different and choose different tradeoffs. The tradeoffs might make a language inapplicable in your domain. Rust has always been marketed as a "systems programming language". C++ is Rust's closes competitor in this domain space and it suffers from terribly slow compiling as well. I would say that due to headers, massive portability baggage and a lack of a standard build system, compil…

C++ projects tend to support massively parallel builds, despite what people tend to complain about with header files. The issue here in this article was partly that Rust only compiles crates in parallel, but while people tend to have lots and lots of C++ translation units, they don't have lots and lots of Rust crates. Languages seem to be moving in the wrong direction here :/.

That's a bit misleading though. The problem with header files is that they tend to get (textually) included in lots of compilation units, and hence rebuilt a lot of times. I did an experiment on a codebase at work where turning on unity builds -- so effectively everything is a single c++ file -- resulted in a 2x speedup in real time, and a 12x reduction in 'user' time.

Of course, such approaches lead to slower hot rebuilds. There's trade-offs worthy of a book. But the point is that C++ is only massively parallel by adding a lot of extra work.

This problem could be overcome by reducing the complexity of header files, but that's generally difficult if template-types are used. In general, the solution is to write code that's closer to C than C++ in the header files. In particular using opaque pointers to hide data.

We're actually seeing a very similiar problem in the linked article. The chief problem is monomorphization (~templates) and the solution is to use boxing (~data hiding via opaque pointers).

Re: Why is my Rust build so slow?

#90

Post by menaerus »

LowLevelMahn wrote: post #71
menaerus wrote: post #50

Earlier quoted context omitted.

To be more precise, rust is a competitor to C++ and not the other way around. Also, compiling C++ is _not_ worse than compiling rust projects but again the other way around. Although compilation times are a hog in both of these languages, it is well known that this is actually one of the biggest rust pain points. In general, long compilation times are mostly attributed by the complexity of things (algorithms) that co…

In general, long compilation times are mostly attributed by the complexity of things (algorithms) that compilers are trying to do for you in order to produce machine code and not because there's a lack of standardized build system (a bad idea) nor because of existence of headers. that is not true - most of the project i worked for suffered from header parsing a lot - sometimes more then 50%, in 100k-1MioK lines of co…

I tend to disagree with this. I've optimized build times on number of occasions on different projects, each being in the venue of multi-million LoC, and none of them benefited greatly from employing precompiled headers, which essentially should help alleviate exact this gap.

Running the build with -ftime-trace almost always has shown in my experiments that the bottlenecks are either in exploding number of template instatiations or more often than not in linking time.

Post reply on HN