The very idea of having compilation unit boundaries look like project boundaries squicks me :/... the lightweight compilation unit structure C/C++ has--with the ability to elide information but the requirement that it be prototyped--frankly always seemed like the perfect tradeoff, and it generally means that projects are able to be compiled in massively parallel environments as all of the dependencies are explicit. This is why distcc is such a trivial project to just drop on almost C codebase, and is why I have been able to get our builds doing like 100x parallel compiles on AWS Lambda (and like, before someone tries to claim that header file bloat somehow prevents this: it doesn't, and build engineers do massively parallel build farms for C/C++ all the time, and it is only in some specific corner cases of C++ projects defined entirely as templates where this comes up, and even then there are trivial techniques you can use to help break the translation units apart). Every time a new language comes out and tries to "fix" translation units by adding tons of cross-file information dependencies without the explicit definition vs. declaration difference I kind of roll my eyes as I know there's almost no chance the build is ever going to fast, and these "we need all the code at once to even begin to analyze it" languages are the worst :(.
Rust's Huge Compilation Units
11–19 of 19 posts
Re: Rust's Huge Compilation Units
#12The URL of this article was changed to the following (so the one linked on Hacker News as of when I posted this comment is now a 404). https://pingcap.com/blog/Rust-s-Huge-Compilation-Units
Sorry, the problem is being fixed now
Re: Rust's Huge Compilation Units
#13>In my experience though projects tend to start in a single crate, without great attention to their internal dependency graph, and once compilation time becomes an issue, they have already created a spaghetti dependency graph that is difficult to refactor into smaller crates. This happened to me too. One important problem to overcome is the orphan rule. This means: you can only implement a trait you own or for a type…
Funny how, I believe, the terms "orphan" and "newtype" originated in Haskell and were carried over to Rust. Evidence of the functional programming heritage that Rust has, I guess.
Yes. For the unaware, the original creator of Rust wanted it to basically be OCaml with better concurrency. It used to have a more ML style syntax, green threads, etc.
Re: Rust's Huge Compilation Units
#14>In my experience though projects tend to start in a single crate, without great attention to their internal dependency graph, and once compilation time becomes an issue, they have already created a spaghetti dependency graph that is difficult to refactor into smaller crates. This happened to me too. One important problem to overcome is the orphan rule. This means: you can only implement a trait you own or for a type…
Adding to my post: I understand the neccessity of the orphan rule (see other post about why). At least I am happy that there's a solution (newtypes) even if it is not nice.
Re: Rust's Huge Compilation Units
#15The URL of this article was changed to the following (so the one linked on Hacker News as of when I posted this comment is now a 404). https://pingcap.com/blog/Rust-s-Huge-Compilation-Units
The link works now.
Re: Rust's Huge Compilation Units
#16Its not just a Rust problem, I've run into the same thing in Java and I'm sure it exists elsewhere. The only answer I can come up with is to keep your app cleanly divided into modules. There's some cognitive overhead but its usually worthwhile. I recently divided a 100k line Java app into 5 different modules and it compiles 4 times faster. The new boundaries have encouraged better code as well. Suddenly we have thing…
I work with both Java and Rust. Definitely not the same. With Java, you decide what you want to compile with your build script. You don't need to compile entire application as a single unit. If you have to recompile a lot of code when you only make small modification it is likely because did not use many of available ways to just recompile the classes that you modified. Also, with Java some optimizations are pushed t…
Java also does optimizations at AOT, it is all a matter which toolchain one uses, plenty of choice since 2000.
Re: Rust's Huge Compilation Units
#17>In my experience though projects tend to start in a single crate, without great attention to their internal dependency graph, and once compilation time becomes an issue, they have already created a spaghetti dependency graph that is difficult to refactor into smaller crates. This happened to me too. One important problem to overcome is the orphan rule. This means: you can only implement a trait you own or for a type…
I agree. Something must absolutely be done about the orphan rules, IMO they are Rust's biggest wart. They damage composition in a fundamental way, and gimp the otherwise extremely expressive trait system.
I find it hard to believe that there exists no set of import declarations which remove all ambiguity, i.e., allowing orphan traits but requiring that they get imported explicitly.
Re: Rust's Huge Compilation Units
#18>In my experience though projects tend to start in a single crate, without great attention to their internal dependency graph, and once compilation time becomes an issue, they have already created a spaghetti dependency graph that is difficult to refactor into smaller crates. This happened to me too. One important problem to overcome is the orphan rule. This means: you can only implement a trait you own or for a type…
I agree. Something must absolutely be done about the orphan rules, IMO they are Rust's biggest wart. They damage composition in a fundamental way, and gimp the otherwise extremely expressive trait system. I find it hard to believe that there exists no set of import declarations which remove all ambiguity, i.e., allowing orphan traits but requiring that they get imported explicitly.
I think this is a fundamentally difficult problem. Life is sometimes not nice and you have to choose. I chose to write newtype wrappers and to accept the complexity.
> I find it hard to believe [...]
I think you should read up stuff about the orphan rule, then you will understand better.
Re: Rust's Huge Compilation Units
#19Earlier quoted context omitted.
I agree. Something must absolutely be done about the orphan rules, IMO they are Rust's biggest wart. They damage composition in a fundamental way, and gimp the otherwise extremely expressive trait system. I find it hard to believe that there exists no set of import declarations which remove all ambiguity, i.e., allowing orphan traits but requiring that they get imported explicitly.
I think this is a fundamentally difficult problem. Life is sometimes not nice and you have to choose. I chose to write newtype wrappers and to accept the complexity. > I find it hard to believe [...] I think you should read up stuff about the orphan rule, then you will understand better.
Yes it does seem to be a very difficult problem. That being said, some hard problems deserve hard solutions!