Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Incremental Compilation

blog.rust-lang.org

Post reply on HN
Print view View original

11–20 of 74 posts

Re: Incremental Compilation

#11

Post by flukus »

pcwalton wrote: post #10
flukus wrote: post #8

How does this differ from what make files have been doing for decades?

You don't have to split up your code manually. You just write one Rust crate, splitting your code into files on whatever granularity you like (with mutual recursion fully supported), and the Rust compiler automatically does the rest. You don't have to write tedious header files; the compiler figures out all the dependencies automatically.

I don't split up my code manually for the compiler, I do it for the humans, being compiler friendly is just a pleasant side effect. You can build java files the same way without header files, worst case scenario is you have some fat compilation units.

Re: Incremental Compilation

#12

Post by pcwalton »

bla2 wrote: post #7

From an uninformed distance this looks a bit underwhelming: initial builds are slower, incremental builds are not _that_ much faster, and build outputs are no longer deterministic functions of just the build inputs (which is useful for example for distributed build systems).

> incremental builds are not _that_ much faster

Right now, they aren't. This will change as things improve. Remember, this is just the groundwork to allow incremental compilation to work at all. There are lots of places where the compiler now needs to be updated to use that framework.

> build outputs are no longer deterministic functions of just the build inputs (which is useful for example for distributed build systems).

You don't have to use incremental compilation if you care about this problem. But you will want to, because it isn't a problem in practice :)

There is no reason why distributed build systems couldn't simply be made aware of incremental compilation.

Re: Incremental Compilation

#13

Post by pcwalton »

flukus wrote: post #11
pcwalton wrote: post #10

Earlier quoted context omitted.

You don't have to split up your code manually. You just write one Rust crate, splitting your code into files on whatever granularity you like (with mutual recursion fully supported), and the Rust compiler automatically does the rest. You don't have to write tedious header files; the compiler figures out all the dependencies automatically.

I don't split up my code manually for the compiler, I do it for the humans, being compiler friendly is just a pleasant side effect. You can build java files the same way without header files, worst case scenario is you have some fat compilation units.

> I don't split up my code manually for the compiler, I do it for the humans, being compiler friendly is just a pleasant side effect.

There is no human benefit to having to copy and paste function signatures—and worse, the bodies of functions you want to be inlined, including all templates—into header files. There's also no benefit to having the compiler parse hundreds of KB of header files over and over and over again.

> You can build java files the same way without header files, worst case scenario is you have some fat compilation units.

Java (production implementations used in practice) performs whole program optimization via its JIT. It's nice for Java, but that isn't how Rust works.

Re: Incremental Compilation

#14

Post by jjnoakes »

flukus wrote: post #11
pcwalton wrote: post #10

Earlier quoted context omitted.

You don't have to split up your code manually. You just write one Rust crate, splitting your code into files on whatever granularity you like (with mutual recursion fully supported), and the Rust compiler automatically does the rest. You don't have to write tedious header files; the compiler figures out all the dependencies automatically.

I don't split up my code manually for the compiler, I do it for the humans, being compiler friendly is just a pleasant side effect. You can build java files the same way without header files, worst case scenario is you have some fat compilation units.

Splitting up code into headers and sources also helps keep interfaces separate from implementations. It allows one to not only read just the header for the full interface, but it also allows one to distribute the implementation in binary form and the interface in source form, which is important for companies.

Re: Incremental Compilation

#15

Post by flukus »

Jtsummers wrote: post #9
flukus wrote: post #8

How does this differ from what make files have been doing for decades?

Seems to be finer grained. Make knows I only changed foo.c and only rebuilds what depends on that. This sees that foo.rs changed and reparsed it. Sees the AST didn't change, maybe we added comments or reformatted it, and stops.

Does it track changes or changes to the surface area? If I changed the implementation of a function but not the signature, would this cause a rebuild of the dependents?

Re: Incremental Compilation

#16

Post by mtanski »

bla2 wrote: post #7

From an uninformed distance this looks a bit underwhelming: initial builds are slower, incremental builds are not _that_ much faster, and build outputs are no longer deterministic functions of just the build inputs (which is useful for example for distributed build systems).

The way Rust works today is that you compile a whole crate together (all the files). The side effect of that is that the compiler can optimize the whole create vs what's in one file. You get an effect that analogous to LTO (like in C/C++). Conversely, if you now go to incremental builds the opportunity for LTO style optimizations. Thus you end up with a worse runtime performance in incremental compilation.

Caveats and fine details missing all over the place. More focused on the general point.

Re: Incremental Compilation

#17

Post by pcwalton »

flukus wrote: post #15
Jtsummers wrote: post #9

Earlier quoted context omitted.

Seems to be finer grained. Make knows I only changed foo.c and only rebuilds what depends on that. This sees that foo.rs changed and reparsed it. Sees the AST didn't change, maybe we added comments or reformatted it, and stops.

Does it track changes or changes to the surface area? If I changed the implementation of a function but not the signature, would this cause a rebuild of the dependents?

No, changing the implementation of a function does not cause a rebuild. If support is added for optimizing builds via LLVM's ThinLTO, however, then it might if LLVM decided to inline the function.

Re: Incremental Compilation

#18

Post by pcwalton »

jjnoakes wrote: post #14
flukus wrote: post #11

Earlier quoted context omitted.

I don't split up my code manually for the compiler, I do it for the humans, being compiler friendly is just a pleasant side effect. You can build java files the same way without header files, worst case scenario is you have some fat compilation units.

Splitting up code into headers and sources also helps keep interfaces separate from implementations. It allows one to not only read just the header for the full interface, but it also allows one to distribute the implementation in binary form and the interface in source form, which is important for companies.

Except for inlineable functions and templates in C++—which become a greater and greater fraction of code the more "modern" your C++ gets. Not to mention that the size of your instance variables leaks into your public interface unless you manually heap allocate and use the pimpl idiom.

In any case, Rust stores the interfaces to libraries in serialized binary form (including documentation, following a standardized format understood by rustdoc), and they can be readily extracted from crates using tools that ship with the language. So what you describe isn't a benefit of headers anyhow.

Re: Incremental Compilation

#19

Post by flukus »

pcwalton wrote: post #13
flukus wrote: post #11

Earlier quoted context omitted.

I don't split up my code manually for the compiler, I do it for the humans, being compiler friendly is just a pleasant side effect. You can build java files the same way without header files, worst case scenario is you have some fat compilation units.

> I don't split up my code manually for the compiler, I do it for the humans, being compiler friendly is just a pleasant side effect. There is no human benefit to having to copy and paste function signatures—and worse, the bodies of functions you want to be inlined, including all templates—into header files. There's also no benefit to having the compiler parse hundreds of KB of header files over and over and over aga…

I don't disagree on the header files, no sane language designed today would include that.

> Java (production implementations used in practice) performs whole program optimization via its JIT. It's nice for Java, but that isn't how Rust works.

Whole program optimization isn't generally something you want on dev builds, which is where incremental compilation is useful. For a production build why wouldn't you do a full rebuild anyway?

Re: Incremental Compilation

#20

Post by flukus »

pcwalton wrote: post #17
flukus wrote: post #15

Earlier quoted context omitted.

Does it track changes or changes to the surface area? If I changed the implementation of a function but not the signature, would this cause a rebuild of the dependents?

No, changing the implementation of a function does not cause a rebuild. If support is added for optimizing builds via LLVM's ThinLTO, however, then it might if LLVM decided to inline the function.

Now I see the value, thanks.
Post reply on HN