Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Incremental Compilation

blog.rust-lang.org

Post reply on HN
Print view View original

51–60 of 74 posts

Re: Incremental Compilation

#51

Post by Manishearth »

guessmyname wrote: post #47

Off topic — I am surprised to find this statement [1] in the blog post considering how negative were my teachers in university with the "trial and error" approach of solving a problem. They always said that you should design a solution before the implementation, and I agree that it makes sense but there are cases where you need to put a print and exit to debug something; and if you think about it, that is what a debu…

Designing a solution is still far from having code that works. You have designed a solution, and now you need to test it. In significantly complicated codebases, there are often all kinds of obscure issues that can crop up and make your (manual or automated) testing fail. These are not things you can think of beforehand in your design because it is not possible for you to have known about all of these issues.

In fact, later in your comment you allude to this yourself -- "with no time for tests". You're worried about interviews not having time for tests. But tests are a edit-compile-debug feedback loop. If a test fails, you need to debug it, fix the issue, and recompile. Often you have to do this many times. This is precisely what the blog post is talking about.

You're generalizing the term "trial and error" to mean something more than what your teachers probably meant.

(I do agree that those interview practices are silly. But the blog post is not talking about stuff like that.)

Re: Incremental Compilation

#52

Post by Manishearth »

barrkel wrote: post #39
pcwalton wrote: post #6

Earlier quoted context omitted.

Production compilers often don't have it either. (Having to split up your code manually into separate .cpp [or what have you] files doesn't qualify as incremental compilation—that's just separate compilation. Incremental compilation is when the compiler automatically figures out what needs to be recompiled, even when all you hand it is a big blob of code that hasn't been manually split up by a human in any way.)

The Delphi compiler has incremental compilation, but by your definition it wouldn't. Delphi uses the source to figure out dependencies. That is, the compiler has make logic, and traces through the 'uses' declarations to recursively discover all the source and object files. It compares timestamps to discover which object files are out of date and need to be recompiled. Thus, if you touch a single file and do a compile…

I don't see where in pcwalton's comment he alludes that the Delphi compiler's definition of incremental compilation doesn't match his own.

He explicitly says "Incremental compilation is when the compiler automatically figures out what needs to be recompiled".

(He does say that production compilers don't have incremental compilation, but that's not a matter of definition, and I read that to mean "usually don't")

Re: Incremental Compilation

#53

Post by Manishearth »

berkut wrote: post #34
dllthomas wrote: post #26

Earlier quoted context omitted.

"C++ did it right" seems a separate position from that put forward by the parent comment. I definitely think there's benefit to being able to view interface as separate from implementation, although that might well be better supported by tooling (editor folding, documentation generation) than manual maintenance - which you seem to be doing well. I don't know whether there is benefit to being able to edit interface se…

Writing applications which have an API other people write plugins against is one: commercial apps ship the .h files, against which plugin-writers only have those and build plugins for the host apps. Yes you still obviously have to keep them in sync, but the point is you (as a commercial software vendor) can ship just the headers and keep the internal implementation details secret and change how things work completely…

This isn't really an argument for .h files though.

.h files are an extra amount of work you have to put in. In the case of plugin APIs, this is work you'd have to put in anyway (for a fraction of the .h files -- the ones which are part of the API). But that work doesn't have to be put in elsewhere.

This is an argument for interface description files for plugins, but that could always have been done by a language with idl files or something. Indeed, apps in most other languages define interfaces via .h files because that's the de-facto plugin API description format, but they don't use .h files everywhere in the source. They only use them where necessary for the API.

Re: Incremental Compilation

#54

Post by jjnoakes »

paulasmuth wrote: post #48
jjnoakes wrote: post #45

Earlier quoted context omitted.

> If you want to distribute your library as a binary object and a separate source-form interface/header today, you'll have to use a (wrapper) C API. This is only true if you want to target different compilers. If you ship your binaries targeting a specific compiler (which is what just about every company does that I've ever worked with), you don't have any abi issues.

Actually, you can even have incompatibilities with the same compiler and different compile flags. So to reliably build a "semi-portable" c++ object one would have to ensure that all objects are compiled with the exact same (i.e. same version of the compiler/same compiler source code) and with the exact same flags. I'm sure there are some compiler vendors that offer ABI backwards-compatibility but it's not part of the…

Oh I'm well aware; as my comment indicated, I've been doing this a long time. I didn't want to get in to the full details in a HN comment, but yes, there's definitely a few things you have to coordinate on between vendors if you want to ship C++ libraries that work together.

Re: Incremental Compilation

#55

Post by mtanski »

ngrilly wrote: post #38
mtanski wrote: post #16

Earlier quoted context omitted.

The way Rust works today is that you compile a whole crate together (all the files). The side effect of that is that the compiler can optimize the whole create vs what's in one file. You get an effect that analogous to LTO (like in C/C++). Conversely, if you now go to incremental builds the opportunity for LTO style optimizations. Thus you end up with a worse runtime performance in incremental compilation. Caveats an…

> Thus you end up with a worse runtime performance in incremental compilation. Are you sure about this (in the current implementation)?

The article it self says "[...] This is because incremental compilation splits the code into smaller optimization units than a regular compilation session, resulting in less time optimizing, but also in less efficient runtime code."

Re: Incremental Compilation

#56

Post by pjmlp »

eonwe wrote: post #42
enitihas wrote: post #41

Earlier quoted context omitted.

Do Java generics work like that? I thought the JVM did not know about the generics as the types are erased at compile time.

JVM doesn't know about generic types at the runtime. There's no specialization of HashMaps, it's just arrays of points to objects all around. With project Valhalla: https://en.wikipedia.org/wiki/Project_Valhalla_(Java_languag... there's been talk about specialization for value types.

It is already available for those that want to play with it.

https://adoptopenjdk.gitbooks.io/adoptopenjdk-getting-starte...

Re: Incremental Compilation

#57

Post by ngrilly »

mtanski wrote: post #55
ngrilly wrote: post #38

Earlier quoted context omitted.

> Thus you end up with a worse runtime performance in incremental compilation. Are you sure about this (in the current implementation)?

The article it self says "[...] This is because incremental compilation splits the code into smaller optimization units than a regular compilation session, resulting in less time optimizing, but also in less efficient runtime code."

Thanks, you're right. I missed that.

Re: Incremental Compilation

#58

Post by pjmlp »

berkut wrote: post #34
dllthomas wrote: post #26

Earlier quoted context omitted.

"C++ did it right" seems a separate position from that put forward by the parent comment. I definitely think there's benefit to being able to view interface as separate from implementation, although that might well be better supported by tooling (editor folding, documentation generation) than manual maintenance - which you seem to be doing well. I don't know whether there is benefit to being able to edit interface se…

Writing applications which have an API other people write plugins against is one: commercial apps ship the .h files, against which plugin-writers only have those and build plugins for the host apps. Yes you still obviously have to keep them in sync, but the point is you (as a commercial software vendor) can ship just the headers and keep the internal implementation details secret and change how things work completely…

This is a common myth about .h files versus modules.

Most languages with native support for modules do have the necessary information stored in the binary file to enable this scenario.

The only problem is exposing those plugins across languages, but it isn't a big problem if the OS has a rich ABI instead of a C one. For example COM on Windows, or WinRT the improved version of it.

Re: Incremental Compilation

#59

Post by pjmlp »

paulasmuth wrote: post #37
pcwalton wrote: post #18

Earlier quoted context omitted.

Except for inlineable functions and templates in C++—which become a greater and greater fraction of code the more "modern" your C++ gets. Not to mention that the size of your instance variables leaks into your public interface unless you manually heap allocate and use the pimpl idiom. In any case, Rust stores the interfaces to libraries in serialized binary form (including documentation, following a standardized form…

> Except for inlineable functions and templates in C++—which become a greater and greater fraction of code the more "modern" your C++ gets. The C++ ABI is currently not portable anyway. So the concern about templates (or any C++ features) forcing you to put the implementation into the header file would not apply in the scenario dllthomas was referring to: If you want to distribute your library as a binary object and…

> If you want to distribute your library as a binary object and a separate source-form interface/header today, you'll have to use a (wrapper) C API. Regardless of how "modern" the C++ code is.

Not really.

Those of us on Windows make use of COM, or since Windows 8, UWP components (formally known as WinRT).

Re: Incremental Compilation

#60

Post by coldtea »

Manishearth wrote: post #52
barrkel wrote: post #39

Earlier quoted context omitted.

The Delphi compiler has incremental compilation, but by your definition it wouldn't. Delphi uses the source to figure out dependencies. That is, the compiler has make logic, and traces through the 'uses' declarations to recursively discover all the source and object files. It compares timestamps to discover which object files are out of date and need to be recompiled. Thus, if you touch a single file and do a compile…

I don't see where in pcwalton's comment he alludes that the Delphi compiler's definition of incremental compilation doesn't match his own. He explicitly says "Incremental compilation is when the compiler automatically figures out what needs to be recompiled". (He does say that production compilers don't have incremental compilation, but that's not a matter of definition, and I read that to mean "usually don't")

>He explicitly says "Incremental compilation is when the compiler automatically figures out what needs to be recompiled".

He also says that merely using the separate files to deduce this (which the parent says is what Delphi does) is not incremental compilation.

Post reply on HN