Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Incremental Compilation

blog.rust-lang.org

Post reply on HN
Print view View original

41–50 of 74 posts

Re: Incremental Compilation

#41

Post by enitihas »

pcwalton wrote: post #23
flukus wrote: post #19

Earlier quoted context omitted.

I don't disagree on the header files, no sane language designed today would include that. > Java (production implementations used in practice) performs whole program optimization via its JIT. It's nice for Java, but that isn't how Rust works. Whole program optimization isn't generally something you want on dev builds, which is where incremental compilation is useful. For a production build why wouldn't you do a full…

> Whole program optimization isn't generally something you want on dev builds, which is where incremental compilation is useful. By "whole program optimization" I also include things like generic/template instantiation. Whenever you use, say, a HashMap, you have to recompile the implementation of the HashMap specialized to the size, alignment, destructor, etc. of types you're using with it. > For a production build w…

Do Java generics work like that? I thought the JVM did not know about the generics as the types are erased at compile time.

Re: Incremental Compilation

#42

Post by eonwe »

enitihas wrote: post #41
pcwalton wrote: post #23

Earlier quoted context omitted.

> Whole program optimization isn't generally something you want on dev builds, which is where incremental compilation is useful. By "whole program optimization" I also include things like generic/template instantiation. Whenever you use, say, a HashMap, you have to recompile the implementation of the HashMap specialized to the size, alignment, destructor, etc. of types you're using with it. > For a production build w…

Do Java generics work like that? I thought the JVM did not know about the generics as the types are erased at compile time.

JVM doesn't know about generic types at the runtime. There's no specialization of HashMaps, it's just arrays of points to objects all around.

With project Valhalla: https://en.wikipedia.org/wiki/Project_Valhalla_(Java_languag... there's been talk about specialization for value types.

Re: Incremental Compilation

#43

Post by enitihas »

eonwe wrote: post #42
enitihas wrote: post #41

Earlier quoted context omitted.

Do Java generics work like that? I thought the JVM did not know about the generics as the types are erased at compile time.

JVM doesn't know about generic types at the runtime. There's no specialization of HashMaps, it's just arrays of points to objects all around. With project Valhalla: https://en.wikipedia.org/wiki/Project_Valhalla_(Java_languag... there's been talk about specialization for value types.

I too thought the same. That's why pcwalton's comment confused me.

Re: Incremental Compilation

#44

Post by pjc50 »

pcwalton wrote: post #6
skybrian wrote: post #3

Nice to see it. Incremental compilation is one of those things that production compilers have but research languages usually don't have. You only need it when you have a lot of code, but then you really need it.

Production compilers often don't have it either. (Having to split up your code manually into separate .cpp [or what have you] files doesn't qualify as incremental compilation—that's just separate compilation. Incremental compilation is when the compiler automatically figures out what needs to be recompiled, even when all you hand it is a big blob of code that hasn't been manually split up by a human in any way.)

Ironically C++ often goes in the other direction: splitting up files causes all of them to rebuilt as soon as you touch a header, but combining them into a huge file gives you a huge speedup in compilation.

Re: Incremental Compilation

#45

Post by jjnoakes »

paulasmuth wrote: post #37
pcwalton wrote: post #18

Earlier quoted context omitted.

Except for inlineable functions and templates in C++—which become a greater and greater fraction of code the more "modern" your C++ gets. Not to mention that the size of your instance variables leaks into your public interface unless you manually heap allocate and use the pimpl idiom. In any case, Rust stores the interfaces to libraries in serialized binary form (including documentation, following a standardized form…

> Except for inlineable functions and templates in C++—which become a greater and greater fraction of code the more "modern" your C++ gets. The C++ ABI is currently not portable anyway. So the concern about templates (or any C++ features) forcing you to put the implementation into the header file would not apply in the scenario dllthomas was referring to: If you want to distribute your library as a binary object and…

> If you want to distribute your library as a binary object and a separate source-form interface/header today, you'll have to use a (wrapper) C API.

This is only true if you want to target different compilers. If you ship your binaries targeting a specific compiler (which is what just about every company does that I've ever worked with), you don't have any abi issues.

Re: Incremental Compilation

#46

Post by jjnoakes »

Retra wrote: post #31
dllthomas wrote: post #26

Earlier quoted context omitted.

"C++ did it right" seems a separate position from that put forward by the parent comment. I definitely think there's benefit to being able to view interface as separate from implementation, although that might well be better supported by tooling (editor folding, documentation generation) than manual maintenance - which you seem to be doing well. I don't know whether there is benefit to being able to edit interface se…

Usually if you edit the interface, you also have to edit the implementation. The only time I've ever wanted to edit an interface alone is when there wasn't any implementation written yet, which makes it a moot point anyway.

One usually does want the reverse though, editing the implementation without touching the interface.

Re: Incremental Compilation

#47

Post by guessmyname »

Off topic — I am surprised to find this statement [1] in the blog post considering how negative were my teachers in university with the "trial and error" approach of solving a problem. They always said that you should design a solution before the implementation, and I agree that it makes sense but there are cases where you need to put a print and exit to debug something; and if you think about it, that is what a debugger does.

Nowadays — and if you are doing interviews will understand — people expect you to write flawless code from scratch using a basic code editor (looking at you HackerRank) with no time for tests. Sixty minutes to write a solution for four challenges, without autocomplete, without tests. Even the genius co-worker that comes with an optimized solution for every problem would need more than fifteen minutes to solve those crazy/random challenges that at the end will measure your ability to resolve those specific questions but not problem solving skills in general.

So is trial-and-error bad or not?

[1] Much of a programmer’s time is spent in an edit-compile-debug workflow.

Re: Incremental Compilation

#48

Post by paulasmuth »

jjnoakes wrote: post #45
paulasmuth wrote: post #37

Earlier quoted context omitted.

> Except for inlineable functions and templates in C++—which become a greater and greater fraction of code the more "modern" your C++ gets. The C++ ABI is currently not portable anyway. So the concern about templates (or any C++ features) forcing you to put the implementation into the header file would not apply in the scenario dllthomas was referring to: If you want to distribute your library as a binary object and…

> If you want to distribute your library as a binary object and a separate source-form interface/header today, you'll have to use a (wrapper) C API. This is only true if you want to target different compilers. If you ship your binaries targeting a specific compiler (which is what just about every company does that I've ever worked with), you don't have any abi issues.

Actually, you can even have incompatibilities with the same compiler and different compile flags. So to reliably build a "semi-portable" c++ object one would have to ensure that all objects are compiled with the exact same (i.e. same version of the compiler/same compiler source code) and with the exact same flags. I'm sure there are some compiler vendors that offer ABI backwards-compatibility but it's not part of the C++ language per-se.

See this article by Herb Sutter on the topic: https://isocpp.org/files/papers/n4028.pdf

Re: Incremental Compilation

#49

Post by finnyspade »

Raphael_Amiard wrote: post #40

Great blog post ! One thing I wonder as a compiler head though is, what is the granularity of this stuff ? Is it per-file or per-function, or even per block ? If it's finer than per file, how do you do the tree diff ? Do you parse whole files and then do a node-by-node diff ? Do you have an incremental parser ? If I was to implement incremental compilation, I'd start with per-module I guess, because it's the atomic u…

I can actually help you out on this one. I spent the summer hacking on the Rust compiler for my internship. If I'm understanding the code correctly, this is the enum of elements represented in the dependency graph.

https://manishearth.github.io/rust-internals-docs/rustc/dep_...

Note that as per:

https://manishearth.github.io/rust-internals-docs/rustc/dep_...

The DepGraph that we actually care about is specialized to talk about DefIds. In rustc, DefIds are attached to the following things:

https://manishearth.github.io/rust-internals-docs/rustc/hir/...

This means dependencies are analyzed down to local variables and uses. But even beyond that it tracks things such as borrow checks, linting and more!

Re: Incremental Compilation

#50

Post by xyzzy_plugh »

guessmyname wrote: post #47

Off topic — I am surprised to find this statement [1] in the blog post considering how negative were my teachers in university with the "trial and error" approach of solving a problem. They always said that you should design a solution before the implementation, and I agree that it makes sense but there are cases where you need to put a print and exit to debug something; and if you think about it, that is what a debu…

I think "design a solution" can be as simple as having a rough idea of what the implementation looks like. This is solid advice.

But the journey is part of the magic. Anyone expecting candidates to produce perfect code is doing it wrong. HackerRank is garbage.

I want to understand how you think. I want you to communicate and ask questions. Yes, it can challenging to ve without your tools, with only a whiteboard, but that's okay.

If I'm interviewing you and you make a typo, great! I don't care. Maybe I'll point it out to you, but I probably won't, and I certainly won't dock you for it. And yeah, you can debug. I often encourage candidates to walk through their code with sample input, which I usually want them to come up with too.

This demonstrates an understanding of how your software works. I don't actually care if it compiles. I don't care if you're a wizard in emacs or can type a million words a minute.

I also don't care if your solution is optimal. In the real world they rarely are. But, I want to talk to you about ways we could optimize it, and maybe I'll ask you to show me what you mean.

Ultimately I want you to explain stuff to me, talk technical, and display your train of thought.

Trial-and-error is fine. It's a tool, and there are no silver bullets.

Post reply on HN