Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zig's Incremental Compilation Internals

mlugg.co.uk

Post reply on HN
Print view View original

191–200 of 292 posts

Re: Zig's Incremental Compilation Internals

#191

Post by tancop »

steveklabnik wrote: post #132
dminik wrote: post #128

Earlier quoted context omitted.

A really dumb 1 AM question. If there is a lot of work thrown away because stuff is compiled even if not needed, would making every function generic and delaying compilation until instantiation help here? Note that it's not a serious suggestion, but I wonder what effect it would have on build times.

I mean, the core thing is like, you have to have a compiler codebase (and language semantics) that's designed around being able to delay in the first place to be able to even try this, and once you've gotten that in place, well, it's not really about this specific idea anymore.

Would this work better in a language where generics are vtable based by default and only monomorphized as a form of LTO? If you have a Zig style machine IR where function calls are a "fake instruction" you can decide between direct and virtual call at final codegen time.

That would mean generics in shared libraries are possible without hacks but you can always choose to statically link and monomorphize for speed. Libraries could even ship both the vtable based catch all code and specializations for common types as a fast path (but that would probably need a custom dynamic linker).

Re: Zig's Incremental Compilation Internals

#192

Post by imtringued »

chaz72 wrote: post #65
brabel wrote: post #57

Earlier quoted context omitted.

I’ve been watching this debate online and in my opinion both sides are guilty. Fil is intentionally trying to be funny or at least “interesting “ when he makes his points and I, for one, enjoy his humor, which includes having a go at Rust and other languages. It seems Rust people just can’t take a little criticism, even when it comes from a clearly trolling language! Yes it’s true Rust has an escape hatch, and we’ve…

Yes! I think Fil is great so far but I think as he gets a bigger audience he should, well, consider that and focus on clarity a little more than humor. You can see Andrew’s growth in that respect. Rust folks, this whole thing is a thread about Zig’s new feature - not even a memory safety-related feature! - and we cannot spend the whole damn time talking about Rust. Steve, even you - I don’t believe I have ever seen y…

>Steve, even you - I don’t believe I have ever seen you say an unkind word. But have you considered that it may be unkind to have written more than half of the words on a thread about a Zig performance feature?

You're accusing steve of what pron is doing... I mean look at how much text pron wrote, it is much more than steve wrote, and how pron is constantly skirting the edges of the HN guidelines.

Re: Zig's Incremental Compilation Internals

#193

Post by wannabe44 »

archargelod wrote: post #139
josephg wrote: post #137

Earlier quoted context omitted.

My main problem with Fil-C is that it combines the worst parts of C with the worst parts of a GC language. In a language like C (or Zig), you need to manually manage memory. This makes programming a lot more complex, and it's really easy to accidentally mess up. Especially in large projects which have a lot of separate modules. The biggest advantage of using a garbage collector is that you don't have to think about f…

> GC languages are slower This is not necessarily true. It depends on a language, e.g. Go is slow, Nim[0] is extremely fast with conventional GC and slightly faster with ARC/ORC[1]. GC programs can be faster than manually managed ones in some cases. It's just manual memory management gives you more control of where and when free is called. And a good type system is a privelege that gives Nim more control with destruc…

You provide no trustworthy benchmark for your claim that Go is slow, or that nim is faster than that. Many engineer hours have been put on the go GC and compiler

Re: Zig's Incremental Compilation Internals

#194

Post by [deleted] »

wannabe44 wrote: post #193
archargelod wrote: post #139

Earlier quoted context omitted.

> GC languages are slower This is not necessarily true. It depends on a language, e.g. Go is slow, Nim[0] is extremely fast with conventional GC and slightly faster with ARC/ORC[1]. GC programs can be faster than manually managed ones in some cases. It's just manual memory management gives you more control of where and when free is called. And a good type system is a privelege that gives Nim more control with destruc…

You provide no trustworthy benchmark for your claim that Go is slow, or that nim is faster than that. Many engineer hours have been put on the go GC and compiler

[deleted]

Re: Zig's Incremental Compilation Internals

#195

Post by mlugg »

tancop wrote: post #188
SkiFire13 wrote: post #159

Earlier quoted context omitted.

> To me compilers should be content-addressed databases Rustc already does something like this. The issues are: - when you have to rehash everything to check that they indeed didn't change from the previous compilation. For big projects this takes _a lot_ of time. - when small changes do indeed change the hash of a lot of seemingly unrelated code, which is more common than you might think.

You dont need to rehash everything if you do multi level hashing (file -> mod/impl block -> function). The expensive part is doing lots of hash passes over small parts of a file, so if you break as early as possible the moment you can guarantee that part is unchanged you save a lot of time. And if you assume (and document) that libraries need to be rebuilt manually you can completely skip checking them. OP handles th…

> OP handles the small change problem by hashing IR instead of source text

OP here---I think you got confused somewhere, because this isn't true! The hashes we take are of source code, they're just stored inside of the first IR. Source code is just pretty convenient to hash. If the change does turn out to be something trivial, we'll only invalidate the first-order dependencies of the source hash, which is never usually a big deal. In a particularly bad case, maybe we re-analyze, re-codegen, and re-link many different instances of a generic function, but even that wouldn't usually take long at all.

Also, it's actually important that we do notice these kinds of "trivial" changes in some parts of the pipeline---see Andrew's comment [0] on the Lobsters discussion for this post.

[0]: https://lobste.rs/s/rmzzdb/inside_zig_s_incremental_compilat...

Re: Zig's Incremental Compilation Internals

#196

Post by mbrock »

steveklabnik wrote: post #55
pron wrote: post #47

Earlier quoted context omitted.

> it promotes an "us vs them" mindset, instead of a "we're all trying to improve memory safety" mindset But you did the same thing when, on the spectrum that ranges from C to ATS, with Zig, Rust, and Java somewhere in the middle (though all closer to C than to ATS), you declared the exact compromise that Rust makes "table stakes"! [1] Zig improves on C's memory safety when it comes to spatial safety, possibly the mor…

> But you did the same thing when, I do not go around posting "omg Rust is SO MUCH BETTER than zig or fil-c, which are TRASH." I talk about engineering tradeoffs, and what matters to me personally. I do not say "if you use Zig, you are a bad person." I am not saying that any comparison is bad. I am saying that the way that the comparison is presented is bad. That is different. > you declared the exact compromise that…

> I do not go around posting "omg Rust is SO MUCH BETTER than zig or fil-c, which are TRASH." I talk about engineering tradeoffs, and what matters to me personally. I do not say "if you use Zig, you are a bad person." I am not saying that any comparison is bad. I am saying that the way that the comparison is presented is bad. That is different.

Oh, who is saying things like that? Do you mean to imply that this kind of vitriol is characteristic of the Fil-C project?

Re: Zig's Incremental Compilation Internals

#197

Post by mbrock »

steveklabnik wrote: post #44
mi_lk wrote: post #29

Earlier quoted context omitted.

Would you mind sharing some thoughts about fil-c? AFAICT its claims mostly check out so besides implementation details (GC?) it seems directionally good.

Okay so: in general, as a rule of thumb: anything that makes stuff have more memory safety is good. And experiments towards that end are also good. What I do not like, primarily comes down to how the project is talked about and marketed. First, because it promotes an "us vs them" mindset, instead of a "we're all trying to improve memory safety" mindset, and second, because in doing so, it also overstates its case. Th…

There is an unsafe call primitive in stdfil that was used to support constant time crypto functions when Fil-C didn't have support for inline assembly.

Fil-C now has support for inline assembly, so it's not needed anymore, and I believe indeed the intention is to remove it, since Fil-C is not supposed to have any unsafe hatches.

Fil-C is not Linux only by design, that's completely false.

By the way, if you don't want a culture war, you gotta stop warring.

Re: Zig's Incremental Compilation Internals

#198

Post by mbrock »

josephg wrote: post #137
mi_lk wrote: post #29

Earlier quoted context omitted.

Would you mind sharing some thoughts about fil-c? AFAICT its claims mostly check out so besides implementation details (GC?) it seems directionally good.

My main problem with Fil-C is that it combines the worst parts of C with the worst parts of a GC language. In a language like C (or Zig), you need to manually manage memory. This makes programming a lot more complex, and it's really easy to accidentally mess up. Especially in large projects which have a lot of separate modules. The biggest advantage of using a garbage collector is that you don't have to think about f…

> Fil-C is the worst of all worlds here. Like C, it forces you to manually manage your own memory.

Can you expand on this?

Re: Zig's Incremental Compilation Internals

#199

Post by archargelod »

wannabe44 wrote: post #193
archargelod wrote: post #139

Earlier quoted context omitted.

> GC languages are slower This is not necessarily true. It depends on a language, e.g. Go is slow, Nim[0] is extremely fast with conventional GC and slightly faster with ARC/ORC[1]. GC programs can be faster than manually managed ones in some cases. It's just manual memory management gives you more control of where and when free is called. And a good type system is a privelege that gives Nim more control with destruc…

You provide no trustworthy benchmark for your claim that Go is slow, or that nim is faster than that. Many engineer hours have been put on the go GC and compiler

You can find a number of benchmarks with similar results, but I really like this one[0]. I didn't check all algorithms for differences, but just looking on mandelbrot bench - both are version 1 (without hacks or some manual loop unrolling thrown in) and both are implemented in semantically identical code. Nevertheless, Nim version runs 7x faster and, literally, on par with C.

[0] - https://programming-language-benchmarks.vercel.app/nim-vs-go

Re: Zig's Incremental Compilation Internals

#200

Post by mbrock »

steveklabnik wrote: post #44
mi_lk wrote: post #29

Earlier quoted context omitted.

Would you mind sharing some thoughts about fil-c? AFAICT its claims mostly check out so besides implementation details (GC?) it seems directionally good.

Okay so: in general, as a rule of thumb: anything that makes stuff have more memory safety is good. And experiments towards that end are also good. What I do not like, primarily comes down to how the project is talked about and marketed. First, because it promotes an "us vs them" mindset, instead of a "we're all trying to improve memory safety" mindset, and second, because in doing so, it also overstates its case. Th…

Fil-C's understanding of memory safety is not "its own" idiosyncratic made up definition. Memory safety in the whole tradition that includes CHERI etc is defined in terms of objects and allocations. In C structs and arrays are not object boundaries. So CHERI will have the same semantics as Fil-C in your struct example, unless you enable a compatibility-breaking mode, which Fil-C could very plausibly acquire too, at the same cost of breaking semantic compatibility with the C/C++ semantics.
Post reply on HN