Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zig's Incremental Compilation Internals

mlugg.co.uk

Post reply on HN
Print view View original

51–60 of 292 posts

Re: Zig's Incremental Compilation Internals

#51

Post by chaz72 »

thefaux wrote: post #43

There is something that I don't fully understand about this design: why are they insisting on building a giant binary for debug builds that contains all of the code? From my perspective, a simpler approach is to generate many smaller shared libraries (perhaps at the file level) and link them in to the final binary. With this approach, the program binary would have a tiny text section and a (potentially long) list of…

If you choose to build static or dynamic libraries, I expect you’d still get to do that, and I expect as long as those don’t change and only the main binary needs patching it’d work the same. (Though I’m waiting for the next tagged release to really try it out myself, so that’s not like a guarantee or anything.)

Re: Zig's Incremental Compilation Internals

#52

Post by steveklabnik »

skavi wrote: post #41
steveklabnik wrote: post #27

Earlier quoted context omitted.

Yes, this idea is sort of similar, just like, more complicated in a sense than the clean design, which is what Zig does. Incidentally, you might want to look at gc-sections, which Rust already does. As well as https://rust-lang.github.io/rust-project-goals/2025h2/relink... which is kinda related. The sort of key here is understanding that "produce a library" means that every public item is "used" in the sense of "do…

> The real trick is to do demand-driven compilation starting from the actual final program's needs this is not so far off from hint-mostly-unused, no?

In my understanding, it's similar, yeah. I don't know enough about how hint-mostly-unused works internally to really speak to the specific differences here.

Re: Zig's Incremental Compilation Internals

#53

Post by dzaima »

thefaux wrote: post #43

There is something that I don't fully understand about this design: why are they insisting on building a giant binary for debug builds that contains all of the code? From my perspective, a simpler approach is to generate many smaller shared libraries (perhaps at the file level) and link them in to the final binary. With this approach, the program binary would have a tiny text section and a (potentially long) list of…

A quick test (C, clang) gives me that a binary depending on 1000 shared libraries, each containing a single function returning an integer, with a main function summing up the results of all those functions, takes ~270ms to run from the dynamic linker overhead. So you'd definitely want a good chunk below thousands.

(a process with 100 shared libraries takes 6ms to run, which is a lot better (0.9ms for 1 library, for reference), but, especially with in-place patching skipping work on unchanged values, static linking still has a good shot at beating dynamic linking, especially if you run the binary multiple times)

Incremental compilation generally already depends on its stored intermediate data not getting corrupted, and the final binary need not be any differently handled in that aspect.

Re: Zig's Incremental Compilation Internals

#54

Post by Maxatar »

pron wrote: post #47
steveklabnik wrote: post #44

Earlier quoted context omitted.

Okay so: in general, as a rule of thumb: anything that makes stuff have more memory safety is good. And experiments towards that end are also good. What I do not like, primarily comes down to how the project is talked about and marketed. First, because it promotes an "us vs them" mindset, instead of a "we're all trying to improve memory safety" mindset, and second, because in doing so, it also overstates its case. Th…

> it promotes an "us vs them" mindset, instead of a "we're all trying to improve memory safety" mindset But you did the same thing when, on the spectrum that ranges from C to ATS, with Zig, Rust, and Java somewhere in the middle (though all closer to C than to ATS), you declared the exact compromise that Rust makes "table stakes"! [1] Zig improves on C's memory safety when it comes to spatial safety, possibly the mor…

There's a distinction between claiming something is objectively better, which is what Fil-C claims with respect to its "idea" about memory safety compared to Rust... and claiming a personal preference for one approach versus another approach, which is what OP is saying about their own personal preference about how Zig reduces errors compared to how Rust reduces errors.

It is absolutely possible that one language might actually have an objectively better approach to memory safety than another, and in such cases it is usually possible to argue for this using sound technical or empirical arguments. But the way the author of Fil-C presents their arguments it often comes across in a kind of antagonistic manner, like he has a chip on his shoulder.

Re: Zig's Incremental Compilation Internals

#55

Post by steveklabnik »

pron wrote: post #47
steveklabnik wrote: post #44

Earlier quoted context omitted.

Okay so: in general, as a rule of thumb: anything that makes stuff have more memory safety is good. And experiments towards that end are also good. What I do not like, primarily comes down to how the project is talked about and marketed. First, because it promotes an "us vs them" mindset, instead of a "we're all trying to improve memory safety" mindset, and second, because in doing so, it also overstates its case. Th…

> it promotes an "us vs them" mindset, instead of a "we're all trying to improve memory safety" mindset But you did the same thing when, on the spectrum that ranges from C to ATS, with Zig, Rust, and Java somewhere in the middle (though all closer to C than to ATS), you declared the exact compromise that Rust makes "table stakes"! [1] Zig improves on C's memory safety when it comes to spatial safety, possibly the mor…

> But you did the same thing when,

I do not go around posting "omg Rust is SO MUCH BETTER than zig or fil-c, which are TRASH." I talk about engineering tradeoffs, and what matters to me personally. I do not say "if you use Zig, you are a bad person." I am not saying that any comparison is bad. I am saying that the way that the comparison is presented is bad. That is different.

> you declared the exact compromise that Rust makes "table stakes"

Table stakes for me.

> Zig improves on C's memory safety when it comes to spatial safety,

I agree that it's an improvement on C!

> yet you excluded it.

I said that it is not pursuing a design that I personally find compelling enough to use to write software. That doesn't mean that I think it's worthless. This whole thing started off with me talking about how much I respect the Zig project! Yet you're trying to turn this into something where I'm talking shit. I presented a specific technical tradeoff that is important to me. That is very different.

> You're trying to draw some hard line on a spectrum that passes exactly between Rust and Zig, and I'm trying to say that that line isn't there (your attempt at a definition of delineating safe and unsafe code also applies to C)

I don't believe you've shown that. And my "attempt" does apply to C: it fails the bar, because it does not delineate between a safe subset and an unsafe superset.

> you may argue that you're only talking about "memory safety" and not correctness in general,

I am in fact talking about "memory safety" and have been this whole time, yes.

> what gives memory safety value is that violations are causes of many dangerous vulnerabilities; but once, say, Java eliminates all of them, 100% of bugs/vulnerability - which are still numerous - will be caused by other problems, all potentially avoidable with ATS, so why isn't ATS table stakes?

This is just an entirely different question. Yes, there are other forms of safety that are important too. That's just not what we're talking about here.

Re: Zig's Incremental Compilation Internals

#56

Post by joshuamorton »

applfanboysbgon wrote: post #28
vlovich123 wrote: post #26

Earlier quoted context omitted.

Java and Rust have actually very similar memory safety profiles. Rust let's you within the language escape the memory safety requirements whereas Java does not but both are considered memory safe languages. Rust also enforces thread safety as well which Java does not, but the slower JVM memory model doesn't let race conditions become memory safety issues whereas Rust is lower-level like Zig/C/C++ and thus thread safe…

> Rust is ~0.2 vulnerabilities per MLoC. Java is effectively 0. C and C++ both seem to be about 1,000 vulnerabilities per MLoC. Zig is too new and hasn't had any analysis done on it, but generously it's likely at least 10-100. You have just described six orders of magnitude in your attempt to rebut pron pointing out the four languages have four levels of memory safety.

"two things are within an order of magnitude, and two other things are within an order of magnitude, and those two groups are three orders of magnitude apart" does sound like two groups to me.

Re: Zig's Incremental Compilation Internals

#57

Post by brabel »

steveklabnik wrote: post #44
mi_lk wrote: post #29

Earlier quoted context omitted.

Would you mind sharing some thoughts about fil-c? AFAICT its claims mostly check out so besides implementation details (GC?) it seems directionally good.

Okay so: in general, as a rule of thumb: anything that makes stuff have more memory safety is good. And experiments towards that end are also good. What I do not like, primarily comes down to how the project is talked about and marketed. First, because it promotes an "us vs them" mindset, instead of a "we're all trying to improve memory safety" mindset, and second, because in doing so, it also overstates its case. Th…

I’ve been watching this debate online and in my opinion both sides are guilty. Fil is intentionally trying to be funny or at least “interesting “ when he makes his points and I, for one, enjoy his humor, which includes having a go at Rust and other languages. It seems Rust people just can’t take a little criticism, even when it comes from a clearly trolling language! Yes it’s true Rust has an escape hatch, and we’ve seen serious memory safety bugs due to unsafe Rust in the wild. Fil-C does not have one, what you post seems to be internal or even temporary stuff given the author clearly has a goal of not providing one? I would say you and others need to just relax and not treat all and every Rust criticism as an offense to you.

Re: Zig's Incremental Compilation Internals

#58

Post by kibwen »

pron wrote: post #33
vlovich123 wrote: post #26

Earlier quoted context omitted.

Java and Rust have actually very similar memory safety profiles. Rust let's you within the language escape the memory safety requirements whereas Java does not but both are considered memory safe languages. Rust also enforces thread safety as well which Java does not, but the slower JVM memory model doesn't let race conditions become memory safety issues whereas Rust is lower-level like Zig/C/C++ and thus thread safe…

> Java and Rust have actually very similar memory safety profiles They really don't. Look at how many basic data structures (in the standard library or outside it) require unsafe features in Java vs Rust. > Zig has an identical memory safety profile to C It really doesn't. Zig gives you the same spatial memory safety as Rust and very much not like C (and violations of spatial memory safety are a bigger cause of vulne…

> Look at how many basic data structures (in the standard library or outside it) require unsafe features in Java vs Rust.

This is a fundamental misunderstanding of how "unsafe" code relates to a platform's trusted computing base. Rust could move all of those unsafe data structures out of the standard library and into the compiler itself, thereby reducing the amount of occurrences of the string "unsafe" in the source, code, but this would do nothing to reduce the size of the trusted computing base that Rust presents. In fact, it would decrease our confidence in that code, because Rust libraries have a robust ecosystem of tools for validating their correctness, unlike whatever bespoke IR the Rust compiler itself is emitting. Java's own data structures are implemented with the support of an extensive runtime written in C++, which forms their own trusted computing base that every user of Java relies upon, and demands just as much careful auditing as any data structure in the Rust standard library.

Re: Zig's Incremental Compilation Internals

#59

Post by applfanboysbgon »

joshuamorton wrote: post #56
applfanboysbgon wrote: post #28

Earlier quoted context omitted.

> Rust is ~0.2 vulnerabilities per MLoC. Java is effectively 0. C and C++ both seem to be about 1,000 vulnerabilities per MLoC. Zig is too new and hasn't had any analysis done on it, but generously it's likely at least 10-100. You have just described six orders of magnitude in your attempt to rebut pron pointing out the four languages have four levels of memory safety.

"two things are within an order of magnitude, and two other things are within an order of magnitude, and those two groups are three orders of magnitude apart" does sound like two groups to me.

> those two groups are three orders of magnitude apart

They aren't necessarily, though. Supposing that Zig were "10 issues per MLoC" (with just as much handwaving as the original poster), it would be equidistant from Rust and C. Java may also be more than one order of magnitude away from Rust; we say ~0 but is it 0.01, 0.001, 0.0001...? And why is "1 issue per MLoC" the acceptable metric that delineates what constitutes table-stakes memory-safe language? Because it's a nice, round-sounding number? I think 0 is a nicer, rounder number than 1, so let's call only Java table stakes and condemn all other languages to the garbage bin, tradeoffs be damned. Or would you say your arbitrary delineation point is worth more than mine?

Re: Zig's Incremental Compilation Internals

#60

Post by awesome_dude »

steveklabnik wrote: post #46
pron wrote: post #32

Earlier quoted context omitted.

So that could be a clear definition, but for it to be "table stakes" it needs to have some universal value and it doesn't (in fact, that very same definition could also classify even C as "memory safe"): https://news.ycombinator.com/item?id=49087458

I can say that something is "table stakes" for me without demanding that everyone else adhere to my values.

There's little point in me telling the world how I like my dinnercooked, unless the world both understands exactly what I mean, and cares
Post reply on HN