Live data from Hacker News

Zig's Incremental Compilation Internals

mlugg.co.uk

81–90 of 292 posts

Re: Zig's Incremental Compilation Internals

#81

Earlier quoted context omitted.

> Or would you say your arbitrary delineation point is worth more than mine? Yes for the reasons I already gave. I think that at the point that you're having to stretch the numbers from their post to the breaking point to remove the pretty clear order of magnitude differences it's not really a constructive way to engage. I think you have two groups with one at ~.1 and one on ~100. You seen to either disagree with tha…

I think it is absolutely arbitrary. First because I believe every order of magnitude is significant. You handwave away that one order of magnitude difference is fine but three is bad. I think this is hypocritcal, and that if you want to be a memory safety purist who ignores all tradeoffs and declares a language fundamentally unusable on safety grounds, even a single order of magnitude of additional issues should clea…

Yes, it's clear you take issue with the assumptions I'm making. That's okay. If you don't accept my premise that's fine, but you could just state that instead of saying that I'm being hypocritical or strutting out a variety of statements I haven't said ("you want to be a memory safety purist who ignores all tradeoffs and declares a language fundamentally unusable on safety grounds").

If you want to have a conversation with me about the things I'm talking about, I welcome it, but I don't see that happening.

Re: Zig's Incremental Compilation Internals

#82
post #16

Earlier quoted context omitted.

> I believe memory safety is table stakes I'm not sure what that means. Java lets you do many things programs may want to do in a memory-safe way but not everything. Rust lets you do fewer things than Java in a memory-safe way, but more things than Zig. Zig lets you do fewer things in a memory-safe way than Rust, but more than C. So among these four languages we already have four levels of memory safety, none of them…

I'll say the same thing I said to you as I said to Andrew, last time he and I talked about this: the way that everyone talks about memory safety (with maybe two exceptions, one okay (go) and one I dislike (fil-c)) is that "memory safe language" is about there being a clear delineation between what is memory safe and what is not, and that the unsafe aspect is a superset. Rust and Java both are memory safe, except wher…

zig creates an ir that you can use to do data dependency analysis and borrow checking.

Re: Zig's Incremental Compilation Internals

#83
post #68
post #61

Earlier quoted context omitted.

> I'm not interested in the definition so much as I am in calling it "table stakes", and so the fact that these languages satisfy their promises is uninteresting in isolation. What matters is the value of their promises. The majority of Rust programs I see, I wouldn't have written in a low-level language, so the fact that it offers memory safety for the things I don't need it to do does nothing for me. I mean, if you…

> I'd argue that the value of Rust is that it makes low-level viable for a lot of stuff that would otherwise require a lack of memory safety; a lot of it is stuff that might be written in a higher level language, but that's just because relatively few programs are impossible to write in higher level languages. Maybe, but I don't see making a low language viable for something it's not needed as offering much value. Lo…

> It's not performance (you often gain performance, especially in large programs). It's warmup and footprint.

To me, those are also performance characteristics. Maybe my view on what constitutes "performance" is broader than average here.

> When I do need that control, I find that Rust requires reaching for unsafe too frequently while still paying the full price for the safety of things I don't use (even Rust's memory management of strings doesn't give me the control I want; I have to work pretty hard for it).

Fair enough, I can't tell you that you don't have that experience when writing Rust. It's pretty different from mine though, and the experience of the large number of former C/C++ devs I've worked with after they learned Rust; the only people I've talked to with that experience didn't really try to learn Rust and went in hoping that it wouldn't work for them, which informs my perception here, but I recognize that individual experiences won't always fit into larger trends.

> Like I said, C also fits in the category, so it's not a meaningful distinction. The difference is in what you can do in the safe subset. Zig lets you do more things in a safe way than C (where the safe subset is effectively empty), Rust lets you do more safe things than Zig, and Java lets you do more safe things than Rust.

I don't think I understand what you're saying here. I don't know of a way to turn off undefined behavior by default in C and only opt into it in discrete segements of the code, but maybe I'm missing something.

> That's the very thing I'm arguing against. I'm saying that different languages eliminate different bugs at a cost (again, Zig eliminates many memory safety bugs you'd find in C or even C++, arguably the most dangerous ones). What I'm saying is that what you get and whether the price is worth it depends both on the program you're writing and on your personal preferences. Just to be clear, "preferences" doesn't mean I care more or less about correctness, but which approaches to correctness I find more or less effective, something on which there is no consensus.

It seems like you're arguing against the idea of memory safety as a category at all then. To me, "I can't write code that's memory unsafe without explicitly opting into it" seems like an objective statement, and it's objectively different than "I can't write certain types of memory safety bugs in a given language". I don't really understand what's useful about being able to write memory unsafe code without having to opt in when in practice the number of bugs from mistaken memory safety are overwhelmingly more common than the cases when you're forced to opt into unsafe because Rust forced you to work around the constraints, and even in low-level programs, the actual number of truly unsafe operations you need to do tend to be fairly low in my experience. I guess I can't say for certain that you don't truly need to do things that you're forced to write unsafe for too often, but to me, it seems like you're refusing to pay a pretty small price for mostly ideological purity rather than pragmatism.

> I think it also depends on the kinds of programs you write, because for many programs I write (and for which I pick Java) Rust's level of memory safety is too low

> [1]: For example, the fact that in Java, references are not required to be stable machine pointers opens the door to some powerful optimisations that are not available to languages where pointers are required to be machine pointers (or something close enough to them). Or the fact that low-level languages require that the machine instructions executed are those present in the compiled image (or close enough), or, if you want, caring about worst-case performance at the expense of average case performance (although both C++ and Rust specifically don't always make that easy) precludes some other very powerful optimisations.

I'm struggling to imagine what the circumstances are where these are genuine concerns rather than theoretical or premature optimizations. What are some examples of programs where you'd get better characteristics running them if they were written in Java rather than Rust due to the lack of enough "memory safety" in Rust?

Re: Zig's Incremental Compilation Internals

#84

Earlier quoted context omitted.

I'll say the same thing I said to you as I said to Andrew, last time he and I talked about this: the way that everyone talks about memory safety (with maybe two exceptions, one okay (go) and one I dislike (fil-c)) is that "memory safe language" is about there being a clear delineation between what is memory safe and what is not, and that the unsafe aspect is a superset. Rust and Java both are memory safe, except wher…

zig creates an ir that you can use to do data dependency analysis and borrow checking.

Yes, it is absolutely possible to build a language that uses Zig's compilation model and do borrow checking. Zig is not going to add a borrow checker though, so as a user, it's sort of a moot point.

Re: Zig's Incremental Compilation Internals

#85
post #29

Earlier quoted context omitted.

Would you mind sharing some thoughts about fil-c? AFAICT its claims mostly check out so besides implementation details (GC?) it seems directionally good.

Okay so: in general, as a rule of thumb: anything that makes stuff have more memory safety is good. And experiments towards that end are also good. What I do not like, primarily comes down to how the project is talked about and marketed. First, because it promotes an "us vs them" mindset, instead of a "we're all trying to improve memory safety" mindset, and second, because in doing so, it also overstates its case. Th…

I generally agree with all of this, but I'll add a few additional remarks. Because it's come up a bunch lately, I decided to do a bunch of code review/audit of the Fil-C codebase, and I'd say while it's got a lot of good bones, there's a long way to go to being a foundation I'd be ready to build on. I've reported a few UAF's upstream, and I've got a few PRs I'll add on, but if it only took me a day or two to find some of these big holes, I'm sure there's more lurking under the surface. I'd consider it to at this point be more of an engineering demo that this approach is feasible and tractable, but not a production ready language that I'd want to ship code in.

On the muddling the discourse, I'm not on twitter and don't engage there, so I don't have an opinion on that, but I did come across https://news.ycombinator.com/item?id=49044561 recently, and I just don't see how the author can make such bold claims while examples like the one Steve provided above are still in the language. Corrupting memory in Fil-C is still easy, type confusion is still easy, intra-object overflows are still easy. Fil-C prevents a range of classes of bugs from being exploitable, but it doesn't stop the bugs from happening.

Re: Zig's Incremental Compilation Internals

#86
post #8

Earlier quoted context omitted.

The thing with rust is that you get safety with slow compilation, it's a tradeoff. Zig doesn't have the same safety guarantees, it's on the dev to use safe coding patterns, so the tradeoff for safety is discipline or experience.

(tongue in cheek) It seems recent history has shown that zig can get you to working software faster, then you can port it to rust once you are acquired or find market fit? Maybe at some point in the future zig could add a rust compilation target ( like with `-ofmt=c` )...

just staple a borrow checker to zig. it seems pretty doable as per my experiments

Re: Zig's Incremental Compilation Internals

#87

Earlier quoted context omitted.

zig creates an ir that you can use to do data dependency analysis and borrow checking.

Yes, it is absolutely possible to build a language that uses Zig's compilation model and do borrow checking. Zig is not going to add a borrow checker though, so as a user, it's sort of a moot point.

no. let me be clearer; it is possible to intercept zigs ir from the compiler NOW (well, 15.2 proven) and have a third party package do borrow checking from the data that flow through, without changing zig (think "how miri works without changing rust"). this is not currently directly possible without changing the compiler (~ 50 loc), however the core team has indicated that exporting ir, the only change needed, will be a supported feature once the language stabilizes.

Re: Zig's Incremental Compilation Internals

#88
post #31
post #19

Earlier quoted context omitted.

> I'm not sure what that means. Java lets you do many things in a memory-safe way but not everything. Rust lets you do fewer things than Java in a memory-safe way, but more things than Zig. I don't think I agree with this framing. The question to me isn't "what can you do while being memory safe", it's "can you accidentally do something memory unsafe without noticing?" Rust and Java are the same here; you need to exp…

That framing may seem intellectually satisfying, but it's not useful in practice. Consider the extreme edge case of C: We can clearly mechanically delineate between the empty program and a non-empty one, we call the empty program safe and any program that isn't empty unsafe (i.e. C is memory-safe if you want to do nothing and not if you want to do anything). And so, we also have this property that in C you can't do a…

Do you find most of your rust code is unsafe? Because I also find I do some unsafe stuff, because of the algorithms I work on I often end up with some unchecked array accesses and a couple of raw pointers into those arrays I pass around. But 98% of the code is safe and I find this makes it easier to reason about.

Re: Zig's Incremental Compilation Internals

#89

Earlier quoted context omitted.

Yes, it is absolutely possible to build a language that uses Zig's compilation model and do borrow checking. Zig is not going to add a borrow checker though, so as a user, it's sort of a moot point.

no. let me be clearer; it is possible to intercept zigs ir from the compiler NOW (well, 15.2 proven) and have a third party package do borrow checking from the data that flow through, without changing zig (think "how miri works without changing rust"). this is not currently directly possible without changing the compiler (~ 50 loc), however the core team has indicated that exporting ir, the only change needed, will b…

How do you get the information to check properly without lifetimes in the signature?

Re: Zig's Incremental Compilation Internals

#90
post #80

Earlier quoted context omitted.

> a plan to transition to it for C compilation That's not planned AFAIK (see https://github.com/ziglang/zig/issues/16269 ). `translate-c` is really only intended for header translation, not C source code. See https://github.com/ziglang/zig/issues/20875 for the (not fully fleshed out yet) plans around C compilation.

Zig already uses arocc! https://codeberg.org/ziglang/zig/src/branch/master/lib/compi...

That's there for translate-c, it's not used for compiling C code

(To be clear, squeek502 is a part of the Zig core team [0], so he knows what he's talking about :D)

[0]: https://ziglang.org/news/welcoming-new-team-members/

Post reply on HN