Live data from Hacker News

Zig's Incremental Compilation Internals

mlugg.co.uk

181–190 of 292 posts

Re: Zig's Incremental Compilation Internals

#181

I just looked up a Hello World program from the Zig Wikipedia article: const std = @import("std"); const File = std.Io.File; pub fn main(init: std.process.Init) !void { _ = try File.stdout().writeStreamingAll(init.io, "Hello, World!\n"); } That's a lot to follow, just to output a plan-text message, especially after this line: "The primary goal of Zig is to be a better solution to the sorts of tasks that are currently…

That's the 'official' hello world which is indeed a bit verbose (it's "correct" in the way that it generally shows how to stream formatted text to stdout though).

Arguably this is the more beginner friendly version, this prints to stderr though:

hello.zig:

    const print = @import("std").debug.print;

    pub fn main() void {
        print("Hello World!\n", .{});
    }
...and then

    zig run hello.zig

Re: Zig's Incremental Compilation Internals

#182
post #43

There is something that I don't fully understand about this design: why are they insisting on building a giant binary for debug builds that contains all of the code? From my perspective, a simpler approach is to generate many smaller shared libraries (perhaps at the file level) and link them in to the final binary. With this approach, the program binary would have a tiny text section and a (potentially long) list of…

Oh, I forgot to respond to the "corrupted binary" thing in my other reply, sorry.

Right now, yep, corrupting the binary that the compiler reads would crash the compiler. In future, we want to detect the corruption and force a clean build. Note however that the Zig compiler is writing the binary to its internal cache directory (typically `.zig-cache/`), and the build system then copies the final artifact to your output ("prefix") directory (`zig-out/` by default), so it doesn't matter if the user messes with the final binary in `zig-out/bin/my_program`, because that's just a copy. Lastly, this is not implemented yet, but we will definitely make sure that Ctrl+C leave the cache in a clean state. I'm pretty sure our incremental compilation system has a nice property that you can just cancel an update partway through and continue it later without too much effort. See also Zig's IO interface [0] for information about cancelation. The compiler should already support graceful-ish cancelation internally (I won't claim to have verified this, because we never actually do cancel compilation right now, but I'm not aware of any glaring issues!). I don't think it'd be a crazy amount of work to improve that so that it also leaves incremental compilation in a valid state.

[0]: https://kristoff.it/blog/zig-new-async-io/

Re: Zig's Incremental Compilation Internals

#183

I just looked up a Hello World program from the Zig Wikipedia article: const std = @import("std"); const File = std.Io.File; pub fn main(init: std.process.Init) !void { _ = try File.stdout().writeStreamingAll(init.io, "Hello, World!\n"); } That's a lot to follow, just to output a plan-text message, especially after this line: "The primary goal of Zig is to be a better solution to the sorts of tasks that are currently…

Whats the point of evaluating technology from hello world programs?

Tbf, it's a useful indicator whether a language follows the "simple things should be simple, complex things should be possible" principle. The vanilla 'Hello World' should always be an example of the "simple things should be simple" part.

Re: Zig's Incremental Compilation Internals

#184

Earlier quoted context omitted.

> GC languages are slower This is not necessarily true. It depends on a language, e.g. Go is slow, Nim[0] is extremely fast with conventional GC and slightly faster with ARC/ORC[1]. GC programs can be faster than manually managed ones in some cases. It's just manual memory management gives you more control of where and when free is called. And a good type system is a privelege that gives Nim more control with destruc…

You can always use things like arenas in C and get similar speed ups without GC overhead. If you know your memory lifetimes in advanced, avoiding granular malloc/free calls is pretty straightforward. A GC language doesn’t usually offer such options.

You can easily use arena allocation in a GCed language. With modern GCs there's not usually much performance benefit, so it tends to be limited to hot paths.

Re: Zig's Incremental Compilation Internals

#185

[flagged]

Neither of those links involves Klabnik. And the second one (about swatting a linux dev) says "Others think someone from the Rust (programming language, not video game) development community was responsible due to how critical René has been of that project, but those claims are entirely unsubstantiated." -- Also wild to complain about botting when I've seen half a dozen accounts that didn't exist an hour ago all pop…

And the first one is about Christoph Hellwig who has a long history of being awful to communicate with. He should have been kicked out of the Linux community two decades ago for that reason alone. For some reason, he has decided that he doesn't like Rust and he does what he can to keep it far, far away from any code he has anything to do with. If that requires 10x more work for others, that is a price he is willing to pay.

Re: Zig's Incremental Compilation Internals

#186
post #105

Earlier quoted context omitted.

Might get downvoted but was thinking this exact thing when reading this debate. Rustations have this very bad habit (IMO) of pushing the "my language is better than yours" to an extreme that I haven't seen elsewhere (but I don't frequent a huge number of language circles so...). Yet when it is done to them they get all upset about it.

It's funny because my comment was intended the other way, i.e. the Zig community & core team is antagonistic towards Rust so they shouldn't be surprised when they get pushback, like in this thread. But it really does go both ways when you look at how the Rust community has acted historically. But hey, nerd holy wars have existed since the internet began. I use vim btw...oh you use emacs? You're an idiot. Etc etc.

I'm not sure this is even about Zig or Rust. I'm honestly getting the feeling that the actual problem is that pron is turning into a troll and it's because he has a history of working on the JVM and he sees everything from the lens of writing language runtimes that ignore the Rust memory model altogether and the troll part is that he is not saying out loud what his niche is.

When he brought up the universality claim, he came up with a niche counter example that he kept inside his head and he makes it out to be the general rule by being extremely vague about literally everything.

Re: Zig's Incremental Compilation Internals

#187
post #32

Earlier quoted context omitted.

So that could be a clear definition, but for it to be "table stakes" it needs to have some universal value and it doesn't (in fact, that very same definition could also classify even C as "memory safe"): https://news.ycombinator.com/item?id=49087458

I'm not exactly sure what you mean by "universal value", but I would say to be "table stakes" (i.e. not optional), it has to have overwhelming value. I think outside some fairly niche areas (e.g. programs that don't process untrusted data at all), it very very clearly has overwhelming value. Now you might argue that the other features of Zig, like `defer`, are so good that they reduce the chance of memory errors and…

His point is literally that writing a JVM (or any other language runtime) in Rust is unpleasant.

He can't write a JVM without using unsafe. So his disagreement on the "table stakes" is that his "table stakes" require using unsafe Rust everywhere and from that perspective Rust is not that different from any other language.

Hence the complaint about the lack of universality, which I personally consider weird. His point is that you cannot write the most interesting low level programs using just the safe subset of Rust.

If you have to write unsafe Rust (emphasis on have, your mileage may wary lot on that), then you have to litter unsafe everywhere in your code base so how does Rust help him? That's his point, but he doesn't want to say it out loud.

Re: Zig's Incremental Compilation Internals

#188

Earlier quoted context omitted.

The traditional model of compilation is the biggest issue holding back fast incremental compilation IMHO. Swift suffers from this as well. Even a simple change to one file results in re-parsing the whole library because definitions come from anywhere and we have to obey the 1970s single file compilation model. The result is a driver spawns 8 threads and each one wastes time re-parsing every file in the library lookin…

> To me compilers should be content-addressed databases Rustc already does something like this. The issues are: - when you have to rehash everything to check that they indeed didn't change from the previous compilation. For big projects this takes _a lot_ of time. - when small changes do indeed change the hash of a lot of seemingly unrelated code, which is more common than you might think.

You dont need to rehash everything if you do multi level hashing (file -> mod/impl block -> function). The expensive part is doing lots of hash passes over small parts of a file, so if you break as early as possible the moment you can guarantee that part is unchanged you save a lot of time. And if you assume (and document) that libraries need to be rebuilt manually you can completely skip checking them.

OP handles the small change problem by hashing IR instead of source text. If the new function compiles to the same IR as the old one its guaranteed to give the same machine code. You should repeat this after each lowering or optimization pass so functions with different HIR but same MIR are also marked as unchanged and dont cause items up the tree to rebuild.

Re: Zig's Incremental Compilation Internals

#189

Earlier quoted context omitted.

> GC languages are slower This is not necessarily true. It depends on a language, e.g. Go is slow, Nim[0] is extremely fast with conventional GC and slightly faster with ARC/ORC[1]. GC programs can be faster than manually managed ones in some cases. It's just manual memory management gives you more control of where and when free is called. And a good type system is a privelege that gives Nim more control with destruc…

Good to know! Would there be any way languages like Go or C# could adopt Nim's new garbage collector? If it's better, what stops other languages from using it? > GC programs can be faster than manually managed ones in some cases. I've seen poorly written programs in C/C++/Rust which are slow because they allocate millions of tiny objects. Its true that generational GCs can be faster in this case. But you usually get…

Even if you don't think about memory management, and do the naive thing in rust or rc in some other systems language, gc only comes out ahead in that many tiny objects case. Which is very domain specific. I don't ever run into that situation, or if I do, they are homogeneous in type so I handle them in bulk, not individually.

Re: Zig's Incremental Compilation Internals

#190
post #31

Earlier quoted context omitted.

That framing may seem intellectually satisfying, but it's not useful in practice. Consider the extreme edge case of C: We can clearly mechanically delineate between the empty program and a non-empty one, we call the empty program safe and any program that isn't empty unsafe (i.e. C is memory-safe if you want to do nothing and not if you want to do anything). And so, we also have this property that in C you can't do a…

Do you find most of your rust code is unsafe? Because I also find I do some unsafe stuff, because of the algorithms I work on I often end up with some unchecked array accesses and a couple of raw pointers into those arrays I pass around. But 98% of the code is safe and I find this makes it easier to reason about.

Yes, it's pron. Pron works on a JVM so he looks at everything from a language runtime lens. To him, the entire code base might as well be unsafe.

The argument that you can contain unsafe in safe abstractions does not interest him, because of the nature of the projects he works on.

Post reply on HN