Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Our Rust-to-Zig Rewrite Is Going

rtfeldman.com

Post reply on HN
Print view View original

51–60 of 336 posts

Re: How Our Rust-to-Zig Rewrite Is Going

#51

Post by djha-skin »

Hinrik wrote: post #5
onlyrealcuzzo wrote: post #3

Zig's incremental builds are DEFINITELY a killer feature. In the short term, I could see why you'd make a switch to get it. But, in the medium term, can we really not expect to see this in Rust in the somewhat near future? I want to go fast, but I don't want to go fast just to shoot my foot off. If only somehow we could get Rust's safety with all of Zig's features and Go's runtime without GC... That's what I'm workin…

Layperson here: what is special about Go's runtime, aside from the GC?

Chief design goals were radically easy concurrency and speed of compilation.

Re: How Our Rust-to-Zig Rewrite Is Going

#52

Post by steveklabnik »

EPWN3D wrote: post #28
steveklabnik wrote: post #11

I think this is a fine post. But one comment: > remember that for compilers which emit machine code, like roc and rustc, doing memory-unsafe things is a big part of the job I don't really think that this is true, in the way that it's written. I think that for the hot binary patching / code reloading features, yes, that is going to need unsafe. But for regular old "producing an executable" compilation? Emitting machin…

Yeah that is definitely 1000% wrong. A compiler can do its job with totally abstract data structures. If anything would need to do unsafe stuff in memory, it would probably be a linker.

> probably be a linker

I don't think that's any different either. The core job of linking isn't particularly unsafe.

(Unless, similarly, you're doing the hot reloading stuff)

Re: How Our Rust-to-Zig Rewrite Is Going

#53

Post by narnarpapadaddy »

steveklabnik wrote: post #11

I think this is a fine post. But one comment: > remember that for compilers which emit machine code, like roc and rustc, doing memory-unsafe things is a big part of the job I don't really think that this is true, in the way that it's written. I think that for the hot binary patching / code reloading features, yes, that is going to need unsafe. But for regular old "producing an executable" compilation? Emitting machin…

I agree that it’s not inherent to emitting machine code but I do think it reflects a different set of priorities.

In extremely high performance code you use different data structures and algorithms and change your approach to memory allocation. TigerBeetle famously does all memory allocation once on startup.

Roc is attempting to make a similar set of trade-offs in their compiler as Zig, so it makes sense that the author finds many shared patterns.

Re: How Our Rust-to-Zig Rewrite Is Going

#55

Post by steveklabnik »

rtfeldman wrote: post #38
steveklabnik wrote: post #11

I think this is a fine post. But one comment: > remember that for compilers which emit machine code, like roc and rustc, doing memory-unsafe things is a big part of the job I don't really think that this is true, in the way that it's written. I think that for the hot binary patching / code reloading features, yes, that is going to need unsafe. But for regular old "producing an executable" compilation? Emitting machin…

> I think that for the hot binary patching / code reloading features, yes, that is going to need unsafe. But for regular old "producing an executable" compilation? Emitting machine code isn't the part that requires unsafe. The language's runtime is a more likely site to find unsafe. Agreed! Emitting machine code is not unsafe, since it's just writing bytes down - it's only once you execute that machine code that ther…

I am disappointed you're downvoted, Richard. This is a fine reply, and I hope you know that a minor quibble with a single line in the post doesn't mean that I think it's a bad one overall. (EDIT: a few minutes later, the parent comment is no longer grey.)

I also think it's a good thing that you wrote the post in general, when I saw it pop up I was like "oh, of course, this post should exist!" I'm surprised I didn't think about it earlier.

> evaluating userspace code at compile time

Usually this would be done via an interpreter, so I'm not sure that it really requires unsafe either. If you are literally executing machine code, sure, but const fn in Rust and constexpr in C++ and many other languages do not do that, as it causes a number of problems (for example, cross-compilation).

Re: How Our Rust-to-Zig Rewrite Is Going

#56

Post by arthurbrown »

steveklabnik wrote: post #17
arthurbrown wrote: post #14

Interesting that OCaml was flexible and expressive enough to be used as a prototype testbed but not chosen as the implementation language, especially given the maturity of both. I would be surprised if Zigs incremental builds could be meaningfully faster than dune's. Cross compilation is great, but not mentioned in the "why Zig" section. Is memory control that crucial for a compiler? Rust itself was originally writte…

Rust moved away from OCaml when it decided to be re-written in Rust. The post alludes to this as being a usual time for a wholesale re-write, and I'd agree.

I appreciate the insight, and on closer reading the post clearly states that realistically only Zig and Rust were ever considered anyway.

Since you're here, could you comment on the approach Rust took in their rewrite? Was it more of a straight translation like Go did when they self hosted -- similar to the recent Bun transliteration? Or were there architectural changes made along the way like this article describes with Roc?

Re: How Our Rust-to-Zig Rewrite Is Going

#57

Post by Aurornis »

skybrian wrote: post #50
Aurornis wrote: post #27

Earlier quoted context omitted.

That line confused me, too. What parts of their compiler require memory-unsafe operations to produce machine code?

They are saying that running the compiled code is memory-unsafe when there is a compiler bug, and that’s what developers do next. The memory corruption happens in a different process. In this respect, effectively all the compiler should be treated sort of like an unsafe region because it requires extra care to avoid memory corruption bugs.

That's not what it says at all. The section we're talking about is for the compiler and emitting machine code

> we ended up with about 1,200 uses of unsafe

> remember that for compilers which emit machine code, like roc and rustc, doing memory-unsafe things is a big part of the job

Anywhere talking about the `unsafe` keyword is within the Rust code.

Re: How Our Rust-to-Zig Rewrite Is Going

#58

Post by demosthanos »

orlp wrote: post #35
steveklabnik wrote: post #11

I think this is a fine post. But one comment: > remember that for compilers which emit machine code, like roc and rustc, doing memory-unsafe things is a big part of the job I don't really think that this is true, in the way that it's written. I think that for the hot binary patching / code reloading features, yes, that is going to need unsafe. But for regular old "producing an executable" compilation? Emitting machin…

I think if you interpret it charitably it means that any bug in the emitted machine code is already a likely memory-unsafe miscompilation if it is ran. The compiler itself might be perfectly "memory safe" but the generated binary fundamentally is always at risk (besides WebAssembly I suppose). I'm fully aware of the separation of compiler and binary, and being able to compile untrusted code safely is nice, but a perf…

In context that's clearly not what he's saying, the next sentence is this:

> Zig has more features than Rust for making memory-unsafe code work correctly, and that was the area where we wanted the most help.

Zig definitely does not have more features for successfully emitting memory-unsafe machine code than Rust does. I can emit memory-unsafe machine code from typescript if I really want to and nothing at all in the language will get in my way. So the sentence quoted above must refer to the idea that the compiler itself needs to be unsafe, which Steve is right is simply untrue.

Re: How Our Rust-to-Zig Rewrite Is Going

#59

Post by Aurornis »

orlp wrote: post #35
steveklabnik wrote: post #11

I think this is a fine post. But one comment: > remember that for compilers which emit machine code, like roc and rustc, doing memory-unsafe things is a big part of the job I don't really think that this is true, in the way that it's written. I think that for the hot binary patching / code reloading features, yes, that is going to need unsafe. But for regular old "producing an executable" compilation? Emitting machin…

I think if you interpret it charitably it means that any bug in the emitted machine code is already a likely memory-unsafe miscompilation if it is ran. The compiler itself might be perfectly "memory safe" but the generated binary fundamentally is always at risk (besides WebAssembly I suppose). I'm fully aware of the separation of compiler and binary, and being able to compile untrusted code safely is nice, but a perf…

The section this came from was talking specifically about usages of `unsafe` in the compiler code.

It's not about the memory safety of the resulting binary.

Re: How Our Rust-to-Zig Rewrite Is Going

#60

Post by afdbcreid »

pjmlp wrote: post #47

Quite interesting the hand waving of security issues with Zig, oh well. If I want to use allocator debuggers I already have the production ready tools that exist for C and C++ for at least 30 years.

Compilers are not security sensitive, usually. And while UB could theoretically poison the generated code, this isn't a bigger risk than logic bugs.
Post reply on HN