Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Our Rust-to-Zig Rewrite Is Going

rtfeldman.com

Post reply on HN
Print view View original

101–110 of 336 posts

Re: How Our Rust-to-Zig Rewrite Is Going

#101

Post by AlotOfReading »

inigyou wrote: post #88
AlotOfReading wrote: post #65

Earlier quoted context omitted.

I've noticed that people equate "low level stuff" with unsafe, regardless of whether it's contextually justified.

I'll play devil's advocate. I think emitting machine code intended to run is unsafe because you could emit unsafe machine code, which could run. It's the whole system that is either safe or not, not the individual components. If your system gets hacked by a buffer overflow in the end, nobody cares whether it was the linker that overflowed or the code emitted by the linker.

"Safe" has a very specific definition in Rust. It's not identical to the broader definition used in technical English. You can easily have safe rust code with behaviors any reasonable layperson would call unsafe, like crashing a plane. The original article, comment, and replies were using the word in the Rust sense from my reading, not the English meaning.

Re: How Our Rust-to-Zig Rewrite Is Going

#102

Post by surajrmal »

steveklabnik wrote: post #52
EPWN3D wrote: post #28

Earlier quoted context omitted.

Yeah that is definitely 1000% wrong. A compiler can do its job with totally abstract data structures. If anything would need to do unsafe stuff in memory, it would probably be a linker.

> probably be a linker I don't think that's any different either. The core job of linking isn't particularly unsafe. (Unless, similarly, you're doing the hot reloading stuff)

Perhaps the parent meant dynamic linker.

Re: How Our Rust-to-Zig Rewrite Is Going

#103

Post by steveklabnik »

inigyou wrote: post #88
AlotOfReading wrote: post #65

Earlier quoted context omitted.

I've noticed that people equate "low level stuff" with unsafe, regardless of whether it's contextually justified.

I'll play devil's advocate. I think emitting machine code intended to run is unsafe because you could emit unsafe machine code, which could run. It's the whole system that is either safe or not, not the individual components. If your system gets hacked by a buffer overflow in the end, nobody cares whether it was the linker that overflowed or the code emitted by the linker.

> It's the whole system that is either safe or not, not the individual components.

This is a core perspective disagreement. While this is true:

> If your system gets hacked by a buffer overflow in the end, nobody cares whether it was the linker that overflowed or the code emitted by the linker.

That does not mean that increasing the amount of safety in the individual components isn't helpful, because it helps minimize the above outcome, even if it will never be zero.

Re: How Our Rust-to-Zig Rewrite Is Going

#104

Post by dnautics »

flohofwoe wrote: post #46
veber-alex wrote: post #41

Earlier quoted context omitted.

I believe you are correct. I think ReleaseSafe just adds bound checking and panics on unreachable code. I don't think Zig offers any temporal memory safety.

The DebugAllocator catches use-after-free (at least on page-level), but at the cost of never recycling memory addresses (e.g. it eats through the virtual address space). https://ziglang.org/documentation/master/std/#src/std/heap/d... For higher level code, "generation-counted index handles" might be the better solution to provide temporal runtime memory safety, not part of Zig the stdlib though. Or even better: never…

as a reminder its a construct in the zig stdlib to take an arbitrary chunk of memory (could be stack, could be in the programs static space) and wrap it in an allocator interface and give that to any data structure that needs an allocator and use it as if it were just malloc/free, with a fixed memory limit and the correct memory errors.

Re: How Our Rust-to-Zig Rewrite Is Going

#105

Post by onlyrealcuzzo »

insanitybit wrote: post #61
onlyrealcuzzo wrote: post #3

Zig's incremental builds are DEFINITELY a killer feature. In the short term, I could see why you'd make a switch to get it. But, in the medium term, can we really not expect to see this in Rust in the somewhat near future? I want to go fast, but I don't want to go fast just to shoot my foot off. If only somehow we could get Rust's safety with all of Zig's features and Go's runtime without GC... That's what I'm workin…

Rust's compile times will get faster long before Zig gets safer.

I'm pretty sure Zig has no plans to ever become safe - by any sane sense of the word - so, yes, I would expect...

Re: How Our Rust-to-Zig Rewrite Is Going

#106

Post by norir »

This piece would have been a lot more compelling if they had actually done science on selecting a language for compiler development. From what I can tell, they had an untested hypothesis that a low level systems language is necessary for a high performance compiler https://www.roc-lang.org/faq#self-hosted-compiler and from that concluded that their only choice besides rust was zig.

I know from experience that this initial assumption is wrong. Compiler performance is dominated by algorithms. The fastes managed languages tend to be at worst within a factor of two for wall time on any given algorithm. Algorithmic differences can be unbounded in their performance gaps. Zig itself is a perfect counterexample to the theory that writing a compiler in a low level systems language will lead to a fast compiler. Roc seems to compile at around 15k lines per second. That is not fast. There were evidently compilers written in ml that did 3k likes per second in 1998 https://flint.cs.yale.edu/cs421/case-for-ml.html

The zig rewrite of roc looks like the author's second compiler. Compiler and language design is a skill like any other and from my vantage point, they appear to have overcommitted to an initial design at the expense of developing their higher level design skills. In my opinion, the best thing they could do for the future of roc is stop working on their current compiler and use it to write a self hosting compiler for a much smaller subset of roc. They should be able to do that in less than 10k lines of code. They might even find that their self hosting compiler is faster than their zig based bootstrap compiler for the self hosted subset of roc. If the self hosting compiler is inadequate. Now they at least have identified a smaller useful subset of roc and can experiment with different compiler implementations in 10k likes of code rather than 300k lines of code. Then they could actually test the theory of whether or not a low level language is necessary to meet whatever arbitrary compiler performance goals they have.

By self hosting, they would also discover what roc features actually matter and they would spend much more time actually writing roc code. The features that are needed to write a self hosted compiler are all features that are generally useful. By improving the self hosted compiler, they also improve downstream programs.

Re: How Our Rust-to-Zig Rewrite Is Going

#107

Post by kstrauser »

epage wrote: post #80
giancarlostoro wrote: post #31

One thing I wish Rust would improve over time is the builds. Its one of the biggest sources of wasted storage space on all my computers, builds a ton of libraries can take tens of gigs, it adds up very quickly. Not sure what the best solution is, one I found is to set the global build folder so dependencies get reused across projects, but imho it should be an OOTB default behavior whatever the real solution should be…

We are trading away disk space for faster builds. We could make them faster in some cases by using even more... On the other hand, it would be good to garbage collect those caches. We are wrapping up work on a new layout for intermediate build artifacts that will make it easier to GC them.

Can you talk a little more about that? How would that work, and what would the developer experience be like when using it?

Re: How Our Rust-to-Zig Rewrite Is Going

#108

Post by dnautics »

veber-alex wrote: post #82
afdbcreid wrote: post #63

Earlier quoted context omitted.

That is possible (clang has experimental lifetime annotations support), but that is not enough to guarantee memory safety. As a simple example, Zig has no private fields. That makes encapsulating any unsafety impossible.

Exactly. Every part of the language must support memory safety from first principles.

empirically untrue. several projects exist that bolt on extra safety to unsafe languages or unsafe parts of language. SeL4 for C, MIRI for rust unsafe. i guess ada/spark for ada too, is the OG, spark being added to ada 4 years after its first release

Re: How Our Rust-to-Zig Rewrite Is Going

#109

Post by drdexebtjl »

Aurornis wrote: post #90
drdexebtjl wrote: post #44

Earlier quoted context omitted.

Zig is not pre-1.0 because it’s not ready for production (bugs or missing features), it’s pre-1.0 because they want to be able to make breaking language changes. Nowadays when you can just point an agent at release notes and have it update everything, I actually prefer not having to wait through rare major releases to get new language features.

> Zig is not pre-1.0 because it’s not ready for production (bugs or missing features), it’s pre-1.0 because they want to be able to make breaking language changes. This is a solved problem in other projects. Either use the version numbers as intended and bump the major version number on breaking changes, or use Rust-style editions to opt in to the newer versions of the changes. Calling a project production-ready but…

[dead]

Re: How Our Rust-to-Zig Rewrite Is Going

#110

Post by Liquid_Fire »

rtfeldman wrote: post #74
steveklabnik wrote: post #55

Earlier quoted context omitted.

I am disappointed you're downvoted, Richard. This is a fine reply, and I hope you know that a minor quibble with a single line in the post doesn't mean that I think it's a bad one overall. (EDIT: a few minutes later, the parent comment is no longer grey.) I also think it's a good thing that you wrote the post in general, when I saw it pop up I was like "oh, of course, this post should exist!" I'm surprised I didn't t…

Also a good point! TIL that Rust and C++ use interpreters for const, although of course that wouldn't work for running tests. Then again, in the specific case of Rust I believe rustc only compiles the tests and then something else like Cargo executes them. Of course, as I noted elsewhere, if rustc emits machine code and then cargo immediately executes it, there's the same opportunity for end user memory being corrupt…

> if rustc emits machine code and then cargo immediately executes it, there's the same opportunity for end user memory being corrupted (due to miscompilation) as if rustc and cargo shared a code base

Your tests run in an entirely separate process from the compiler (and from cargo). This makes it very different from memory corruption in the compiler:

- The test process can only corrupt its own memory.

- You don't need "unsafe" to run tests. Just the ability to start another process.

- If you're cross-compiling, you wouldn't even be able to run the tests on the same machine (without emulation/compatibility layers)

Does roc run tests in the same process as the compiler?

Post reply on HN