Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Our Rust-to-Zig Rewrite Is Going

rtfeldman.com

Post reply on HN
Print view View original

81–90 of 336 posts

Re: How Our Rust-to-Zig Rewrite Is Going

#81

Post by landr0id »

flohofwoe wrote: post #46
veber-alex wrote: post #41

Earlier quoted context omitted.

I believe you are correct. I think ReleaseSafe just adds bound checking and panics on unreachable code. I don't think Zig offers any temporal memory safety.

The DebugAllocator catches use-after-free (at least on page-level), but at the cost of never recycling memory addresses (e.g. it eats through the virtual address space). https://ziglang.org/documentation/master/std/#src/std/heap/d... For higher level code, "generation-counted index handles" might be the better solution to provide temporal runtime memory safety, not part of Zig the stdlib though. Or even better: never…

>The DebugAllocator catches use-after-free (at least on page-level)

To clarify, is that to say that you have to use the `std.heap.page_allocator` as its backing allocator?

Re: How Our Rust-to-Zig Rewrite Is Going

#82

Post by veber-alex »

afdbcreid wrote: post #63
solatic wrote: post #49

Earlier quoted context omitted.

I wouldn't say it's impossible, rather un-ergonomic. TypeScript can add type information to ordinary JavaScript code via JSDoc comments; the result can both be executed as ordinary JavaScript as-is and type-checked with TypeScript. But it's a huge pain to try to write (and maintain) everything that way, it was supported as a hack to help migrate legacy codebases. You could probably take a similar "the lifetimes are e…

That is possible (clang has experimental lifetime annotations support), but that is not enough to guarantee memory safety. As a simple example, Zig has no private fields. That makes encapsulating any unsafety impossible.

Exactly.

Every part of the language must support memory safety from first principles.

Re: How Our Rust-to-Zig Rewrite Is Going

#84

Post by inigyou »

tredre3 wrote: post #73
c-hendricks wrote: post #36

Earlier quoted context omitted.

I always got a kick out of that, coming from a JavaScript background where people constantly harp on the size of node modules. My Tauri project, where the backend is much smaller code-wise than the frontend, has 9gb of rust artifacts (node_modules is 550mb for comparison)

Rust isn't great, and it shouldn't be a surprised since it's designed after npm. However one metric where nodes_modules is still worse for me is the sheer number of small files in it. Having nearly one million files in nodes_modules isn't that unusual. The problem is that on most common file systems the minimum allocation is usually at least 4KB. So even if the actual data is less than 500MB, you end up with 4GB disk…

I wish ext4 had a feature to mark a file as "atomic" where it would allocate all atomic files in a long run, without room for expansion, and I suppose with very inefficient compaction upon deletion, but without any padding bytes.

Re: How Our Rust-to-Zig Rewrite Is Going

#85

Post by dnautics »

afdbcreid wrote: post #63
solatic wrote: post #49

Earlier quoted context omitted.

I wouldn't say it's impossible, rather un-ergonomic. TypeScript can add type information to ordinary JavaScript code via JSDoc comments; the result can both be executed as ordinary JavaScript as-is and type-checked with TypeScript. But it's a huge pain to try to write (and maintain) everything that way, it was supported as a hack to help migrate legacy codebases. You could probably take a similar "the lifetimes are e…

That is possible (clang has experimental lifetime annotations support), but that is not enough to guarantee memory safety. As a simple example, Zig has no private fields. That makes encapsulating any unsafety impossible.

no. You don't need private fields. All you have to do is analyze the code, harness the compiler to generate a time-dependent data dependency graph, and map allocation/frees/uses, if you can 'color' branches where data are shared you can also track and check to see there isn't an aliasing violation too.

it is easy to patch the zig compiler to enable this this (export the code graph; about 50 LOC). The analysis is much much harder to get right.

Re: How Our Rust-to-Zig Rewrite Is Going

#86

Post by pjmlp »

steveklabnik wrote: post #11

I think this is a fine post. But one comment: > remember that for compilers which emit machine code, like roc and rustc, doing memory-unsafe things is a big part of the job I don't really think that this is true, in the way that it's written. I think that for the hot binary patching / code reloading features, yes, that is going to need unsafe. But for regular old "producing an executable" compilation? Emitting machin…

Many people try to twist the fact memory safe languages have unsafe code blocks to make the pivot that why bother.

It is like someone arguing that since they always bump the head somehow while wearing seatbelts, then they are only a nuisance and should not be used.

Re: How Our Rust-to-Zig Rewrite Is Going

#87

Post by demosthanos »

afdbcreid wrote: post #60
pjmlp wrote: post #47

Quite interesting the hand waving of security issues with Zig, oh well. If I want to use allocator debuggers I already have the production ready tools that exist for C and C++ for at least 30 years.

Compilers are not security sensitive, usually. And while UB could theoretically poison the generated code, this isn't a bigger risk than logic bugs.

> Compilers are not security sensitive, usually.

The compiler is one of the most significant trust boundaries we have. Its decisions can intentionally or unintentionally create vulnerabilities in programs compiled by the compiler, which means that if you can compromise a compiler you can compromise everything downstream.

Unsafe memory access in a compiler can be exploited in order to hijack the compiler itself (this is reported regularly in production compilers), allowing the attacker to then insert arbitrary code into compiled binaries. Not everything that a compiler absorbs from its environment is meant to be treated as source to be compiled, and in a memory unsafe compiler any of that input can silently turn into machine code in the compiled binary if an attacker is able to exploit the memory safety bug and hijack the compiler.

Re: How Our Rust-to-Zig Rewrite Is Going

#88

Post by inigyou »

AlotOfReading wrote: post #65
steveklabnik wrote: post #52

Earlier quoted context omitted.

> probably be a linker I don't think that's any different either. The core job of linking isn't particularly unsafe. (Unless, similarly, you're doing the hot reloading stuff)

I've noticed that people equate "low level stuff" with unsafe, regardless of whether it's contextually justified.

I'll play devil's advocate. I think emitting machine code intended to run is unsafe because you could emit unsafe machine code, which could run. It's the whole system that is either safe or not, not the individual components. If your system gets hacked by a buffer overflow in the end, nobody cares whether it was the linker that overflowed or the code emitted by the linker.

Re: How Our Rust-to-Zig Rewrite Is Going

#89

Post by pjmlp »

grayrest wrote: post #23
arthurbrown wrote: post #14

Interesting that OCaml was flexible and expressive enough to be used as a prototype testbed but not chosen as the implementation language, especially given the maturity of both. I would be surprised if Zigs incremental builds could be meaningfully faster than dune's. Cross compilation is great, but not mentioned in the "why Zig" section. Is memory control that crucial for a compiler? Rust itself was originally writte…

One of the primary goals for the Roc project is compiler speed. I presume OCaml is out of the running because it's not a systems language.

Depends on the beholder.

Unix system programming in OCaml

https://ocaml.github.io/ocamlunix/

https://mirage.io/

Re: How Our Rust-to-Zig Rewrite Is Going

#90

Post by Aurornis »

drdexebtjl wrote: post #44
dev_l1x_be wrote: post #19

Zig is a pre-1.0 language while Rust is post-1.0. This alone is settles which one to pick for may developers. The library support is probably favours Rust too. Rust build times are much slower than Zig, I get that, but I rarely optimize software for build times.

Zig is not pre-1.0 because it’s not ready for production (bugs or missing features), it’s pre-1.0 because they want to be able to make breaking language changes. Nowadays when you can just point an agent at release notes and have it update everything, I actually prefer not having to wait through rare major releases to get new language features.

> Zig is not pre-1.0 because it’s not ready for production (bugs or missing features), it’s pre-1.0 because they want to be able to make breaking language changes.

This is a solved problem in other projects. Either use the version numbers as intended and bump the major version number on breaking changes, or use Rust-style editions to opt in to the newer versions of the changes.

Calling a project production-ready but keeping the version number below 1.0 and saying breaking changes are expected is a tired game. We've seen it backfire across a number of language projects like Elm, where the exact same claim was used to both encourage people to use it and then blame them when it backfired.

If it's production ready, go to 1.0 and then follow semver for breaking changes. I don't care if we get to Zig v73.2.0 as a result. At least we can see from a glance which versions need to be checked for breaking changes.

Post reply on HN