Live data from Hacker News

How Our Rust-to-Zig Rewrite Is Going

rtfeldman.com

221–230 of 336 posts

Re: How Our Rust-to-Zig Rewrite Is Going

#221
post #184

Earlier quoted context omitted.

You can because all allocations are tracked and explicit

That's not sufficient - consider the following pseudocode x = malloc(); if (opaque_cond()) free(x); if (other_opaque_cond()) use(x); Conditions can be opaque and non-analyzable due to rices theorem - in any turing complete language. This code is correct (or at least not memory unsound) if opaque_cond and other_opaque_cond are never both true. Otherwise it isn't. And functionally compiler analyses of whether condition…

so yes it is possible to detect those patterns and ban them as unsafe, and have a"safety checked alternative. clr does this currently:

https://github.com/ityonemo/clr#safety-oriented-architecture

Re: How Our Rust-to-Zig Rewrite Is Going

#222
post #76

Earlier quoted context omitted.

Compilers are not security sensitive, usually. And while UB could theoretically poison the generated code, this isn't a bigger risk than logic bugs.

Of course they are, anything can be a gateway to inject backdoors, if security is not taken into account. And as mentioned, if what Zig offers is already in Purify, there is hardly any added value over C and C++, without the headaches of a niche language.

Considering that you often run the code after you compile it, it might not matter. Anyway, like it or not, most compilers don't consider themselves security sensitive and will not consider malicious code that is able to hijack the compiler a security vulnerability.

Re: How Our Rust-to-Zig Rewrite Is Going

#223

Earlier quoted context omitted.

Because they view "unsafe" as an escape hatch instead of a feature. It's a way to encapsulate dangerous behavior, tightly, with clear postcondiitions. Sometimes it's the only way to do things like interact with inherently unsafe FFI code, or hardware.

I adore unsafe, appreciate it as a feature... but it is an escape hatch. One that is sometimes necessary, one that is sometimes not necessary but might still be (ab)used for performance, or initial 1:1 porting of C/C++ code. There are a lot of cases where that escape hatch should probably welded shut though. Fortunately, the Rust ecosystem has tools like `cargo geiger`, and straight out of the box I can also write: /…

I feel like this perpetuates a bad mental model. Unsafe is not an escape hatch. Code within unsafe blocks must uphold the same semantics as code outside it but the compiler cannot guarantee that those semantics are upheld.

If we're using analogies, `unsafe` is like a "hard hat required" sign. There's nothing intrinsically different about the space inside or outside of it, other than that you can't be sure a brick isn't going to fall on your head once you cross over. So it's on you to wear a hard hat. And to not drop any bricks and trust other people to do their best not to drop any bricks.

You wouldn't call that an escape hatch.

Re: How Our Rust-to-Zig Rewrite Is Going

#224

Earlier quoted context omitted.

empirically untrue. several projects exist that bolt on extra safety to unsafe languages or unsafe parts of language. SeL4 for C, MIRI for rust unsafe. i guess ada/spark for ada too, is the OG, spark being added to ada 4 years after its first release

Hardening is definitely possible, we've had sanitizers in C/C++ for a long time. It's not full memory safety though. Miri is the same. SeL4C is formal verification, and while it can prove memory safety (and much more) it is much more difficult, to the point that you're basically programming in a different language. Ada/SPARK is your best example, and also the example I know the least of, so I won't comment on.

SPARK omits some features of Ada, so it would only reinforce the sentiment that bolting on verifiability after-the-fact is difficult. Expressivity is generally the antithesis of static analysis, and it's very easy and tempting to make a language that is accidentally too expressive to support a given analysis without being required to make breaking changes to reduce expressivity.

Re: How Our Rust-to-Zig Rewrite Is Going

#226
post #167

Earlier quoted context omitted.

I don’t think there’s too many of us on the ‘GC did nothing wrong’ hill. Reading the average HN opinion, it seems everybody is writing high-performance latency-sensitive systems that would implode if a response would take 1 ms longer than normal.

Sampling bias. Most of the people responding are probably those with a strong opinion because of what they work on. Everyone else is likely relatively indifferent to it. It is a misconception that GCs only affect latency-sensitive systems. High-performance throughput-optimized systems are also sensitive at ~1µs granularity for different reasons, so GCs are not used there either. That a GC is adverse to the performanc…

> Maybe systems that are severely I/O bound but is barely a thing these days.

Any kind of web service is barely a thing today? Which is what 99% of HN posters are working on, hence my comment.

> High-performance throughput-optimized systems are also sensitive at ~1µs granularity for different reasons, so GCs are not used there either

Games are high-performance throughput-optimized systems that have adopted GC languages for 15+ years now, and again a type of application which is much more latency sensitive than most people deal in their day to day.

Nobody is claiming GC is a panacea, but it’s good enough for a lot more use cases people give it credit for.

Re: How Our Rust-to-Zig Rewrite Is Going

#227
post #175

Earlier quoted context omitted.

> I invite you to read the release notes and see for yourself the types of breaking changes we’re talking about. I did, and I immediately found this in the latest release: https://ziglang.org/download/0.16.0/release-notes.html#IO-as... That seems like it would require changing a lot of code. Calling it "production ready" is dishonest at best

Not really. It’s find and replace work. If by production-ready you mean you can forever avoid changing code you wrote 8 months ago, sure, pick something else. To me production-ready means it can be trusted to power production workloads, has all tooling I need, and has a consistent long-term vision. Zig ticks all the boxes.

Changing core features every few months doesn't seem very consistent...

Re: How Our Rust-to-Zig Rewrite Is Going

#228
post #204

Earlier quoted context omitted.

Zig does offer some amount of temporal memory safety. Link: https://zig.guide/standard-library/allocators/ Text: > The Zig standard library also has a general-purpose debug allocator. This is a safe allocator that can prevent double-free, use-after-free and can detect leaks. For more detail, see: https://github.com/ziglang/zig/issues/3180#issuecomment-5284...

I still don't think that does anything regarding use-after-frees, only double-frees. Here's the code: https://codeberg.org/ziglang/zig/src/commit/e44e927d33d37c44... The closest callout in the doc comment is: >Never reuses memory addresses, making it easier for Zig to detect branch on undefined values in case of dangling pointers. This relies on the backing allocator to also not reuse addresses. But it's not really c…

I'm not sure what "branch on undefined values" means there, yeah, but never reusing memory addresses is enough to prevent use-after-free.

Or, rather, you can use a value after freeing it, but it will not be exploitable, because it will contain valid data of the right type. This is the same idea as Type-After-Type,

https://dl.acm.org/doi/10.1145/3274694.3274705

(Also similar to when you use indexes to an array in Rust and happen to read from a wrong but in-bounds index.)

Re: How Our Rust-to-Zig Rewrite Is Going

#229

Earlier quoted context omitted.

Sampling bias. Most of the people responding are probably those with a strong opinion because of what they work on. Everyone else is likely relatively indifferent to it. It is a misconception that GCs only affect latency-sensitive systems. High-performance throughput-optimized systems are also sensitive at ~1µs granularity for different reasons, so GCs are not used there either. That a GC is adverse to the performanc…

Could you elaborate on "GCs are not used there [high-performance throughput-optimized systems]"? Are you referring to the cascading effects of tail latency on systems with high fanout?

Sophisticated throughput-optimized systems rely on deep latency-hiding. Schedulers see millions of atomic operations into the future, continuously rewriting the schedule globally to maximize locality and minimize resource contention based on real-time changes to workload, resource availability, and system behaviors.

In short, for each of the millions of in-flight operations (which might only map to a handful of user operations), it is trying to precisely optimize the concurrency, timing, and dependency sequencing such that when operations are executed every resource required is hot, uncontended, and available with high probability. When this works well it dramatically reduces the number of hidden stalls in execution. The schedule is constrained by tail latency requirements; a theoretically throughput-optimal schedule can defer execution indefinitely.

For an analytical database engine, an "atomic operation" is typically a query operation on a database page. A modern server can retire 100M ops/sec. While I am oversimplifying a bit, a 1 millisecond GC pause can blindly wreck the schedule for 100,000 operations in an unpredictable way. In these architectures we try to eliminate all context switches for the same reason which are 100x cheaper.

Practically, 1µs stall is a good heuristic for a noise floor. The schedulers have pretty wide concurrency on big systems, so the implied 100 operations are unlikely to have a dependency. Many stalls that are difficult to precisely control like cache line fills fit in here too.

If there was a GC that had a worst-case stall of 1µs then you could probably use it for these cases. Unfortunately, "low-latency" GCs tend to be more like 1000x that. I don't think there is any way of closing that gap short of putting a GC in hardware.

Re: How Our Rust-to-Zig Rewrite Is Going

#230
post #106

This piece would have been a lot more compelling if they had actually done science on selecting a language for compiler development. From what I can tell, they had an untested hypothesis that a low level systems language is necessary for a high performance compiler https://www.roc-lang.org/faq#self-hosted-compiler and from that concluded that their only choice besides rust was zig. I know from experience that this in…

> Compiler performance is dominated by algorithms

But you can always use the best algorithm no matter what your implementation language is, so it still makes sense to prefer a language that makes it easy to write fast code.

Post reply on HN