Live data from Hacker News

Zig and Rust

matklad.github.io

121–130 of 247 posts

Re: Zig and Rust

#121

Earlier quoted context omitted.

RAII is not a silver bullet. RAII wants to have neat recursive ownership and cleanup of everything, which is undoubtedly a nice model, but sometimes in low-level code you have to choose between nice recursive ownership and performance. For example, Protocol Buffers are trees of objects. Originally they were simple and had recursively-owned, heap-allocated nodes. But it turns out that traversing and deleting a big gra…

Which is why low level languages like C++ and Rust make RAII the default in their standard library (easy, simple), but it's still completely possible to manage the memory on your own if that's necessary for the task at hand. Having easy defaults with the capacity to do everything by hand if you need to is the most flexible system.

The last time I looked (which has been a while, I admit), that meant that you had a very hard time using any of the standard library types with an arena or whatever form of management you used for your specialized code.

What does that look like now?

Re: Zig and Rust

#122
post #91

Earlier quoted context omitted.

> ...and want to control exactly where when and how memory is allocated... Isn't this the whole point and the reason why manual memory management is still a thing - and actually getting more important with the ever widening CPU/memory latency gap? There simply is no silver bullet for memory management, you can either have high performance or automatic memory management, but not both. (vastly simplified of course - bu…

> There simply is no silver bullet for memory management, you can either have high performance or automatic memory management, but not both. RAII is the way.

[deleted]

Re: Zig and Rust

#123

I tried zig after reading so many HN articles and I am not impressed. You have to remember to free after each allocation. The language is supposed to be simple, yet there exist try, catch, defer, errdefer. Rust enums make defining errors so simple with arbitrary payload. Using data structures in rust is much nicer. I think unless you have some precise needs about memory allocation, and want to control exactly where w…

"(Zig's) collections are not parametrized by an allocator, like in C++ or (future) Rust. Rather, an allocator is passed in explicitly to every method which actually needs to allocate. This is Call Site Dependency Injection, and it is more flexible.' I'd once considered the idea of having "space" as a type. Space is an array of bytes, and it cannot be read or written. Constructors take in "space" and turn it into an o…

I actually did find the allocation control very interesting and useful. It seemed like something that can be so useful in the context of request/response servers. Use an allocator per request and reset it at the end.

However, it alone does not compensate for all the other missing niceties. e.g. Just dealing with strings is not straightforward compared to Rust/C++.

Re: Zig and Rust

#124

I tried zig after reading so many HN articles and I am not impressed. You have to remember to free after each allocation. The language is supposed to be simple, yet there exist try, catch, defer, errdefer. Rust enums make defining errors so simple with arbitrary payload. Using data structures in rust is much nicer. I think unless you have some precise needs about memory allocation, and want to control exactly where w…

"(Zig's) collections are not parametrized by an allocator, like in C++ or (future) Rust. Rather, an allocator is passed in explicitly to every method which actually needs to allocate. This is Call Site Dependency Injection, and it is more flexible.' I'd once considered the idea of having "space" as a type. Space is an array of bytes, and it cannot be read or written. Constructors take in "space" and turn it into an o…

Goddamn, this is a great way to consider this problem! I dabble in a lot of languages, Zig included, and I never had a good way to consider the allocation model that fit in my head in a reasonable way. See, I like to think of software as a transformation process that happens to physical objects that I actually visualize, and so thinking of an allocator as a "space" that can be transformed is super useful to me. I also love very-strictly-typed programming. I enjoy making everything a type, so thinking about a "space" type is something I can use in the future.

Re: Zig and Rust

#125

Still not convinced that memory semantics are critical in the vast majority of domains. Incredibly fast speeds can be achieved with simple GC and RC for short-running programs, and programs with sustained runtimes can rely on generational GC. These methods have the advantage of nearly eliminating the need for memory semantics, leaving only business logic behind. The best example might be Nim, which drastically reduce…

You can get the idea that a GC has minimal impact on performance when you just look at things like how fast the GC runs, or what % of time is spent in GC, but that only represents a portion of performance implications, there are lots of indirect effects: 1. More memory is used, which means more CPU cache evictions happen 2. There will be things the language won't let you do, because it is garbage collected 3. There w…

Nim uses stack allocated value types by default, and GC (or ptr) is an optional tag to the type definition.

GC types use borrow & move analysis like rust (not as good yet tho) so it can elide GC work when possible. GC is also not stop-the-world, deterministic (assuming no cycles), and configurable to soft realtime performance.

So you dont need to use the GC, but it will give you RAII if you want without needing to perform the dance of the borrow checker.

BTW Nim is also closing in on Rust's borrow checking semantics, too!

Re: Zig and Rust

#126

Still not convinced that memory semantics are critical in the vast majority of domains. Incredibly fast speeds can be achieved with simple GC and RC for short-running programs, and programs with sustained runtimes can rely on generational GC. These methods have the advantage of nearly eliminating the need for memory semantics, leaving only business logic behind. The best example might be Nim, which drastically reduce…

You can get the idea that a GC has minimal impact on performance when you just look at things like how fast the GC runs, or what % of time is spent in GC, but that only represents a portion of performance implications, there are lots of indirect effects: 1. More memory is used, which means more CPU cache evictions happen 2. There will be things the language won't let you do, because it is garbage collected 3. There w…

Also, aside from the fact GC is optional in Nim, what are you thinking of that cant be done with a GC?

Re: Zig and Rust

#127
post #120

Earlier quoted context omitted.

JIT compilers, games, database server. Anything where you need high performance and do a lot of allocations.

JIT compilers don’t actually need that low-level programming, and compilers in general use memory in a very haphazard way so they are likely better off with a GC. In fact, you probably loose way more by not being able to write as many/great optimizations in a low-level language than what you lose on a slight overhead — e.g. Java’s Graal JIT compiler is nowadays performing better (written in Java itself) than the “ori…

I second that; there is nothing inherent about a JIT that requires low-level memory tricks, except that last step of making the machine code executable and callable. I'm not sure that Rust is a good fit for optimizing compilers, either. E.g. most compiler IRs are fundamentally graph-based and use unrestricted cycles. TurboFan in V8 uses arena-style allocation, as do most of the JITs in other JSVMs, mostly because managing every single node is overkill. A GC with a big enough young gen will do well for the kinds of allocation patterns of a JIT.

Re: Zig and Rust

#128

I tried zig after reading so many HN articles and I am not impressed. You have to remember to free after each allocation. The language is supposed to be simple, yet there exist try, catch, defer, errdefer. Rust enums make defining errors so simple with arbitrary payload. Using data structures in rust is much nicer. I think unless you have some precise needs about memory allocation, and want to control exactly where w…

> I think unless you have some precise needs about memory allocation, and want to control exactly where when and how memory is allocated, it is hard to argue in favour of Zig. I do have a hard time to think of use cases other than embedded microcontroller work where that matters nowadays. That said, I suspect that while they could have written TigerBeetle in Rust (no_std does not assume the existence of an allocator)…

Re: TigerBeetle... caveat with I haven't looked at the TigerBeetle internals&source code, but. Some thoughts...

The kind of page buffer allocation done inside databases is kind of a different nature than the kind where you, e.g. override the default allocator in STL collections, or pass a different allocator into your standard collections library in Zig.

It's generally such that only specific data structures -- usually BTrees or HashTables meant to store relations/tables/indexes -- are managed this way. And so they're usually built from scratch around an explicit specific page buffer mgmt system. In some systems (like Umbra or LeanStore) this might be somewhat murkier in that it might use pointer swizzling etc behind the scenes, but it's still usually the case that the data structures used for relation storage are tied directly to the DB's own page buffer implementation anyways.

Now, other parts of the application stack may benefit from having a custom allocator in the same way as any performance critical system might, but that's of a different nature.

(That said, Rust is also not easily suited to the kind of "pointer-swizzling" behind the scenes bait-and-switch with memory that e.g. LeanStore does with C++. I've tried, and, while it's possible, the language in general gets in the way and you're `unsafe` all over the place anyways.)

Anyways, all this to say, TLDR given that all serious databases do explicit memory / buffer pool mgmt ... and build data structures specific to them, I don't see any intrinsic advantage to Zig for this purpose really? Other than it's a decently modern systems programming language that mostly stays out of your way.

But... I can see major advantages to using Rust: more developers, bigger community, larger ecosystem, safer for memory mgmt elsewhere in the stack, safer for concurrency, etc.

Re: Zig and Rust

#129

Earlier quoted context omitted.

I might be misreading this comment, but the answer to bloated software is not writing all of it in rust/C++/zig. The reasons for bloat is not garage collection.

It is not about the minutiae of language choices no, it's about a pervasive programmer mindset of not wanting to think about resources like memory of clock cycles or bandwidth as things that actually matter.

A lot of it is that APIs, particularly for GC'd languages, often require lots of intermediate garbage and defensive copies. It wasn't until Java 17 that there was a standard way of parsing integers that didn't allocate. I blame libraries more than languages, TBH.

Re: Zig and Rust

#130

I tried zig after reading so many HN articles and I am not impressed. You have to remember to free after each allocation. The language is supposed to be simple, yet there exist try, catch, defer, errdefer. Rust enums make defining errors so simple with arbitrary payload. Using data structures in rust is much nicer. I think unless you have some precise needs about memory allocation, and want to control exactly where w…

Rust without std lib can control all details of allocations and you still get safe sum types, automatic free etc.

The real problem with Rust is hostile syntax for unsafe code that one often needs when doing low-level programming.

Post reply on HN