Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zig and Rust

matklad.github.io

Post reply on HN
Print view View original

171–180 of 247 posts

Re: Zig and Rust

#171

Post by kaba0 »

connicpu wrote: post #161
kaba0 wrote: post #117

Earlier quoted context omitted.

This is by definition manual memory management.

I would argue RAII is still a type of automatic memory management, but it has the determinism and ability to reason about what's going on the same way fully manual management does.

From the Garbage Collector handbook I got a very great quote about what automatic memory management really is. I will butcher it up, but the basic essence is that automatic memory management is above all a software design tool — well designed programs are built from components that are highly cohesive but loosely coupled. What it means for memory management is that a component shouldn’t have to care about how other modules handle memory for correct working. This is possible with a GC, but isn’t with manual memory management, and RAII doesn’t change it — function signatures leak memory management concerns in both Rust and C++.

Ongoing refactoring and maintenance costs are also higher in low-level languages due to this reason.

Re: Zig and Rust

#172

Post by kaba0 »

Yoric wrote: post #169
kaba0 wrote: post #151

Earlier quoted context omitted.

Well, “never took hold” is relatively. It is and will likely always be a niche language, but one with a quite big audience and ecosystem. There are people happily using different languages that are much smaller, they just happen to occupy a less academic niche — e.g. I’m guessing but you probably don’t have such feelings against D, when it might very well be smaller than Haskell.

To clarify, I have actually coded a bit in Haskell and I have actually released open-source code in SML/NJ, a commercial application in OCaml and worked (as a very minor contributor) on industrial code in Coq. But if I have to start a new industrial project, I will most likely use Rust, because it is easier to hire and because there is lots of momentum in the Rust ecosystem, which means that there are crates for just…

> which means that there are crates for just about everything

There is definitely momentum, but it doesn’t make Rust’s ecosystem big relative to the top 5 in any way.

Re: Zig and Rust

#173

Post by tialaramex »

Georgelemental wrote: post #10
MrBuddyCasino wrote: post #8

> Collections are not parametrized by an allocator, like in C++ or (future) Rust. What does he mean by this?

My understanding is, instead of the allocator being a generic parameter of the type, it's a value you pass to the constructor.

The crucial trick is that it's not just the constructor, it's everywhere which might allocate.

Rust's proposed allocator API (what the article calls future Rust) takes an allocator for constructors, but the effect is the type parameter is just inferred during construction, the same way if you say OK, make me a Vec of this array of Strings, Rust infers the Vec's type Vec.

Zig's standard library provides both conventional compound structures like those I described for Rust, and "Unmanaged" variants in which you must provide an allocator every time you do anything which might allocate, so e.g. addOne on ArrayListUnmanaged requires the allocator, which it will only actually use if adding a single element to the ArrayList exceeded its current capacity. It will assume this is the correct allocator to de-allocate the old backing storage and allocate new storage, so you can't use this design to move from one allocator to another.

Interestingly Zig's ensureTotalCapacity not only avoids the problem of C++ reserve where it destroys the amortized growth behaviour, but it actually insists on exponential growth even if that considerably outstrips the growth requested.

Say we've got 15 Foozles in an ArrayList with capacity 16 (or Rust's Vec or C++ std::vector). We know we want to put 20 more foozles in, for a total of 35, although maybe more.

Rust's Vec says OK, reserve(20), we were thinking of next growing from 16 to 32, but 35 won't fit in 32, so 35 it is. Capacity becomes 35.

Popular C++ std::vector implementations likewise will pick 35 here. But Zig's ArrayList says 16 + 8 + 8 = 32 not big enough, try 32 + 16 + 8 = 56. Capacity becomes 56!

Re: Zig and Rust

#174

Post by Animats »

spacechild1 wrote: post #133
Animats wrote: post #114

Earlier quoted context omitted.

"(Zig's) collections are not parametrized by an allocator, like in C++ or (future) Rust. Rather, an allocator is passed in explicitly to every method which actually needs to allocate. This is Call Site Dependency Injection, and it is more flexible.' I'd once considered the idea of having "space" as a type. Space is an array of bytes, and it cannot be read or written. Constructors take in "space" and turn it into an o…

> This separates construction, which is a typed thing, from allocation. But this is what C++ actually does! C++ ctors run after memory has been allocated (on the stack, on the heap, from a pool, etc.) Or maybe you mean something different?

The lifetime of the space can be different, but must enclose, the objects created in that space. C++ doesn't track lifetimes.

Such a separation is only useful in specific situations, such as the one the original article mentions. It could be useful in avionics or industrial control, where you often avoid memory pools and have arrays of specific kinds of objects instead.

Re: Zig and Rust

#175

Post by lenkite »

munificent wrote: post #103
flohofwoe wrote: post #62

Earlier quoted context omitted.

> ...and want to control exactly where when and how memory is allocated... Isn't this the whole point and the reason why manual memory management is still a thing - and actually getting more important with the ever widening CPU/memory latency gap? There simply is no silver bullet for memory management, you can either have high performance or automatic memory management, but not both. (vastly simplified of course - bu…

> you need to spend so much effort to reduce memory management overhead that it is usually less work to do it all manually in the first place That has not been the case in the vast majority of applications I've written across a number of domains using garbage collected languages. Yes, there is runtime overhead to GC. But for many many programs, the cost of the overhead is acceptable and you can spend all of your time…

"and you can spend all of your time worrying about the application domain and not worrying about allocation."

This has never been my experience in 10+ years of middleware development in GC languages in large teams. The problem is just deferred to production where you need to analyze heap dumps and run a profiler to hunt down excessive allocation issues and why the GC is running all the time.

Sometimes, troubleshooting and correcting many of these issues take multiple more man hours than coding the service in the first place! The usual solution of "throw more hardware at it" only works until mgmt starts complaining about costs and uptime.

I have spent more time debugging memory issues in Java/C# projects than in my first large scale project in C++ where the lead architect strictly mandated the use of a custom allocator for everything.

Re: Zig and Rust

#176

Post by kaba0 »

59nadir wrote: post #139
munificent wrote: post #103

Earlier quoted context omitted.

> you need to spend so much effort to reduce memory management overhead that it is usually less work to do it all manually in the first place That has not been the case in the vast majority of applications I've written across a number of domains using garbage collected languages. Yes, there is runtime overhead to GC. But for many many programs, the cost of the overhead is acceptable and you can spend all of your time…

Not allowing you to talk about allocation in any great detail certainly on the surface seems to simplify things, but as with most things of that nature it tends to complicate things in edge cases (or just entire parts of industry). What the GP is alluding to, it seems to me, is that in those cases the effort to remedy the problem tends to become a constant hunt and guessing game of "Where are we accidentally allocati…

> Where are we accidentally allocating and ruining everything

This is trivial to profile under any platform that has remotely sane tooling, and usually it is trivial to fix as well.

Object pooling is a very last resort thing to do, I really can’t think of any project I have worked on where there wasn’t another solution to performance problems.

Re: Zig and Rust

#177

Post by munificent »

lenkite wrote: post #175
munificent wrote: post #103

Earlier quoted context omitted.

> you need to spend so much effort to reduce memory management overhead that it is usually less work to do it all manually in the first place That has not been the case in the vast majority of applications I've written across a number of domains using garbage collected languages. Yes, there is runtime overhead to GC. But for many many programs, the cost of the overhead is acceptable and you can spend all of your time…

"and you can spend all of your time worrying about the application domain and not worrying about allocation." This has never been my experience in 10+ years of middleware development in GC languages in large teams. The problem is just deferred to production where you need to analyze heap dumps and run a profiler to hunt down excessive allocation issues and why the GC is running all the time. Sometimes, troubleshootin…

I'm not sure why you trimmed the beginning of my sentence.

The parent comment says: "(vastly simplified of course - but in languages with automatic memory management - no matter if garbage collection or refcounting - you need to spend so much effort to reduce memory management overhead that it is usually less work to do it all manually in the first place)"

That suggests they are claiming that all users of GC languages will eventually spend more effort dealing with memory than if they'd skipped a GC in the first place.

I am making an existence proof counterargument: I have worked on many many programs in GC languages where at no point in the program's lifecycle did I need to spend much time worrying about memory.

I believe the claim "all programmers will spend more time dealing with memory in GC languages" is false.

I have absolutely not made any claim that "no programmer will spend more time dealing with memory in GC languages". There are certainly programs that are better written with manual memory management. I've written plenty.

My point was only that there are also plenty of programs where that's not true.

Re: Zig and Rust

#178

Post by kaba0 »

titzer wrote: post #129
AnIdiotOnTheNet wrote: post #94

Earlier quoted context omitted.

It is not about the minutiae of language choices no, it's about a pervasive programmer mindset of not wanting to think about resources like memory of clock cycles or bandwidth as things that actually matter.

A lot of it is that APIs, particularly for GC'd languages, often require lots of intermediate garbage and defensive copies. It wasn't until Java 17 that there was a standard way of parsing integers that didn't allocate. I blame libraries more than languages, TBH.

Sure, I am just confirming that mindset, but can you really reason about the performance impact of that allocation? Even if it does end up allocating due to escape analysis not being sufficient, it will do a thread local pointer bump allocation on a hot, in-cache arena basically, and will just be zero-cost cleared once the still achievable objects are moved.

My point is, knowing when to think/not think about allocating and its relevant costs is the proper way to program in a high level language. Only care about it when you are at a part that runs in a hot loop or very often, etc. Pretty much as per the second part of the often partially quoted “premature optimization…”.

Re: Zig and Rust

#179

Post by rrishi »

Animats wrote: post #114
crop_rotation wrote: post #19

I tried zig after reading so many HN articles and I am not impressed. You have to remember to free after each allocation. The language is supposed to be simple, yet there exist try, catch, defer, errdefer. Rust enums make defining errors so simple with arbitrary payload. Using data structures in rust is much nicer. I think unless you have some precise needs about memory allocation, and want to control exactly where w…

"(Zig's) collections are not parametrized by an allocator, like in C++ or (future) Rust. Rather, an allocator is passed in explicitly to every method which actually needs to allocate. This is Call Site Dependency Injection, and it is more flexible.' I'd once considered the idea of having "space" as a type. Space is an array of bytes, and it cannot be read or written. Constructors take in "space" and turn it into an o…

Super interesting idea.

Re: Zig and Rust

#180

Post by kaba0 »

csnover wrote: post #87

> Zig forces you to pass the allocator in, so you might as well think about the most appropriate one! I really feel like this is an underappreciated aspect of some of the more complained-about constraints that these newer languages place on programmers. Like an i32, the default allocator will do a reasonable thing most of the time, but having to think about each allocation (is this short-lived or long-lived? is it ac…

Ad absurdum that would make assembly a better choice — we can just as well reason about better register allocation strategies.

I think the fundamental abstraction level is very important to get right. Of course it depends on the problem domain, and it might just make sense for Zig to expose explicitly allocators everywhere, but maybe some implicitly passed allocator when not otherwise specified could be a better trade off (correct me if it is a thing)

Post reply on HN