Live data from Hacker News

Zig self hosted compiler is now capable of building itself

github.com

211–220 of 285 posts

Re: Zig self hosted compiler is now capable of building itself

#211
post #207

Earlier quoted context omitted.

> It took me so much effort to feel like I could understand FFTs enough to actually implement them.) Do you understand how little DSP is involved in writing a DAW? It has almost nothing to do with DSP and everything to do with application architecture, data management, threading and more.

I think you may be reading too much into what I said; I was working on a DAW-like toy, not a full DAW that uses VST plugins, and because I chose to write DSP code, I had to understand it. I don’t know what a real DAW looks like because I am not a subject expert. That said, Rust seemed pretty promising for writing the audio engine of a DAW due to the memory ownership model. It was relatively easy to come up with a way…

Most DAWs do not avoid locks in RT context.

Take a listen to my interview with Justin Frankel of Reaper - somehow in our 3hr+ epic chat, he acknowledges that they don't avoid locks entirely. And Ardour, despite trying, also fails to do so. Anecdotally, from my conversations with other DAW developers, they also do no manage it 100%. As Justin put it, it's more about contention avoidance than lock avoidance.

Avoiding actual stack-based allocation is pretty easy in C++. You just have to want to do it, and remember to do it.

Memory correctness is really easy in the RT threads of a DAW precisely because they do (should) not allocate. You're dealing with pre-allocated memory blocks that do not change over time. It's almost the simplest case of memory mgmt that I know of within a DAW. I have seen several new-to-DAW developers adopting ill-advised schemes for memory mgmt in an RT context. The "list of blocks" is one pattern that I consider something to avoid (and unnecessady). Single-reader/single-writer lock-free (circular) FIFOs get you almost all the way there, for everything.

Re: Zig self hosted compiler is now capable of building itself

#212
post #146

Earlier quoted context omitted.

Would you tell about those languages or they are left as an exercise to readers?

Sort of left it to the readers and hopefully encourage people to investigate for themselves. It's far too easy to get into "this benchmark vs this benchmark", etc. Memory usage is overall a complicated topic, which I think Andrew's comment doesn't do justice to. Granted the Zig team has done some impressive work, much of the memory usage & bloat in the C++ world comes from real world programs and libraries and the in…

Zig has a few extra tricks up their sleeves compared to most languages. The ones that come to mind are

* MultiArrayList

* arbitrary size-integers that make it easy to pack data very tightly

* easy access to several special purpose allocators which reduce the need for strategies like reference counting

Re: Zig self hosted compiler is now capable of building itself

#213

Earlier quoted context omitted.

> * a Digital Audio Workstation ORLY? Maybe you can put together a better, faster team (like Presonus managed to do for Studio One), but most current DAWs have been in existence for 20 years or more. Catching up with that is a challenge, and if you don't catch up then it's an interesting toy or half-a-DAW. So why? ps. obviously I am biased.

This argument applies equally to programming languages, and Zig seemed to go OK?

Oh yes, it absolutely applies to all programming languages.

Re: Zig self hosted compiler is now capable of building itself

#214
post #39

Earlier quoted context omitted.

Zig uses manual memory management too (even more manual than Rust), so that's a bit strange question.

It's really easy in Zig to be honest. Just put `defer thing.deinit()` in the right scope and you're done. You gain explicitness and know exactly what's going on in your Code. Everything is obvious. That's the reason Zig is so incredible simple and easy to read. Zig also has a GPA that will tell you about memory leaks or anything.

Rust's ownership rules are no less explicit. The object dies when the owner of the data goes out of scope.

Re: Zig self hosted compiler is now capable of building itself

#215

Earlier quoted context omitted.

By quarantining all memory forever. This is not a scalable solution because keeping one allocation in a 4kB page alive will leak the whole rest of the page. And if you don't quarantine all memory forever then use after free comes back. If it were that easy to solve UAF then C++ would have solved it by now. There is a scalable solution for UAF that doesn't involve introducing a lifetime/region system: garbage collecti…

The problem of a single allocation keeping a 4 KB page alive is something that you might commonly find with the C++ or Rust way of programming that encourages allocating many individual objects on the heap, but in Zig land this is pretty rare. In fact, compare a Zig implementation of a given application with an equivalent in any other language and you will find there is no contest with respect to the size of the memo…

An allocator that can leak (4kB - epsilon) of memory for each allocation is broken. It's not a question of whether the language makes such allocations unusual: free() not actually allowing memory to be freed is a violation of the contract of a memory allocator.

From the man page (which I believe quotes ISO C): "The free() function shall cause the space pointed to by ptr to be deallocated; that is, made available for further allocation." If free doesn't do that because another allocation is pointing into that page, that's a violation of the contract. Standard quarantine doesn't violate the spec because the space will eventually become available no matter what, but perpetual quarantine does.

One way to fix the problem would be to round up all allocations to 4kB, but that's very wasteful and slow due to cache and page table traffic; you'd be better off from a performance point of view with a garbage collector. More promising is ARM MTE, though the tag is currently too small to really be called a solution to UAF as opposed to a mitigation. A 64-bit tag would be enough, but I'm not sure what the performance costs of that would be--I wouldn't be surprised if the increased memory traffic makes it slower than a GC.

Re: Zig self hosted compiler is now capable of building itself

#217
post #207

Earlier quoted context omitted.

I think you may be reading too much into what I said; I was working on a DAW-like toy, not a full DAW that uses VST plugins, and because I chose to write DSP code, I had to understand it. I don’t know what a real DAW looks like because I am not a subject expert. That said, Rust seemed pretty promising for writing the audio engine of a DAW due to the memory ownership model. It was relatively easy to come up with a way…

Most DAWs do not avoid locks in RT context. Take a listen to my interview with Justin Frankel of Reaper - somehow in our 3hr+ epic chat, he acknowledges that they don't avoid locks entirely. And Ardour, despite trying, also fails to do so. Anecdotally, from my conversations with other DAW developers, they also do no manage it 100%. As Justin put it, it's more about contention avoidance than lock avoidance. Avoiding a…

Interesting. I actually thought this was the case, but when I made a similar argument in a chatroom I got pretty severely roasted by people who were sure real audio engines never used locks in the audio thread. It seems like whichever way I go I’m wrong with audio :P but that’s OK, because I really am not pretending to be knowledgable here, for me it was only ever for fun.

The main reason I felt Rust was nice was simply the re-assurance that I could not, even if I wanted to, accidentally cause a data race within the confines of safe Rust. It’s not that the actual code was hard, but it did force me to ensure that what I was doing with the data model was actually safe. It wound up guiding how data flow worked in the playback engine.

Anyway, thanks for the pointer to the interview. I don’t expect myself to be writing the next Reaper or Ardour so it is probably OK if I don’t quite grok what you’re getting at. But, I do find this stuff rather interesting, so I would love to have a listen when I get a chance.

Re: Zig self hosted compiler is now capable of building itself

#218

Earlier quoted context omitted.

What can C and Zig do that Rust can't?

Nothing if unsafe Rust is considered, though Zig does not require you to fight the language nearly as much. This is especially obvious with embedded. Zig's philosophy of no hidden control flow or allocation makes things simple. Simplicity is power. For certain problems you would want a TLA+ specification for safety and especially liveness either way. It's not like Rust absolutely guarantees correctness in all cases.…

Certainly Rust sits where C++ is. Zig and Rust are definitely best used for different things, but I think sweating rust off as in between C and Java is inaccurate.

Re: Zig self hosted compiler is now capable of building itself

#219

Earlier quoted context omitted.

> because automatic deallocations lead to hard-to-predict lifetimes (excess memory usage, and bugs for resource handles that are destructed at hard to predict moments). I don't really feel this is the case in Rust.

But it is. If you have a String on the stack, it's memory is only reclaimed at the end of the scope, while it often could be free'd before. This is especially bad in async code around await point since it means the memory need to be kept alive more than needed.

This doesn't match the experience of GC'd languages. I've never heard of problems in practice that arose from treating all values present on the stack/registers as GC roots, even if those values are dead in the data flow sense.

Re: Zig self hosted compiler is now capable of building itself

#220

I was thinking of learning Rust but it seems a bit overkill due to manual memory management as compared to languages with similar speed like Nim, Zig, and Crystal. How would one compare these languages? Is it worth learning Rust or Zig and dealing with the borrow checker or manual memory management in general, or are GC languages like Nim or Crystal good enough? I'm not doing any embedded programming by the way, just…

I haven’t tried Zig yet, but I liked Common Lisp better than Go and Rust for fast CLI apps/scripts. You need to dump an image with all your deps already loaded, and then you can run small scripts with instant compilation very fast. It’s an ergonomic language with good IDE support in emacs. There are some rough edges around (inconsistent API design, the package manager not using HTTPS, …), but tolerable.
Post reply on HN