How safe is Zig? • Hacker News Board

Re: How safe is Zig?

#231

Post by crabbygrabby » Fri, Jun 24, 2022, 11:44 AM UTC

LAC-Tech wrote: post #151

Klonoar wrote: post #147

Earlier quoted context omitted.

>Zigs solution is hands down better for actually getting work done Rust has seen significant usage in large companies; they wouldn't be using it unless it was usable for "real work". >Full disclaimer, I'm pretty bad at systems programming. Zig is the only one I've used where I didn't feel like memory management was a massive headache. I'd say this about Rust, though. Rust's mental model is very straightforward if you…

Rust has seen significant usage in large companies; they wouldn't be using it unless it was usable for "real work". I didn't say it wasn't usable. I said I found Zig more usable. I'd say this about Rust, though. Rust's mental model is very straightforward if you accept the borrow-checker and stop fighting it. Can you list any examples of what you think is a headache...? Mate, I didn't start learning rust in order to…

Sounds like you got annoyed trying to learn something new, quit, and decided it's not worth your time. When people tell you it takes a few weeks to get the hang of rust they aren't kidding. Most people aren't the exception to that, but once you do get it, it's really great... Not kidding about that either...

Re: How safe is Zig?

#232

Post by galangalalgol » Fri, Jun 24, 2022, 11:48 AM UTC

longrod wrote: post #198

galangalalgol wrote: post #185

Earlier quoted context omitted.

why does go always get brought up when talking about rust or nim or now zig? Its got gc, a large std lib, less latency than java but noticably slower, and its not suited for embedded or drivers. Its a completely different language for a completely different niche than zig or rist, though maybe I could see a comparison to nim... Maybe. I do like go for what it is, batteries included back end language with syntax nicer…

Go is a compiled language with comparable performance to Zig/Rust/C in a lot of use cases. It's heavily ergonomic for network/server side programming, it's safe, easy to use with a tiny footprint. Rust is not only or even primarily used in embedded or driver programming. Indeed, I see more user space software built in rust than in embedded nowadays. As for Zig, it's so new and alpha that it didn't even find a niche y…

A very good point about rust (and c++ and c) being used outside embedded and systems. That's fair. But when people do that, they do it for performance, and go is easily 5x slower in my experience. It feels faster than java because the lower latency makes it feel more responsive, but its takes half again as long to finish the same tasks. And c# absolutely blows its doors off, closer to twice as long as c#. And c# feels responsive as well. And don't all those languages make the same safety guarantees as go?

About detecting footguns, I was honestly asking, I thought maybe you could have a borrow checker, but none of the zero cost abstractions that rust uses to make it mostly tolerable. I'm not saying I want to use that language, but I do like explicit languages if they aren't too verbose. Go is a good example of that I think, until you get into the abstractions around parallelism.

Re: How safe is Zig?

#233

Post by jhgb » Fri, Jun 24, 2022, 11:59 AM UTC

notriddle wrote: post #157

coldtea wrote: post #140

Earlier quoted context omitted.

> If option A has 20 defects and option B has the superset of 25 defects then option A is better Only if "defect count" is what you care for. What if you don't give a fuck about defect count, but prefer simplicity to explore/experiment quickly, ease of use, time to market, and so on?

Then you don't want Zig or Rust. Use a language with a GC. Exploratory programming is a lot more pleasant when you don't have to worry about calling free() at the right time. I've had success with PHP and Elixir for productive, exploratory programming, not just because of their GCs, but also because they both support REPL-driven development and hot code reloading.

> Then you don't want Zig or Rust. Use a language with a GC.

Zig allows custom allocators pretty much everywhere, right? Would it be impossible to provide it with a GC-based allocator for increased convenience at the cost of a little performance for programs (or parts of programs) where convenience is preferred? Perhaps libgc could be an inspiration here.

Re: How safe is Zig?

#234

Post by catlifeonmars » Fri, Jun 24, 2022, 1:57 PM UTC

_vvhw wrote: post #42

verdagon wrote: post #17

A lot of embedded devices and safety critical software sometimes don't even use a heap, and instead use pre-allocated chunks of memory whose size is calculated beforehand. It's memory safe, and has much more deterministic execution time. This is also a popular approach in games, especially ones with entity-component-system architectures. I'm excited about Zig for these use cases especially, it can be a much easier ap…

This is almost what we do for TigerBeetle, a new distributed database being written in Zig. All memory is statically allocated at startup [1]. Thereafter, there are zero calls to malloc() or free(). We run a single-threaded control plane for a simple concurrency model, and because we use io_uring—multithreaded I/O is less of a necessary evil than it used to be. I find that the design is more memory efficient because…

> for example, our new storage engine can address 100 TiB of storage using only 1 GiB of RAM.

I’m a little confused by this statement. I assume by “address” you mean indexing, and the size of an index is related to the number of entries, not the amount of data being indexed. (For example, you could trivially address 100TiB using 1 address width of memory if all 100TiB belongs to the same key).

Re: How safe is Zig?

#235

Post by _vvhw » Fri, Jun 24, 2022, 3:01 PM UTC

catlifeonmars wrote: post #234

_vvhw wrote: post #42

Earlier quoted context omitted.

This is almost what we do for TigerBeetle, a new distributed database being written in Zig. All memory is statically allocated at startup [1]. Thereafter, there are zero calls to malloc() or free(). We run a single-threaded control plane for a simple concurrency model, and because we use io_uring—multithreaded I/O is less of a necessary evil than it used to be. I find that the design is more memory efficient because…

> for example, our new storage engine can address 100 TiB of storage using only 1 GiB of RAM. I’m a little confused by this statement. I assume by “address” you mean indexing, and the size of an index is related to the number of entries, not the amount of data being indexed. (For example, you could trivially address 100TiB using 1 address width of memory if all 100TiB belongs to the same key).

> I’m a little confused by this statement. I assume by “address” you mean indexing, and the size of an index is related to the number of entries, not the amount of data being indexed.

Thanks for the question!

What's in view here is an LSM-tree database storage engine. In general, these typically store keys between 8 and 32 bytes and values up to a few MiB.

In our case, the question is how much memory is required for the LSM-tree to be able to index 100 TiB worth of key/value storage, where:

  * keys are between 8 and 32 bytes,
  * values are between 8 and 128 bytes,
  * keys and values are stored in tables up to 64 MiB,
  * each table requires between 128 and 256 bytes of metadata to be kept in memory,
  * auxiliary data structures such as a mutable/immutable table must be kept in memory, and where
  * all memory required by the engine must be statically allocated.

That's alot of small keys and values!

Typically, a storage system might require at least an order of magnitude more than 1 GiB of memory to keep track of that many keys using an LSM-tree as index, even using dynamic allocation, which only needs to allocate as needed.

Another way to think of this is as a filesystem, since it's a very similar problem. Imagine you stored 100 TiB worth of 4096 byte files in ZFS. How much RAM would that require for ZFS to be able to keep track of everything?

Re: How safe is Zig?

#236

Post by notriddle » Fri, Jun 24, 2022, 3:19 PM UTC

pcwalton wrote: post #164

notriddle wrote: post #160

Earlier quoted context omitted.

It's actually not very hard to cause. It would happen if your system reuses buffers without actually going through the allocator. Something like this: let mut buf = vec![0; DEFAULT_BUFFER_SIZE]; let database_frame_len = database_socket.read(&mut buf)?; // use 1 let database_result = Database::parse(&buf[..database_frame_len])?; let html_template = HtmlTemplate::new(database_result); let html_len = html_template.write…

Couldn't you fix that by just clearing the buffer in between the two calls? It won't be measurably slower to do that because the buffer should retain its allocated capacity.

It's not a difficult bug to fix, but it is a very difficult bug to detect, since miri won't think anything is wrong.

Re: How safe is Zig?

#237

Post by randyrand » Fri, Jun 24, 2022, 5:06 PM UTC

kaba0 wrote: post #227

randyrand wrote: post #159

Earlier quoted context omitted.

I feel like the most user friendly solution for Use After Free is to just use Reference Counting. Basically, just copy Objective-C's "weak_ptr" design. For every allocation, also store on the heap a "reference object", that keeps track of this new reference. struct reference_t { // the ptr returned by malloc void* ptr; // the stack/heap location that ptr was written to. // i.e the reference location. void* referenceA…

Reference counting is an incomplete solution it itself - it leaks circular references, and on multithreaded programs it is quite slow. Though some optimizations could be done statically to improve on that. All in all, a mark-and-sweep will likely be faster (in throughput at least) let alone a good GC.

Circular references wouldn’t matter.

The programmer would still decide when to free the object, not some automatic system. Manual memory management, with Ref Counting just to add some additional Use After Free checks.

Re: How safe is Zig?

#238

Post by hndc » Fri, Jun 24, 2022, 5:29 PM UTC

galangalalgol wrote: post #232

longrod wrote: post #198

Earlier quoted context omitted.

Go is a compiled language with comparable performance to Zig/Rust/C in a lot of use cases. It's heavily ergonomic for network/server side programming, it's safe, easy to use with a tiny footprint. Rust is not only or even primarily used in embedded or driver programming. Indeed, I see more user space software built in rust than in embedded nowadays. As for Zig, it's so new and alpha that it didn't even find a niche y…

A very good point about rust (and c++ and c) being used outside embedded and systems. That's fair. But when people do that, they do it for performance, and go is easily 5x slower in my experience. It feels faster than java because the lower latency makes it feel more responsive, but its takes half again as long to finish the same tasks. And c# absolutely blows its doors off, closer to twice as long as c#. And c# feel…

> "in my experience"

Here's some data: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Ignoring the programs using x86 intrinsics to do vectorized math, the top-performing Rust, Go, Java, and C# programs are all written in a simple, straightforward style; each is practically a direct translation from the other. The Rust program is fastest, but the others come in at 1.6x, 1.7x, and 1.7x the Rust program, respectively. These are not significant differences for the vast majority of applications.

Of course this is one artificial benchmark, and on that website you will find others where the Java, C#, or Go program will do especially well or especially poorly. But it's clear that those three are all roughly in the same performance class..

Re: How safe is Zig?

#239

Post by modeless » Fri, Jun 24, 2022, 6:01 PM UTC

crabbygrabby wrote: post #230

modeless wrote: post #195

Earlier quoted context omitted.

Whether you allocate or not is a property of the language, not the GC. A lot of GC'd languages encourage or even force allocations all over the place. But maybe we could do better in a new language.

I see a lot of people put rust down for what it is today, and then in the next breathe wistfully wonder about the future of zig, or some other language, etc. It's just not fair. Rust today let's you avoid allocations, if you want smart pointers you can have them today, I'd you want a GC inside rust, you can have that today as well. Don't need to wait for another language lol.

Enough of Rust is set in stone now that it will never be the language I want. There's nothing fair or unfair about it. That's just the way it is. I still think it is a good language.

Re: How safe is Zig?

#240

Post by LAC-Tech » Fri, Jun 24, 2022, 8:05 PM UTC

crabbygrabby wrote: post #231

LAC-Tech wrote: post #151

Earlier quoted context omitted.

Rust has seen significant usage in large companies; they wouldn't be using it unless it was usable for "real work". I didn't say it wasn't usable. I said I found Zig more usable. I'd say this about Rust, though. Rust's mental model is very straightforward if you accept the borrow-checker and stop fighting it. Can you list any examples of what you think is a headache...? Mate, I didn't start learning rust in order to…

Sounds like you got annoyed trying to learn something new, quit, and decided it's not worth your time. When people tell you it takes a few weeks to get the hang of rust they aren't kidding. Most people aren't the exception to that, but once you do get it, it's really great... Not kidding about that either...

I think I spent about two months. You're right, I do resent the time I spent on it. In a way I'd like to warn others.

The turning point came for me when talking to someone using rust pretty much since it first came out. He basically said "yeah, I still don't know why it doesn't compile either sometimes". He's smarter than me, and all he's managed to do in years of learning it is have a knack for fixing errors without understanding them.

I need some brain capacity left over for actually solving a problem.