Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How safe is Zig?

scattered-thoughts.net

Post reply on HN
Print view View original

251–259 of 259 posts

Re: How safe is Zig?

#251

Post by brabel »

Klonoar wrote: post #219
brabel wrote: post #217

Earlier quoted context omitted.

> they wouldn't be using it unless it was usable for "real work". We had to introduce Rust at work because we really needed some WASM functionality in one of our mostly JS frontends... anyway, I was excited about Rust and all and pushed the idea, implemented the whole thing and made presentations for other developers about Rust. I was thinking everyone would be as excited as I was and would jump at the chance of main…

I would be curious how your shop fares with Swift then, considering it may also look "like greek" to them. (Legit point, no snark)

Everyone seems alright with Kotlin, and Swift seems close enough to Kotlin that I think they would do fine (we do have a few frontend devs that do Swift BTW).

Re: How safe is Zig?

#252

Post by dkersten »

anonymoushn wrote: post #124
dkersten wrote: post #47

I’m not sure I understand the value of an allocator that doesn’t reuse allocations, as a bug prevention thing. Is it just for performance? (Since its never reused, allocation can simply be incrementing an offset by the size of the allocation)? Because beyond that, you can get the same benefit in C by simply never calling free on the memory you want to “protect” against use-after-free.

The allocations are freed and the addresses are never reused. So heap use-after-frees are segfaults.

Does that mean that each allocation is always page-aligned?

Re: How safe is Zig?

#253

Post by anonymoushn »

dkersten wrote: post #252
anonymoushn wrote: post #124

Earlier quoted context omitted.

The allocations are freed and the addresses are never reused. So heap use-after-frees are segfaults.

Does that mean that each allocation is always page-aligned?

Not sure on the details here. I'd have to try it out and see.

Larger allocations will be page-aligned, but if you make a bunch of very small allocations, they may go into the same pages, and freeing all but one per page of them may leave you with the pages still mapped. I've skimmed the GeneralPurposeAllocator code and know it has this sort of behavior at least sometimes, but I'm not really familiar with which things change in safe builds.

Re: How safe is Zig?

#254

Post by dkersten »

anonymoushn wrote: post #253
dkersten wrote: post #252

Earlier quoted context omitted.

Does that mean that each allocation is always page-aligned?

Not sure on the details here. I'd have to try it out and see. Larger allocations will be page-aligned, but if you make a bunch of very small allocations, they may go into the same pages, and freeing all but one per page of them may leave you with the pages still mapped. I've skimmed the GeneralPurposeAllocator code and know it has this sort of behavior at least sometimes, but I'm not really familiar with which things…

Thanks for the reply! I wasn't sure if you'd see mine, since it was two days late :)

So I suppose the "use after free" protection is "best effort": it will try to trap such use by unmapping the page, but there's no guarantee that it will definitely be the case.

I suppose that's ok, since use after free is a bug anyway. We want to catch the bugs and protect against data corruption, but the programmer is still on the hook for not making the bugs in the first place, so if the protection does miss one, well, at least it tried -- its just an extra layer. It would certainly be wasteful if allocating a few bytes uses up an entire page, just so it can be unmapped when the memory is freed.

Re: How safe is Zig?

#255

Post by staticassertion »

nyanpasu64 wrote: post #153
avgcorrection wrote: post #118

A meta point to make here but I don’t quite understand the pushback that Rust has gotten. How often does a language come around that flat out eliminates certain errors statically, and at the same time manages to stay in that low-level-capable pocket? And doesn’t require a PhD (or heck, a scholarly stipend) to use? Honestly that might be a once in a lifetime kind of thing. But not requiring a PhD (hyperbole) is not en…

Unsafe Rust is an esoteric language without iron-clad guarantees, and type-level programming and async Rust is an esoteric metalanguage ( https://hirrolot.github.io/posts/rust-is-hard-or-the-misery-... ). For example, matklad made a recent blog post on "Caches In Rust" ( https://matklad.github.io/2022/06/11/caches-in-rust.html ). The cache is built around https://docs.rs/elsa , which is built around https://docs.rs/s…

I don't think this is a great example of "sound C program turning into unsound Rust program". The crate isn't "unsound" in the way a C program would be - it's unsound in the sense that, given either 'unsafe' elsewhere or changes to how Rust constructs work (that are not guaranteed) a consumer of this crate could accidentally violate one of the necessary guarantees.

For a Rust program the bar is "has to be safe, even if some other part of the program uses unsafe". That seems like it's arguably a higher bar than C where everything is already "unsafe" in that same way.

Re: How safe is Zig?

#256

Post by staticassertion »

haberman wrote: post #196
avgcorrection wrote: post #118

A meta point to make here but I don’t quite understand the pushback that Rust has gotten. How often does a language come around that flat out eliminates certain errors statically, and at the same time manages to stay in that low-level-capable pocket? And doesn’t require a PhD (or heck, a scholarly stipend) to use? Honestly that might be a once in a lifetime kind of thing. But not requiring a PhD (hyperbole) is not en…

I invested a lot of time porting some parsing code I had written to Rust, with the vision that Rust is the memory-safe future. The code I was porting from used arenas, so I tried to use arenas in Rust also. Using arenas required a bunch of lifetime annotations everywhere, but I was happy to do it if I could get provable memory safety. I got everything working, but the moment I tried to wrap it in Python, it failed. T…

You can tell pyo3 that your type isn't Send and then it'll panic if the object is accessed from multiple threads. Given that that's the only safe option, that seems fine? You say that that's not acceptable for a production library but I don't see the issue.

You have the same restrictions in C++ except with worse consequences.

Re: How safe is Zig?

#257

Post by haberman »

staticassertion wrote: post #256
haberman wrote: post #196

Earlier quoted context omitted.

I invested a lot of time porting some parsing code I had written to Rust, with the vision that Rust is the memory-safe future. The code I was porting from used arenas, so I tried to use arenas in Rust also. Using arenas required a bunch of lifetime annotations everywhere, but I was happy to do it if I could get provable memory safety. I got everything working, but the moment I tried to wrap it in Python, it failed. T…

You can tell pyo3 that your type isn't Send and then it'll panic if the object is accessed from multiple threads. Given that that's the only safe option, that seems fine? You say that that's not acceptable for a production library but I don't see the issue. You have the same restrictions in C++ except with worse consequences.

> You have the same restrictions in C++ except with worse consequences.

In C++ it is safe because the arena is only used from one thread at a time.

To model this C++ pattern in Rust, what I would really want is:

1. Arena should be Sync, and not use interior mutability.

2. Arena::alloc() should do a dual borrow: (a) a mut borrow of the Arena metadata, only for the duration of the alloc() call, and (b) a non-mut (shared) borrow of the Arena data.

Because this kind of split borrow cannot be expressed in Rust AFAICS, (2) is not possible, so (1) is not feasible. This forces Bumpalo to be !Sync, which makes a direct Rust port of the C++ pattern impossible.

I've heard this called the "factory problem" for Rust: you cannot easily make a factory type in Rust that returns references, because if the create() operation mutates the factory, then the returned reference will have a mutable borrow on the factory.

The alternative would be to make a truly thread-safe arena/factory, which could be Sync with interior mutability, but that is an efficiency compromise due to synchronization overhead.

Re: How safe is Zig?

#258

Post by catlifeonmars »

_vvhw wrote: post #235
catlifeonmars wrote: post #234

Earlier quoted context omitted.

> for example, our new storage engine can address 100 TiB of storage using only 1 GiB of RAM. I’m a little confused by this statement. I assume by “address” you mean indexing, and the size of an index is related to the number of entries, not the amount of data being indexed. (For example, you could trivially address 100TiB using 1 address width of memory if all 100TiB belongs to the same key).

> I’m a little confused by this statement. I assume by “address” you mean indexing, and the size of an index is related to the number of entries, not the amount of data being indexed. Thanks for the question! What's in view here is an LSM-tree database storage engine. In general, these typically store keys between 8 and 32 bytes and values up to a few MiB. In our case, the question is how much memory is required for…

Thanks for taking the time to reply in detail! The metric makes much more sense when put into context.

Re: How safe is Zig?

#259

Post by _vvhw »

catlifeonmars wrote: post #258
_vvhw wrote: post #235

Earlier quoted context omitted.

> I’m a little confused by this statement. I assume by “address” you mean indexing, and the size of an index is related to the number of entries, not the amount of data being indexed. Thanks for the question! What's in view here is an LSM-tree database storage engine. In general, these typically store keys between 8 and 32 bytes and values up to a few MiB. In our case, the question is how much memory is required for…

Thanks for taking the time to reply in detail! The metric makes much more sense when put into context.

It's a pleasure. Let me know if you have any more questions about TigerBeetle. Our design doc is also here: https://github.com/coilhq/tigerbeetle/blob/main/docs/DESIGN....
Post reply on HN