Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zig: Pointer Stability for ArrayLists

ziglang.org

Post reply on HN
Print view View original

61–70 of 91 posts

Re: Zig: Pointer Stability for ArrayLists

#61

Post by insanitybit »

amluto wrote: post #7

This seems weak. In a language like Rust, the compiler will “lock” the pointers for you, and you can’t forget. In a language like C++ (and presumably Zig), one could, in theory at least, have the iterators and slices that reference the storage of a dynamic array hold some sort of lock that pins the storage. But this API requires the programmer to remember to lock the pointers and also requires the programmer to keep…

2026 and developers still use memory unsafe languages. I hope we get regulated at this point, disgusting.

Re: Zig: Pointer Stability for ArrayLists

#62

Post by insanitybit »

yefol wrote: post #48
duped wrote: post #21

Earlier quoted context omitted.

What are some examples of things you "always" need that require unsafe Rust?

[dead]

I haven't needed `unsafe` for performance since crates like zerocopy etc exist. It's been years, and I've worked hard to shave nanoseconds off of code, using valgrind to measure single digit changes to branch predictions.

Re: Zig: Pointer Stability for ArrayLists

#63

Post by insanitybit »

yefol wrote: post #50
svachalek wrote: post #13

Earlier quoted context omitted.

Hasn't the past 30 years of the Internet age taught us that given such trust, programmers will make the incorrect decision with horrifying predictability? The most trivial level of software security requires that pointer safety needs to be mathematically proven not up to human (or LLM) judgment.

[flagged]

Who cares what mojo handles? This is about Zig and memory safety.

Re: Zig: Pointer Stability for ArrayLists

#64

Post by insanitybit »

svachalek wrote: post #13
_bohm wrote: post #9

Earlier quoted context omitted.

Having written a bunch of Zig, I wouldn't say that the language design or culture explicitly encourages the use of pointers over indices in such situations. I would say it's more a language which trusts the programmer to make correct decisions about which constructs are appropriate in any given circumstance.

Hasn't the past 30 years of the Internet age taught us that given such trust, programmers will make the incorrect decision with horrifying predictability? The most trivial level of software security requires that pointer safety needs to be mathematically proven not up to human (or LLM) judgment.

Yes, undoubtedly. Anyone in denial of this should be legally barred from programming.

Re: Zig: Pointer Stability for ArrayLists

#65

Post by dataflow »

kllrnohj wrote: post #41
dataflow wrote: post #12

Earlier quoted context omitted.

I don't know Zig, but conceptually: a direct pointer is the fastest way to access an object. An arraylist is the fastest dynamic sequence of objects (fattest in access, not in growth). You use these when you need the performance. It's not often but it certainly happens. The most trivial example is a string that you append to but still need to pass to a C API in between that expects it to be contagious, but it's far m…

Indexing into an array is direct pointer access, there's just an addition in front of it but it's hard to imagine that showing up at all in even the tightest of benchmark loops

I can't speak for your imagination, but this absolutely does come up if you're writing high-performance code.

Also note that being able to access arbitrary objects (as opposed to objects in the same array) requires storing two pieces of information: an index and a pointer to the beginning of the array. So it can use twice as much memory, which affects your cache etc., though you don't even need that to see the effect.

Re: Zig: Pointer Stability for ArrayLists

#66

Post by sparkie »

frmdstryr wrote: post #19
amluto wrote: post #7

This seems weak. In a language like Rust, the compiler will “lock” the pointers for you, and you can’t forget. In a language like C++ (and presumably Zig), one could, in theory at least, have the iterators and slices that reference the storage of a dynamic array hold some sort of lock that pins the storage. But this API requires the programmer to remember to lock the pointers and also requires the programmer to keep…

Do any languages have a notion of "relative pointers"? So in the example if instead of appending "line" as ptr & len, it'd instead be appending an offset & len which could in theory be used to safely compute the actual location even with relocations.

I made a mini example in C.

It's awkward to do get right because you need an indirect pointer whose address remains fixed, but points to another pointer which can change (and is volatile).

While it might be possible to make something like this lockless - it's much simpler to stick a mutex in the array header. When we access the array_segment we can take a lock to prevent some other thread reallocating mid-way through accessing.

There's probably a few improvements that could be made. In particular it doesn't handle use-after-free, so it's not thread safe w.r.t cleanup.

https://godbolt.org/z/rYzn5KGre

Re: Zig: Pointer Stability for ArrayLists

#67

Post by mirashii »

amluto wrote: post #7

This seems weak. In a language like Rust, the compiler will “lock” the pointers for you, and you can’t forget. In a language like C++ (and presumably Zig), one could, in theory at least, have the iterators and slices that reference the storage of a dynamic array hold some sort of lock that pins the storage. But this API requires the programmer to remember to lock the pointers and also requires the programmer to keep…

To make matters worse, there’s also a weaker documentation problem. Where should one learn that they need to do this? zig.guide’s page on ArrayList doesn’t mention it. https://ziglang.org/documentation/master/std/#std.ArrayList doesn’t mention it, https://ziglang.org/documentation/master/std/#std.ArrayList doesn’t mention it at the top level, just a method in the midst of dozens of other methods. I honestly don’t know how one is meant to discover this outside of random blog posts.

Re: Zig: Pointer Stability for ArrayLists

#68

Post by surajrmal »

pron wrote: post #16
amluto wrote: post #7

This seems weak. In a language like Rust, the compiler will “lock” the pointers for you, and you can’t forget. In a language like C++ (and presumably Zig), one could, in theory at least, have the iterators and slices that reference the storage of a dynamic array hold some sort of lock that pins the storage. But this API requires the programmer to remember to lock the pointers and also requires the programmer to keep…

I reach for a low-level language only when I want low-level control over what operations happen and when, what memory is used and when etc.. At present, no language offers me this control and safety at the same time. With Rust, when I need such control (which is always, otherwise I would use a higher-level language), I need to give up safety, anyway, at which point I have no safety and the complexity of a language th…

I've written systems level code (drivers and os code) for years and outside of ffi, I've managed to go on year long stretches without touching unsafe. It's really not a commonly needed tool in a well architected code base with good libraries to encapsulate common reasons it might otherwise be necessary. And we don't really consider using unsafe taboo, it's just not necessary.

Re: Zig: Pointer Stability for ArrayLists

#69

Post by the__alchemist »

_bohm wrote: post #9
the__alchemist wrote: post #6

Earlier quoted context omitted.

My mental model of Zig is that it is explicitly the language for developers who prefer using pointers in business logic (instead of just in MMIMO, and are looking for something with improvements over C); i.e. exactly this class of abstraction.

Having written a bunch of Zig, I wouldn't say that the language design or culture explicitly encourages the use of pointers over indices in such situations. I would say it's more a language which trusts the programmer to make correct decisions about which constructs are appropriate in any given circumstance.

I appreciate the explanation! I'm a noob at this sort of thing. (My experience with pointers is limited to MMIO and C FFI).

Regarding trusting the programmer: is that in the context of pointers, or more broadly? I'm betting I'm missing more, e.g. maybe mutation control too across threads/cores etc? Or as a broad design principle? (I have written some Zig as a learning dive, but don't Grok it, as am waiting for some core functionality like HALs, GUI libs, 3D libs etc. Was also a bit miffed by the 'operator overloading will never be allowed' as the use cases where I use low level languages have a near total overlap with the ones where I use vectors, quaternions, and matrices.

Stated another way: I learned with higher level langs first, which I believe biased my mental model. Example: Say I want to use a C library in my rust program/lib. (Example: CMSIS DSP). One of the actions I take in the wrapper is convert the pointer to an array ref; my mental model is the param is a list of items; it is divorced from memory. If I want to read/write a reg, or access FLASH, that's where I look to pointers. I e I think zig vs others is about if you want to conflate or divorce collections and memory.

Re: Zig: Pointer Stability for ArrayLists

#70

Post by tialaramex »

sparkie wrote: post #44
afdbcreid wrote: post #36

Earlier quoted context omitted.

That is called an index. If you want it to be standalone, you can bundle it with the ArrayList.

I think parent was after base+offset+index rather than just base+index. Examples would be eg, `string_view` or `ArraySegment`. They hold some offset relative to a base allocation, and when we index the string_view or ArraySegment we're indexing relative to that offset.

Presumably `string_view` is referring to C++ std::string_view ?

It's not holding an "offset", it's a fat pointer, (ptr,len) or possibly (start,end)

You're imagining this as (string_ptr,offset,len) but that's 50% bigger for no practical benefit, you cannot unwind a std::string_view to get the string it's a view into, indeed there may never have been such a string.

Post reply on HN