Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zig: Pointer Stability for ArrayLists

ziglang.org

Post reply on HN
Print view View original

11–20 of 91 posts

Re: Zig: Pointer Stability for ArrayLists

#11

Post by hansvm »

_bohm wrote: post #2

It's a nice feature but I can't help feeling like, if you need a stable pointer to an item in a collection, ArrayList is the wrong data structure to use? Maybe someone can chime in and give me an example of when you'd do this instead of, e.g., just storing an index. Alternatively, you could use an Unrolled Linked List (FKA SegmentedList in Zig before it was removed in 0.16, not sure why).

I use it in a lot of places where I know the max capacity ahead of time -- ensureCapacity() followed by a lot of *AssumeCapacity()-styled commands. It's convenient for all of the ... convenience ... methods (append() requires some bookkeeping somewhere, appendSlice() requires more, and so on). In those usages, it's basically syntactic sugar over a slice. That's not a perfect solution, but it's reasonably good often enough that I keep doing it.

The proposed change doesn't do much for me personally (memory safety is ensured in other ways, and if it weren't I wouldn't be annoyed debugging the allocator-observed errors), but I could see myself using it at some other point in time for the same class of usages, or I could see other people relying on it when they choose that class of coding.

Re: Zig: Pointer Stability for ArrayLists

#12

Post by dataflow »

_bohm wrote: post #2

It's a nice feature but I can't help feeling like, if you need a stable pointer to an item in a collection, ArrayList is the wrong data structure to use? Maybe someone can chime in and give me an example of when you'd do this instead of, e.g., just storing an index. Alternatively, you could use an Unrolled Linked List (FKA SegmentedList in Zig before it was removed in 0.16, not sure why).

I don't know Zig, but conceptually: a direct pointer is the fastest way to access an object. An arraylist is the fastest dynamic sequence of objects (fattest in access, not in growth). You use these when you need the performance. It's not often but it certainly happens. The most trivial example is a string that you append to but still need to pass to a C API in between that expects it to be contagious, but it's far more useful than just for storing characters.

Re: Zig: Pointer Stability for ArrayLists

#13

Post by svachalek »

_bohm wrote: post #9
the__alchemist wrote: post #6

Earlier quoted context omitted.

My mental model of Zig is that it is explicitly the language for developers who prefer using pointers in business logic (instead of just in MMIMO, and are looking for something with improvements over C); i.e. exactly this class of abstraction.

Having written a bunch of Zig, I wouldn't say that the language design or culture explicitly encourages the use of pointers over indices in such situations. I would say it's more a language which trusts the programmer to make correct decisions about which constructs are appropriate in any given circumstance.

Hasn't the past 30 years of the Internet age taught us that given such trust, programmers will make the incorrect decision with horrifying predictability? The most trivial level of software security requires that pointer safety needs to be mathematically proven not up to human (or LLM) judgment.

Re: Zig: Pointer Stability for ArrayLists

#14

Post by Rendello »

Aside: one Zig (syntax) feature that I really missed in Rust is shown in the second code block, namely prefixed multi-line string literals à la:

    const text =
        \\This is a long comment
        \\But I can split it among lines arbitrarily
        \\And keep my indentation.
    ;
I've started using the Rust macro library `docstr` [1], which does the same thing:

    const TEXT: &'static str = docstr!(
        /// Now I can do it in Rust, too.
        /// I prefer this style a lot of the time
        /// for long texts.
    );
It even works with macros (example from the docs):

    let greeting: String = docstr!(format!
        /// Hello, my name is {name}.
        /// I am {} years old!
        age
    );
1. https://docs.rs/docstr/latest/docstr/

Re: Zig: Pointer Stability for ArrayLists

#15

Post by _bohm »

dataflow wrote: post #12
_bohm wrote: post #2

It's a nice feature but I can't help feeling like, if you need a stable pointer to an item in a collection, ArrayList is the wrong data structure to use? Maybe someone can chime in and give me an example of when you'd do this instead of, e.g., just storing an index. Alternatively, you could use an Unrolled Linked List (FKA SegmentedList in Zig before it was removed in 0.16, not sure why).

I don't know Zig, but conceptually: a direct pointer is the fastest way to access an object. An arraylist is the fastest dynamic sequence of objects (fattest in access, not in growth). You use these when you need the performance. It's not often but it certainly happens. The most trivial example is a string that you append to but still need to pass to a C API in between that expects it to be contagious, but it's far m…

Sure, in cases where you need elements to be contiguous in memory then certainly an unrolled linked list is not appropriate. There's usually not a meaningful performance difference between a pointer deref and an indexed array access, however.

Re: Zig: Pointer Stability for ArrayLists

#16

Post by pron »

amluto wrote: post #7

This seems weak. In a language like Rust, the compiler will “lock” the pointers for you, and you can’t forget. In a language like C++ (and presumably Zig), one could, in theory at least, have the iterators and slices that reference the storage of a dynamic array hold some sort of lock that pins the storage. But this API requires the programmer to remember to lock the pointers and also requires the programmer to keep…

I reach for a low-level language only when I want low-level control over what operations happen and when, what memory is used and when etc.. At present, no language offers me this control and safety at the same time. With Rust, when I need such control (which is always, otherwise I would use a higher-level language), I need to give up safety, anyway, at which point I have no safety and the complexity of a language that offers safety.

So right now, when we want control, we need to give up some safety, but weaker things are still helpful.

Also, in low-level code, the problem of "I might forget to do something" sometimes clashes with the problem of "I need to see exactly what operations are done and where". Various kinds of implicitness help with the former at the expense of the latter.

I'm not saying this is universally better than other approaches, but many people who do serious low-level programming would prefer this.

Re: Zig: Pointer Stability for ArrayLists

#17

Post by Someone »

amluto wrote: post #7

This seems weak. In a language like Rust, the compiler will “lock” the pointers for you, and you can’t forget. In a language like C++ (and presumably Zig), one could, in theory at least, have the iterators and slices that reference the storage of a dynamic array hold some sort of lock that pins the storage. But this API requires the programmer to remember to lock the pointers and also requires the programmer to keep…

I agree. https://news.ycombinator.com/item?id=49501582 says:

“I use it in a lot of places where I know the max capacity ahead of time -- ensureCapacity() followed by a lot of AssumeCapacity()-styled commands. It's convenient for all of the ... convenience ... methods (append() requires some bookkeeping somewhere, appendSlice() requires more, and so on). In those usages, it's basically syntactic sugar over a slice”*

I suspect “where I know the max capacity ahead of time” covers most if not all use cases (if it you use this without knowing max capacity, you either accept your code may panic, or you do some unlock, grow, lock again dance when you discover your initial estimate is wrong)

If so, wouldn’t adding a growable container where you specify capacity at construction time and removing access to the internal pointers of ArrayList be a better way to handle this?

Re: Zig: Pointer Stability for ArrayLists

#18

Post by afdbcreid »

They forgot to add (I believe this was not deliberate, maybe their users already infer that) that this only actually performs the check on Debug and ReleaseSafe modes, not on ReleaseFast mode. Which is reasonable I guess because this is a memory write/read/branch in a super hot code path, but undermines a large part of the guarantee in my opinion (doesn't Zig have a debug allocator that could catch the mistake in the example just as well?).

Re: Zig: Pointer Stability for ArrayLists

#19

Post by frmdstryr »

amluto wrote: post #7

This seems weak. In a language like Rust, the compiler will “lock” the pointers for you, and you can’t forget. In a language like C++ (and presumably Zig), one could, in theory at least, have the iterators and slices that reference the storage of a dynamic array hold some sort of lock that pins the storage. But this API requires the programmer to remember to lock the pointers and also requires the programmer to keep…

Do any languages have a notion of "relative pointers"? So in the example if instead of appending "line" as ptr & len, it'd instead be appending an offset & len which could in theory be used to safely compute the actual location even with relocations.

Re: Zig: Pointer Stability for ArrayLists

#20

Post by pjmlp »

pron wrote: post #16
amluto wrote: post #7

This seems weak. In a language like Rust, the compiler will “lock” the pointers for you, and you can’t forget. In a language like C++ (and presumably Zig), one could, in theory at least, have the iterators and slices that reference the storage of a dynamic array hold some sort of lock that pins the storage. But this API requires the programmer to remember to lock the pointers and also requires the programmer to keep…

I reach for a low-level language only when I want low-level control over what operations happen and when, what memory is used and when etc.. At present, no language offers me this control and safety at the same time. With Rust, when I need such control (which is always, otherwise I would use a higher-level language), I need to give up safety, anyway, at which point I have no safety and the complexity of a language th…

Except the point that Zig should do better than Object Pascal, Modula-2, with solutions already available on Insure++ and friends for use after free, 30 years ago.
Post reply on HN