Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zig: Pointer Stability for ArrayLists

ziglang.org

Post reply on HN
Print view View original

31–40 of 91 posts

Re: Zig: Pointer Stability for ArrayLists

#31

Post by ivanjermakov »

_bohm wrote: post #2

It's a nice feature but I can't help feeling like, if you need a stable pointer to an item in a collection, ArrayList is the wrong data structure to use? Maybe someone can chime in and give me an example of when you'd do this instead of, e.g., just storing an index. Alternatively, you could use an Unrolled Linked List (FKA SegmentedList in Zig before it was removed in 0.16, not sure why).

I count this as a "rookie at system programming" mistake alongside returning a reference to a local variable. Rust is great at this because borrow checker can catch those at compile time and it can _teach_ devs to not do that.

Re: Zig: Pointer Stability for ArrayLists

#32

Post by ivanjermakov »

afdbcreid wrote: post #18

They forgot to add (I believe this was not deliberate, maybe their users already infer that) that this only actually performs the check on Debug and ReleaseSafe modes, not on ReleaseFast mode. Which is reasonable I guess because this is a memory write/read/branch in a super hot code path, but undermines a large part of the guarantee in my opinion (doesn't Zig have a debug allocator that could catch the mistake in the…

Debug allocator can't catch it because it's not an allocation bug. Debug allocator finds bugs by marking memory during alloc/free and inspects them upon deinit. Pointer to a memory location change is not something allocator has control over. Possible solutions: smart array list implementation (this article), move semantic analysis (Rust's borrow checker), runtime introspection (https://fil-c.org/).

Re: Zig: Pointer Stability for ArrayLists

#33

Post by slopinthebag »

amluto wrote: post #7

This seems weak. In a language like Rust, the compiler will “lock” the pointers for you, and you can’t forget. In a language like C++ (and presumably Zig), one could, in theory at least, have the iterators and slices that reference the storage of a dynamic array hold some sort of lock that pins the storage. But this API requires the programmer to remember to lock the pointers and also requires the programmer to keep…

This is how it is with languages which provide less guarantees than Rust. Sure you can try to hold all the invariants and restrictions in your head, but a sufficiently advanced compiler can do this for you without the possibility of making mistakes. I have no idea why people claim that's too restrictive - if you're not enforcing those rules manually you're just setting yourself up for issues down the road.

Re: Zig: Pointer Stability for ArrayLists

#34

Post by slopinthebag »

pron wrote: post #16
amluto wrote: post #7

This seems weak. In a language like Rust, the compiler will “lock” the pointers for you, and you can’t forget. In a language like C++ (and presumably Zig), one could, in theory at least, have the iterators and slices that reference the storage of a dynamic array hold some sort of lock that pins the storage. But this API requires the programmer to remember to lock the pointers and also requires the programmer to keep…

I reach for a low-level language only when I want low-level control over what operations happen and when, what memory is used and when etc.. At present, no language offers me this control and safety at the same time. With Rust, when I need such control (which is always, otherwise I would use a higher-level language), I need to give up safety, anyway, at which point I have no safety and the complexity of a language th…

But the point of unsafe {} in Rust is not that you should never use it, it's that it creates a clear boundary between code that is safe and the code that needs that lower level control. In other languages, everything is inside an unsafe block. If everything you do requires such low level control over every allocation and access, it sounds like you should be using assembly.

Re: Zig: Pointer Stability for ArrayLists

#35

Post by afdbcreid »

ivanjermakov wrote: post #32
afdbcreid wrote: post #18

They forgot to add (I believe this was not deliberate, maybe their users already infer that) that this only actually performs the check on Debug and ReleaseSafe modes, not on ReleaseFast mode. Which is reasonable I guess because this is a memory write/read/branch in a super hot code path, but undermines a large part of the guarantee in my opinion (doesn't Zig have a debug allocator that could catch the mistake in the…

Debug allocator can't catch it because it's not an allocation bug. Debug allocator finds bugs by marking memory during alloc/free and inspects them upon deinit. Pointer to a memory location change is not something allocator has control over. Possible solutions: smart array list implementation (this article), move semantic analysis (Rust's borrow checker), runtime introspection ( https://fil-c.org/ ).

It is an allocation bug, it's a use-after-free. It's only a UAF if you actually have a reallocation (something that happens in the given example), but debug allocators don't require you to annotate your code.

Re: Zig: Pointer Stability for ArrayLists

#36

Post by afdbcreid »

frmdstryr wrote: post #19
amluto wrote: post #7

This seems weak. In a language like Rust, the compiler will “lock” the pointers for you, and you can’t forget. In a language like C++ (and presumably Zig), one could, in theory at least, have the iterators and slices that reference the storage of a dynamic array hold some sort of lock that pins the storage. But this API requires the programmer to remember to lock the pointers and also requires the programmer to keep…

Do any languages have a notion of "relative pointers"? So in the example if instead of appending "line" as ptr & len, it'd instead be appending an offset & len which could in theory be used to safely compute the actual location even with relocations.

That is called an index. If you want it to be standalone, you can bundle it with the ArrayList.

Re: Zig: Pointer Stability for ArrayLists

#37

Post by afdbcreid »

pron wrote: post #16
amluto wrote: post #7

This seems weak. In a language like Rust, the compiler will “lock” the pointers for you, and you can’t forget. In a language like C++ (and presumably Zig), one could, in theory at least, have the iterators and slices that reference the storage of a dynamic array hold some sort of lock that pins the storage. But this API requires the programmer to remember to lock the pointers and also requires the programmer to keep…

I reach for a low-level language only when I want low-level control over what operations happen and when, what memory is used and when etc.. At present, no language offers me this control and safety at the same time. With Rust, when I need such control (which is always, otherwise I would use a higher-level language), I need to give up safety, anyway, at which point I have no safety and the complexity of a language th…

> With Rust, when I need such control (which is always, otherwise I would use a higher-level language), I need to give up safety, anyway, at which point I have no safety and the complexity of a language that offers safety.

This is a very, very, very common claim. And unfortunately I have no other way to describe it other than a strawman.

In 95% (at least) of the application that need systems programming (not to talk about all applications that don't necessarily need it but will benefit from the performance and it wasn't an option because C++ wasn't an option), you have at most 20% (wildly overestimating) of code that needs to be unsafe. The rest could be completely safe. And amongst code that must be unsafe, you can very commonly encapsulate it in some safe pattern. Many times even extract it to a reusable crate.

That is the point of Rust. Not avoiding unsafety, but limiting and encapsulating it. And evidence proves that to work (for example https://blog.google/security/rust-in-android-move-fast-fix-t...).

Re: Zig: Pointer Stability for ArrayLists

#38

Post by geon »

frmdstryr wrote: post #19
amluto wrote: post #7

This seems weak. In a language like Rust, the compiler will “lock” the pointers for you, and you can’t forget. In a language like C++ (and presumably Zig), one could, in theory at least, have the iterators and slices that reference the storage of a dynamic array hold some sort of lock that pins the storage. But this API requires the programmer to remember to lock the pointers and also requires the programmer to keep…

Do any languages have a notion of "relative pointers"? So in the example if instead of appending "line" as ptr & len, it'd instead be appending an offset & len which could in theory be used to safely compute the actual location even with relocations.

Array indexing?

Re: Zig: Pointer Stability for ArrayLists

#39

Post by Gibbon1 »

Rendello wrote: post #14

Aside: one Zig (syntax) feature that I really missed in Rust is shown in the second code block, namely prefixed multi-line string literals à la: const text = \\This is a long comment \\But I can split it among lines arbitrarily \\And keep my indentation. ; I've started using the Rust macro library `docstr` [1], which does the same thing: const TEXT: &'static str = docstr!( /// Now I can do it in Rust, too. /// I pref…

In C you can do that.

   printf ("Things:\n"
     " thing1=%u\n"
     " thing2=%u\n"
     " thing3=%u\n",
        thing1,
        thing2,
        thing3);

Re: Zig: Pointer Stability for ArrayLists

#40

Post by Joker_vD »

sparkie wrote: post #30
frmdstryr wrote: post #29

Earlier quoted context omitted.

After more searching I found this article https://www.gingerbill.org/article/2020/05/17/relative-point... A far pointer sounds like the global based pointer described in that article. The far pointer Wikipedia article says they are problematic but doesn't give much reasoning as to why.

Far pointers are for accessing memory in different segments. They're basically obsolete now. They were necessary in older machines with limited sized pointers or address spaces. GCC still supports `__seg_fs` and `__seg_gs`, which behave similar to `far` in the example on the wiki page, as the FS and GS segment registers are still valid in x86-64 and used for TLS. Clang uses attributes `address_space(257)` and `addres…

> Far pointers are for accessing memory in different segments. They're basically obsolete now.

Project CHERI would like to disagree.

Post reply on HN