Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zig: Pointer Stability for ArrayLists

ziglang.org

Post reply on HN
Print view View original

41–50 of 91 posts

Re: Zig: Pointer Stability for ArrayLists

#41

Post by kllrnohj »

dataflow wrote: post #12
_bohm wrote: post #2

It's a nice feature but I can't help feeling like, if you need a stable pointer to an item in a collection, ArrayList is the wrong data structure to use? Maybe someone can chime in and give me an example of when you'd do this instead of, e.g., just storing an index. Alternatively, you could use an Unrolled Linked List (FKA SegmentedList in Zig before it was removed in 0.16, not sure why).

I don't know Zig, but conceptually: a direct pointer is the fastest way to access an object. An arraylist is the fastest dynamic sequence of objects (fattest in access, not in growth). You use these when you need the performance. It's not often but it certainly happens. The most trivial example is a string that you append to but still need to pass to a C API in between that expects it to be contagious, but it's far m…

Indexing into an array is direct pointer access, there's just an addition in front of it but it's hard to imagine that showing up at all in even the tightest of benchmark loops

Re: Zig: Pointer Stability for ArrayLists

#42

Post by sparkie »

Joker_vD wrote: post #40
sparkie wrote: post #30

Earlier quoted context omitted.

Far pointers are for accessing memory in different segments. They're basically obsolete now. They were necessary in older machines with limited sized pointers or address spaces. GCC still supports `__seg_fs` and `__seg_gs`, which behave similar to `far` in the example on the wiki page, as the FS and GS segment registers are still valid in x86-64 and used for TLS. Clang uses attributes `address_space(257)` and `addres…

> Far pointers are for accessing memory in different segments. They're basically obsolete now. Project CHERI would like to disagree.

That's Fat pointers, not Far pointers. A fat pointer is a pointer with some other associated data which is stored in the pointer itself - typically by widening the number of bits used to hold a pointer value. The addressable bits usually remain unchanged - the added bits contain the auxiliary data.

Segmentation isn't used. There's no separate registers to hold the bounds information in CHERI - the bounds are held in the pointer value, unlike for example, the now obsolete Intel MPX, which held bounds information in separate registers.

There's some similarity to segmentation because the CHERI pointer restricts which addresses can be accessed, but I wouldn't compare them to far pointers.

Most modern processors have a single linear virtual address space and don't use segmentation, and even where segment registers exist (eg, FS and GS on x86-64), they're only superficial "address spaces" - allocated sections of the process's linear virtual address space which could be accessed without segmentation registers if you knew the base address held in FS or GS.

Re: Zig: Pointer Stability for ArrayLists

#43

Post by adzm »

frmdstryr wrote: post #19
amluto wrote: post #7

This seems weak. In a language like Rust, the compiler will “lock” the pointers for you, and you can’t forget. In a language like C++ (and presumably Zig), one could, in theory at least, have the iterators and slices that reference the storage of a dynamic array hold some sort of lock that pins the storage. But this API requires the programmer to remember to lock the pointers and also requires the programmer to keep…

Do any languages have a notion of "relative pointers"? So in the example if instead of appending "line" as ptr & len, it'd instead be appending an offset & len which could in theory be used to safely compute the actual location even with relocations.

in c++, boost interprocess has offset_ptr which is useful since the shared data structure may be mapped at different locations in memory in each process

Re: Zig: Pointer Stability for ArrayLists

#44

Post by sparkie »

afdbcreid wrote: post #36
frmdstryr wrote: post #19

Earlier quoted context omitted.

Do any languages have a notion of "relative pointers"? So in the example if instead of appending "line" as ptr & len, it'd instead be appending an offset & len which could in theory be used to safely compute the actual location even with relocations.

That is called an index. If you want it to be standalone, you can bundle it with the ArrayList.

I think parent was after base+offset+index rather than just base+index.

Examples would be eg, `string_view` or `ArraySegment`. They hold some offset relative to a base allocation, and when we index the string_view or ArraySegment we're indexing relative to that offset.

Re: Zig: Pointer Stability for ArrayLists

#45

Post by dminik »

afdbcreid wrote: post #36
frmdstryr wrote: post #19

Earlier quoted context omitted.

Do any languages have a notion of "relative pointers"? So in the example if instead of appending "line" as ptr & len, it'd instead be appending an offset & len which could in theory be used to safely compute the actual location even with relocations.

That is called an index. If you want it to be standalone, you can bundle it with the ArrayList.

I wish languages made it easier (or possible) to track index ownership at compile time.

Re: Zig: Pointer Stability for ArrayLists

#47

Post by yefol »

afdbcreid wrote: post #37
pron wrote: post #16

Earlier quoted context omitted.

I reach for a low-level language only when I want low-level control over what operations happen and when, what memory is used and when etc.. At present, no language offers me this control and safety at the same time. With Rust, when I need such control (which is always, otherwise I would use a higher-level language), I need to give up safety, anyway, at which point I have no safety and the complexity of a language th…

> With Rust, when I need such control (which is always, otherwise I would use a higher-level language), I need to give up safety, anyway, at which point I have no safety and the complexity of a language that offers safety. This is a very, very, very common claim. And unfortunately I have no other way to describe it other than a strawman. In 95% (at least) of the application that need systems programming (not to talk…

[flagged]

Re: Zig: Pointer Stability for ArrayLists

#48

Post by yefol »

duped wrote: post #21
pron wrote: post #16

Earlier quoted context omitted.

I reach for a low-level language only when I want low-level control over what operations happen and when, what memory is used and when etc.. At present, no language offers me this control and safety at the same time. With Rust, when I need such control (which is always, otherwise I would use a higher-level language), I need to give up safety, anyway, at which point I have no safety and the complexity of a language th…

What are some examples of things you "always" need that require unsafe Rust?

[dead]

Re: Zig: Pointer Stability for ArrayLists

#50

Post by yefol »

svachalek wrote: post #13
_bohm wrote: post #9

Earlier quoted context omitted.

Having written a bunch of Zig, I wouldn't say that the language design or culture explicitly encourages the use of pointers over indices in such situations. I would say it's more a language which trusts the programmer to make correct decisions about which constructs are appropriate in any given circumstance.

Hasn't the past 30 years of the Internet age taught us that given such trust, programmers will make the incorrect decision with horrifying predictability? The most trivial level of software security requires that pointer safety needs to be mathematically proven not up to human (or LLM) judgment.

[flagged]
Post reply on HN