Go(lang): Robust generic functions on slices
1–10 of 85 posts
Re: Go(lang): Robust generic functions on slices
#2A missing tidbit that may help contextualize this post: One of the things about Go that surprised me is that if you have a slice which does not represent the full capacity of the underlying array, you can go ahead and reslice it up to that full capacity even though it's a panic to access the things you're reslicing directly: https://go.dev/play/p/oThz2bNFwgr
Consequently, the GC has to assume that anything forward of any given slice into the underlying array may become accessible in the future as there is legal Go that can access it. It's still memory safe, but it surprised me.
I had some code that was using my incorrect belief that slices could not be resliced up in size to implement some light security boundaries. Fortunately it was still legal, because the code in question simply didn't slice things larger and it's not like I was allowing arbitrary user-supplied code to run, so it was still correct in what it was doing. But I was expecting the runtime to scream if I did somehow screw it up when in fact it may not, depending on the exact capacity and what happened when.
It's also asymmetric, as far as I know; you can slice forward into the array if there is capacity, but if you've got a slice that starts after index 0 in the backing array you can't use that slice to walk back into the underlying array. That is, with
s := []int{11, 12, 13, 14, 15}
s = s[2:]
Corrections (again not involving "unsafe", it's obvious those two values are still accessible through "unsafe") welcome. I was wrong once, it's easy to believe I could be wrong again.
Re: Go(lang): Robust generic functions on slices
#3A missing tidbit that may help contextualize this post: One of the things about Go that surprised me is that if you have a slice which does not represent the full capacity of the underlying array, you can go ahead and reslice it up to that full capacity even though it's a panic to access the things you're reslicing directly: https://go.dev/play/p/oThz2bNFwgr Consequently, the GC has to assume that anything forward of…
interesting catch, thanks for sharing. my poor memory is already racing thinking about cases where i may have left traps with the same assumption
Re: Go(lang): Robust generic functions on slices
#4A missing tidbit that may help contextualize this post: One of the things about Go that surprised me is that if you have a slice which does not represent the full capacity of the underlying array, you can go ahead and reslice it up to that full capacity even though it's a panic to access the things you're reslicing directly: https://go.dev/play/p/oThz2bNFwgr Consequently, the GC has to assume that anything forward of…
You can prevent later longer reslicing by add an additional cap() element to the slicing to shrink the capacity.
s = s[:3:3]
Re: Go(lang): Robust generic functions on slices
#5A missing tidbit that may help contextualize this post: One of the things about Go that surprised me is that if you have a slice which does not represent the full capacity of the underlying array, you can go ahead and reslice it up to that full capacity even though it's a panic to access the things you're reslicing directly: https://go.dev/play/p/oThz2bNFwgr Consequently, the GC has to assume that anything forward of…
It is reinforcing my belief that language design is hard. Go is supposed to be a simple language, one may write it for long time, but there will be some trap you discover once in a while.
Re: Go(lang): Robust generic functions on slices
#6A missing tidbit that may help contextualize this post: One of the things about Go that surprised me is that if you have a slice which does not represent the full capacity of the underlying array, you can go ahead and reslice it up to that full capacity even though it's a panic to access the things you're reslicing directly: https://go.dev/play/p/oThz2bNFwgr Consequently, the GC has to assume that anything forward of…
One detail in the latest 1.22 release is a change in the "slices" library [1] (the library is based on generics, introduced in 1.21) that helps avoid such cases:
> Functions that shrink the size of a slice (Delete, DeleteFunc, Compact, CompactFunc, and Replace) now zero the elements between the new length and the old length.
Re: Go(lang): Robust generic functions on slices
#7The problems with the API they point out are almost all things that rust's ownership system was built to solve.
Things like:
slices.Sort(s) // correct
slices.Compact(s) // incorrect
slices.Delete(s, ...) // incorrect
s := slices.Delete(s, ...) // incorrect if 's' is referenced again in the outer scope
s = slices.Delete(s, ...) // correct
Re: Go(lang): Robust generic functions on slices
#8The problems with the API they point out are almost all things that rust's ownership system was built to solve. Things like: slices.Sort(s) // correct slices.Compact(s) // incorrect slices.Delete(s, ...) // incorrect s := slices.Delete(s, ...) // incorrect if 's' is referenced again in the outer scope s = slices.Delete(s, ...) // correct All of those are solved by having functions like 'slices.Sort' take a '&mut' ref…
You don't even need a notion of ownership. A distinction between an immutable and mutable slice should be enough, because an immutable slice can never be changed which implies that its excess capacity (if any) can't be exploited for optimization.
Re: Go(lang): Robust generic functions on slices
#9The problems with the API they point out are almost all things that rust's ownership system was built to solve. Things like: slices.Sort(s) // correct slices.Compact(s) // incorrect slices.Delete(s, ...) // incorrect s := slices.Delete(s, ...) // incorrect if 's' is referenced again in the outer scope s = slices.Delete(s, ...) // correct All of those are solved by having functions like 'slices.Sort' take a '&mut' ref…
You don't even need a notion of ownership. A distinction between an immutable and mutable slice should be enough, because an immutable slice can never be changed which implies that its excess capacity (if any) can't be exploited for optimization.
None of these functions would apply to an immutable slice, so how is it related?
Re: Go(lang): Robust generic functions on slices
#10Earlier quoted context omitted.
You don't even need a notion of ownership. A distinction between an immutable and mutable slice should be enough, because an immutable slice can never be changed which implies that its excess capacity (if any) can't be exploited for optimization.
None of these functions would apply to an immutable slice, so how is it related?
If immutable and mutable slices are differently typed [2], it is natural to define two functions (say, `slices.Compact` vs. `slices.Compacted`) to handle each type, like Python `list.sort` vs. `sorted`. It should be natural to expect `slices.Compacted` to never alter its input, and any attempt to use a mutable version will be very explicit except for slicing [1].
[1] Especially given that the capacity is preserved by default, which contributes to the current confusion. See my older comment: https://news.ycombinator.com/item?id=39112735
[2] Originally "...are different" but edited for clarity.