Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In Go, pointers (mostly) don't go with slices in practice

utcc.utoronto.ca

Post reply on HN
Print view View original

91–98 of 98 posts

Re: In Go, pointers (mostly) don't go with slices in practice

#91

Post by masklinn »

jatone wrote: post #87
masklinn wrote: post #74

Earlier quoted context omitted.

> Go tells you very explicitly how it resizes slices by forcing you to write this: No, what most readers intuit from that is that `append` performs no mutation and that this is fine: s2 := append(s1, item) because it looks very much like, say, (def s2 (conj s1 item)) and often it will look like it works, especially at the smaller sizes, or if you never modify (or even use) s1. Except it's absolutely not fine. That's…

the idea that append performs no mutation is fairly insane. its name implies a mutation.

No, it does not. Appending is the addition of a suffix, it does not say anything about how it works.

Example: https://hackage.haskell.org/package/bytestring-0.11.1.0/docs...

Re: In Go, pointers (mostly) don't go with slices in practice

#92

Post by Thorrez »

unrealhoang wrote: post #40
watt wrote: post #28

Earlier quoted context omitted.

I don't get why address of slice returned from "append" does not change. Maybe in a trivial program like this the backing array can always be extended in-place, because there in memory fragmentation. Is that still true in an app that has considerable memory pressure and has GC running now and then?

Because there’s still space left in that slice (capacity > len), and the strategy of pre allocate capacity is to double the current (1,2,4,8,…), in this case 3 elements added => that slice was having a capacity of 4.

No, you can see that it was reallocated. At first the backing array started at 0xc0000be000. The append needed to do a reallocation and created a new backing array that starts at 0xc0000b8030.

Re: In Go, pointers (mostly) don't go with slices in practice

#93

Post by hsn915 »

masklinn wrote: post #74
hsn915 wrote: post #72

Earlier quoted context omitted.

Go tells you very explicitly how it resizes slices by forcing you to write this: slice = append(slice, item) By merely typing this all the time when you add elements, you intuitively understand that appending can potentially re-allocate the slice data in a totally different location, so any pointers you have taken before the resize are not guaranteed to be pointing to items in the new slice; just the old slice.

> Go tells you very explicitly how it resizes slices by forcing you to write this: No, what most readers intuit from that is that `append` performs no mutation and that this is fine: s2 := append(s1, item) because it looks very much like, say, (def s2 (conj s1 item)) and often it will look like it works, especially at the smaller sizes, or if you never modify (or even use) s1. Except it's absolutely not fine. That's…

This is fine:

    s2 := append(s1, item)
It's only not fine if you some how assume that s2 and s1 will always refer to the same data.

> other language separate slices and vectors

Go has a separate type for arrays, which is a value type

    var arr [10]int

Re: In Go, pointers (mostly) don't go with slices in practice

#94

Post by Thorrez »

edflsafoiewq wrote: post #13
Thorrez wrote: post #6

>Honestly, this is a strange and peculiar situation, although Go programmers have acclimatized to it. To programmers from other languages, such as C or C++, the concept of pointers to dynamically extensible arrays seems like a perfectly decent idea that surely should exist and work in Go. Well, it exists, and it "works" in the sense that it yields results and doesn't crash your program, but it doesn't "work" in the s…

C++ has std::vector, which is one level of abstraction above a slice; you push to a vector, and maybe the backing slice changes, but it's still the same vector. Go is unusual in not having a equivalent of vector.

Yes, std::vector and slice are different. My point is that a slice is similar to std::span.

With both slice and std::span, = does a shallow copy of just 2 or 3 pointers.

With std::vector = does a deep copy of every element, you have 2 distinct backing arrays.

Go forces you to use copy() and append() and rely on the garbage collector to make a slice fill the roll of std::vector. IMO it leads to some confusing code.

Re: In Go, pointers (mostly) don't go with slices in practice

#95

Post by steveklabnik »

klodolph wrote: post #89
steveklabnik wrote: post #88

Earlier quoted context omitted.

Off the top of my head you do something like https://play.rust-lang.org/?version=stable&mode=debug&editio... Yeah you gotta write it out per number of times. Luckily this rarely comes up in this specific form.

Wow, that code is as bad as I thought it would be. Yeesh. > Luckily this rarely comes up in this specific form. Well, yeah—it would come up rarely because it only solves the problem under very specific circumstances!

Yeah I mean you might be able to make it better, this is just what I would reach for first.

It is so rare that I don’t think I’ve ever seen it in any rust codebase I’ve ever looked at.

Re: In Go, pointers (mostly) don't go with slices in practice

#96

Post by Fire-Dragon-DoL »

s = append(s, elem)

I read this immediately as "create a new copy of the original slice with one additional element", so I presumed that was the case. It would actually be shocking the opposite, if I could end up modifying the original one (before the append) with a pointer to the new s, which seems to be the case!

Big gotcha there: treat slices as stateful at all time.

Since it has an assignment operation, it must be creating something new, otherwise it would have been a method of the slice itself

EDIT: I just realized the gotcha is not there at all, Go would consider the first slice to be of N length and the second slice of length N+1. Comparing the two slices would give an error at some point because one is shorter than the other, so the fact that the address changes or not is irrelevant. However I can see this becoming problematic with pointers, which proves the point of the article.

Re: In Go, pointers (mostly) don't go with slices in practice

#97

Post by masklinn »

hsn915 wrote: post #93
masklinn wrote: post #74

Earlier quoted context omitted.

> Go tells you very explicitly how it resizes slices by forcing you to write this: No, what most readers intuit from that is that `append` performs no mutation and that this is fine: s2 := append(s1, item) because it looks very much like, say, (def s2 (conj s1 item)) and often it will look like it works, especially at the smaller sizes, or if you never modify (or even use) s1. Except it's absolutely not fine. That's…

This is fine: s2 := append(s1, item) It's only not fine if you some how assume that s2 and s1 will always refer to the same data. > other language separate slices and vectors Go has a separate type for arrays, which is a value type var arr [10]int

> This is fine:

No, it is not.

> It's only not fine if you some how assume that s2 and s1 will always refer to the same data.

It's not fine if you assume anything about the interaction of s1 and s2. It's not fine if you assume they do alias, it's not fine if you assume they don't.

There is, fundamentally, no situation in which that construct is anything other than a footgun. `append` should only ever be used with the same slice on the LHS and RHS[0], or a brand new slice object constructed for the occasion on the RHS.

> Go has a separate type for arrays, which is a value type

An array is neither a vector nor a slice.

The issue is that Go's slices serve as both a vector and an actual slice, and the union of these interfaces creates footguns which don't exist in either.

[0] IFF that RHS is a either a non-parameter local, or a pointer to a slice

Re: In Go, pointers (mostly) don't go with slices in practice

#98

Post by jatone »

masklinn wrote: post #91
jatone wrote: post #87

Earlier quoted context omitted.

the idea that append performs no mutation is fairly insane. its name implies a mutation.

No, it does not. Appending is the addition of a suffix, it does not say anything about how it works. Example: https://hackage.haskell.org/package/bytestring-0.11.1.0/docs...

if you add a suffix, you've changed the value. that is a mutation.
Post reply on HN