Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In Go, pointers (mostly) don't go with slices in practice

utcc.utoronto.ca

Post reply on HN
Print view View original

71–80 of 98 posts

Re: In Go, pointers (mostly) don't go with slices in practice

#71

Post by masklinn »

sly010 wrote: post #49
friendzis wrote: post #10

Earlier quoted context omitted.

> This is safe, but confuses some people. Rust has created a weird perception that memory safety equals safety. Language is a tool and it should work with me: it is extremely important that my understanding of what the program should do aligns with what it actually does. The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container". Taking a pointer/reference…

> it does wrong thing without warning It's not without warning, it's a well documented behavior. Just think of slices as "immutable", "pass-by-value" data structures (with a relatively efficient implementation) and everything falls into place. Mutating them in any way is actually a special case that you do only for performance reason (i.e. you can pre-allocate and fill if you know the size ahead of time) but - as alw…

> It's not without warning, it's a well documented behavior.

Ah yes, the usual excuse for it being fine that C APIs are completely broken and half of them can not be used correctly.

> Just think of slices as "immutable", "pass-by-value" data structures (with a relatively efficient implementation) and everything falls into place.

Except when they don't because `append` itself amortises allocations, which means if you treat slices as immutable and pass by value you will end up with slices sharing a backing array with leftover capacity and stomping on one another's data.

> Mutating them in any way is actually a special case that you do only for performance reason

Mutating them is literally what the normal Go API usage has you do. If you want to avoid mutating slices you need to write this abortion:

    s1 := append(append([]int(nil), s0...), item)
and if you do that in a loop, you get to feature on https://accidentallyquadratic.tumblr.com

Re: In Go, pointers (mostly) don't go with slices in practice

#72

Post by hsn915 »

friendzis wrote: post #10
Animats wrote: post #3

Well, of course slices work that way. Think about what happens if you have a reference to a slice in an array and you shrank the array to 0. You've just created a dangling pointer. In Go, you get a stable version of the old data, and the garbage collector tracks that you still have a reference to it. This is safe, but confuses some people. In Rust, the borrow checker won't let you modify the array while you have a re…

> This is safe, but confuses some people. Rust has created a weird perception that memory safety equals safety. Language is a tool and it should work with me: it is extremely important that my understanding of what the program should do aligns with what it actually does. The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container". Taking a pointer/reference…

Go tells you very explicitly how it resizes slices by forcing you to write this:

    slice = append(slice, item)
By merely typing this all the time when you add elements, you intuitively understand that appending can potentially re-allocate the slice data in a totally different location, so any pointers you have taken before the resize are not guaranteed to be pointing to items in the new slice; just the old slice.

Re: In Go, pointers (mostly) don't go with slices in practice

#73

Post by sly010 »

masklinn wrote: post #71
sly010 wrote: post #49

Earlier quoted context omitted.

> it does wrong thing without warning It's not without warning, it's a well documented behavior. Just think of slices as "immutable", "pass-by-value" data structures (with a relatively efficient implementation) and everything falls into place. Mutating them in any way is actually a special case that you do only for performance reason (i.e. you can pre-allocate and fill if you know the size ahead of time) but - as alw…

> It's not without warning, it's a well documented behavior. Ah yes, the usual excuse for it being fine that C APIs are completely broken and half of them can not be used correctly. > Just think of slices as "immutable", "pass-by-value" data structures (with a relatively efficient implementation) and everything falls into place. Except when they don't because `append` itself amortises allocations, which means if you…

Ok, I think I just understood the problem. The following would cause a problem:

   s0 := append([]string{}, "zoo")
   sa := append(s0, "foo")
   sb := append(s0, "bar") // overwrites sa[1]
Go is truly unique in this sense, and you could not actually treat go slices as immutable structures. Point taken.

Re: In Go, pointers (mostly) don't go with slices in practice

#74

Post by masklinn »

hsn915 wrote: post #72
friendzis wrote: post #10

Earlier quoted context omitted.

> This is safe, but confuses some people. Rust has created a weird perception that memory safety equals safety. Language is a tool and it should work with me: it is extremely important that my understanding of what the program should do aligns with what it actually does. The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container". Taking a pointer/reference…

Go tells you very explicitly how it resizes slices by forcing you to write this: slice = append(slice, item) By merely typing this all the time when you add elements, you intuitively understand that appending can potentially re-allocate the slice data in a totally different location, so any pointers you have taken before the resize are not guaranteed to be pointing to items in the new slice; just the old slice.

> Go tells you very explicitly how it resizes slices by forcing you to write this:

No, what most readers intuit from that is that `append` performs no mutation and that this is fine:

    s2 := append(s1, item)
because it looks very much like, say,

    (def s2 (conj s1 item))
and often it will look like it works, especially at the smaller sizes, or if you never modify (or even use) s1.

Except it's absolutely not fine.

That's why other language separate slices and vectors and avoid confusing two objects which have different behaviours and uses even if their representation is very similar.

Re: In Go, pointers (mostly) don't go with slices in practice

#75

Post by masklinn »

kubb wrote: post #45

Considering the confusion of the author, it seems like not all junior programmers can understand Go, which makes me wonder: is it simple enough? One pitfall is when getting a slice by value in a function. You cannot be sure that someone is not going to pass you a slice into a buffer that they themselves use, so you have to be careful when appending - someone might be using that buffer and you’ll be writing over it.

> which makes me wonder: is it simple enough?

Obviously not given Go was never simple in the first place. Go was built to be easy — for a certain value of easy.

Simple tools are often not easy, and simple programming languages are definitely not easy: they tend to be built out of a small set of very powerful concepts which are directly exposed to the language user, said language user has close to the power of the language designer in building abstractions. Lisps, and Smalltalks and Forths are simple, which means they are mind-bending and not only can you build what you want out of them (hello turing equivalence) you can build how you want.

And of course the simplest of languages (the turing tarpits) are barely usable at all.

Re: In Go, pointers (mostly) don't go with slices in practice

#76

Post by masklinn »

Joker_vD wrote: post #35
simiones wrote: post #24

Earlier quoted context omitted.

There are no such things as "reference types" in Go, though slices do have the extremely odd behavior that they can take the value `nil`, similarly to interfaces and unlike any other non-pointer type in Go. Pointers to "fat pointer" types are sometimes needed, just like you sometimes need pointers-to-pointers.

What about maps? Non-nil maps sure seem like they're "reference types", look at [0]. [0] https://play.golang.org/p/xtY_ASExQzR

"reference types" is a very specific concept from a specific category of languages: types which are always heap-allocated and sitting behind an invisible (and un-interactible) pointer.

But Go doesn't have that distinction, and has actual pointers you can use directly. A map is just a heap-allocated structure sitting behind a pointer.

If you create a type which is a pointer to a struct, sure you can say you've built a reference type if you want, but that doesn't actually say much to anyone, because that's not a distinction the language makes, unlike Java or C#.

Re: In Go, pointers (mostly) don't go with slices in practice

#77

Post by anderskaseorg »

feffe wrote: post #32
friendzis wrote: post #10

Earlier quoted context omitted.

> This is safe, but confuses some people. Rust has created a weird perception that memory safety equals safety. Language is a tool and it should work with me: it is extremely important that my understanding of what the program should do aligns with what it actually does. The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container". Taking a pointer/reference…

This is how I think about Go slices (may help other understand them). A slice itself is just a window into a backing array of fixed size. The slice carries three data members. The pointer to the backing array and its remaining capacity and the length of the slice data. Typically slices are passed around by value but you can take their address and modify a "shared" slice. The built-in append() returns a new slice by v…

If it were true in the typical sense that “append() returns a new slice by value”, then you would expect to be able to mutate the old slice and the new slice independently from each other. But in reality, you can only do this if append() decided to reallocate, which only happens at some implementation-defined exponential pattern of sizes.

    package main
    
    import "fmt"
    
    func main() {
            a := []int{0}
            for i := 0; i
→ 0 0 1 0 1 1 1 0 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1

Re: In Go, pointers (mostly) don't go with slices in practice

#78

Post by klodolph »

kevincox wrote: post #68
klodolph wrote: post #60

> Honestly, this is a strange and peculiar situation, although Go programmers have acclimatized to it. To programmers from other languages, such as C or C++, the concept of pointers to dynamically extensible arrays seems like a perfectly decent idea that surely should exist and work in Go. Well, it exists, and it "works" in the sense that it yields results and doesn't crash your program, but it doesn't "work" in the…

I agree that it is better than C and C++ but not much better. C and C++ are "It is very easy to make this mistake which leads to undefined behaviour and a maybe incorrect program." Go is "It is very easy to make this mistake which leads to a maybe incorrect program." Yes, better! But the problem is still there. I much prefer the Rust solution where there is no common mistake.

The catch is that code is hard to translate into Rust.

"I have this code, you see... and it takes a couple mutable references into an array... how do I translate this into Rust?"

There is no one-size-fits-all answer to that question. The code may be correct in C or C++, but the Rust type system may give you one hell of a hard time proving that it is correct to the Rust type system's satisfaction... so you refactor your code completely, or you use integer indexes into arrays rather than references, or you use unsafe code...

I've written some amount of Rust code at this point. About half of the time, when I write a project in Rust, there comes a point at which I'm fighting with the type system. I feel like this should stop happening, at some point.

Re: In Go, pointers (mostly) don't go with slices in practice

#79

Post by kevincox »

klodolph wrote: post #78
kevincox wrote: post #68

Earlier quoted context omitted.

I agree that it is better than C and C++ but not much better. C and C++ are "It is very easy to make this mistake which leads to undefined behaviour and a maybe incorrect program." Go is "It is very easy to make this mistake which leads to a maybe incorrect program." Yes, better! But the problem is still there. I much prefer the Rust solution where there is no common mistake.

The catch is that code is hard to translate into Rust. "I have this code, you see... and it takes a couple mutable references into an array... how do I translate this into Rust?" There is no one-size-fits-all answer to that question. The code may be correct in C or C++, but the Rust type system may give you one hell of a hard time proving that it is correct to the Rust type system's satisfaction... so you refactor yo…

For sure. The solution has downsides but it does largely solve the problem. It isn't strictly better than go, but it is an important difference.

Mostly off-topic but FWIW "mutable reference into an array" is typically very easy in Rust you just accept a &mut [T]. This also nicely enforces that you don't try to append because that would be wrong 99% of the time.

Re: In Go, pointers (mostly) don't go with slices in practice

#80

Post by klodolph »

kevincox wrote: post #79
klodolph wrote: post #78

Earlier quoted context omitted.

The catch is that code is hard to translate into Rust. "I have this code, you see... and it takes a couple mutable references into an array... how do I translate this into Rust?" There is no one-size-fits-all answer to that question. The code may be correct in C or C++, but the Rust type system may give you one hell of a hard time proving that it is correct to the Rust type system's satisfaction... so you refactor yo…

For sure. The solution has downsides but it does largely solve the problem. It isn't strictly better than go, but it is an important difference. Mostly off-topic but FWIW "mutable reference into an array" is typically very easy in Rust you just accept a &mut [T]. This also nicely enforces that you don't try to append because that would be wrong 99% of the time.

> Mostly off-topic but FWIW "mutable reference into an array" is typically very easy in Rust you just accept a &mut [T].

I had specifically worded it as "a couple mutable references into an array". How do I take a small number of references into an array, say two or three?

Post reply on HN