Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In Go, pointers (mostly) don't go with slices in practice

utcc.utoronto.ca

Post reply on HN
Print view View original

21–30 of 98 posts

Re: In Go, pointers (mostly) don't go with slices in practice

#21

Post by pjmlp »

edflsafoiewq wrote: post #13
Thorrez wrote: post #6

>Honestly, this is a strange and peculiar situation, although Go programmers have acclimatized to it. To programmers from other languages, such as C or C++, the concept of pointers to dynamically extensible arrays seems like a perfectly decent idea that surely should exist and work in Go. Well, it exists, and it "works" in the sense that it yields results and doesn't crash your program, but it doesn't "work" in the s…

C++ has std::vector, which is one level of abstraction above a slice; you push to a vector, and maybe the backing slice changes, but it's still the same vector. Go is unusual in not having a equivalent of vector.

Not really, if reallocation takes place.

So if you got a pointer to a vector element, it now points to garbage.

Re: In Go, pointers (mostly) don't go with slices in practice

#22

Post by heleninboodler »

friendzis wrote: post #10
Animats wrote: post #3

Well, of course slices work that way. Think about what happens if you have a reference to a slice in an array and you shrank the array to 0. You've just created a dangling pointer. In Go, you get a stable version of the old data, and the garbage collector tracks that you still have a reference to it. This is safe, but confuses some people. In Rust, the borrow checker won't let you modify the array while you have a re…

> This is safe, but confuses some people. Rust has created a weird perception that memory safety equals safety. Language is a tool and it should work with me: it is extremely important that my understanding of what the program should do aligns with what it actually does. The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container". Taking a pointer/reference…

> The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container"

No, there is no mention of a "snapshot". You get a reference to the current backing array, which may or may not continue being used by the slice (depending on reallocations). You're pointing to the live slice backing array, and the values in it may change if someone else is manipulating the slice, up to the point where the slice backing array must be reallocated, at which point you'll continue pointing to the old backing array and be keeping it from getting GC'd.

Re: In Go, pointers (mostly) don't go with slices in practice

#23

Post by simiones »

pjmlp wrote: post #21
edflsafoiewq wrote: post #13

Earlier quoted context omitted.

C++ has std::vector, which is one level of abstraction above a slice; you push to a vector, and maybe the backing slice changes, but it's still the same vector. Go is unusual in not having a equivalent of vector.

Not really, if reallocation takes place. So if you got a pointer to a vector element, it now points to garbage.

I guess the difference is this:

C++:

  std::vector v {1, 2, 3};
  void foo(std::vector *ref) {
    ref.push_back(4); 
  }
  foo(&v);
  //v[3] == 4 is true here
Go:

  v := []int{1, 2, 3}
  func foo(ref *[]int) {
    append(ref, 4)
  }
  foo(&v)
  //v[3] == 4 may or may not be true here.
Pointers to elements in the vector do indeed have the same problems both in Go and C++ (except for memory safety).

Re: In Go, pointers (mostly) don't go with slices in practice

#24

Post by simiones »

vanderZwan wrote: post #12

I get the confusion (to people unfamiliar with pointers) about pointers to elements in reference types, but why would anyone want pointers to reference types? They're basically pointers with extra features

There are no such things as "reference types" in Go, though slices do have the extremely odd behavior that they can take the value `nil`, similarly to interfaces and unlike any other non-pointer type in Go.

Pointers to "fat pointer" types are sometimes needed, just like you sometimes need pointers-to-pointers.

Re: In Go, pointers (mostly) don't go with slices in practice

#25

Post by db48x »

heleninboodler wrote: post #22
friendzis wrote: post #10

Earlier quoted context omitted.

> This is safe, but confuses some people. Rust has created a weird perception that memory safety equals safety. Language is a tool and it should work with me: it is extremely important that my understanding of what the program should do aligns with what it actually does. The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container". Taking a pointer/reference…

> The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container" No, there is no mention of a "snapshot". You get a reference to the current backing array, which may or may not continue being used by the slice (depending on reallocations). You're pointing to the live slice backing array, and the values in it may change if someone else is manipulating the slice…

And that’s really the problem with it. If you want to ensure that you have exclusive access to the element(s), then you have to explicitly copy them first or you get silent data corruption.

And if you want to ensure that multiple things have access to the elements, then you have to avoid reallocations or you get silent data loss.

No matter what you’re doing, a pointer to an element of an array or slice is usually the wrong thing in Go. The language would be better off without them.

Re: In Go, pointers (mostly) don't go with slices in practice

#26

Post by pjmlp »

simiones wrote: post #23
pjmlp wrote: post #21

Earlier quoted context omitted.

Not really, if reallocation takes place. So if you got a pointer to a vector element, it now points to garbage.

I guess the difference is this: C++: std::vector v {1, 2, 3}; void foo(std::vector *ref) { ref.push_back(4); } foo(&v); //v[3] == 4 is true here Go: v := []int{1, 2, 3} func foo(ref *[]int) { append(ref, 4) } foo(&v) //v[3] == 4 may or may not be true here. Pointers to elements in the vector do indeed have the same problems both in Go and C++ (except for memory safety).

Ah ok, although for the audience not versed in C++ the correct code is,

  void foo(std::vector *ref) {
    ref->push_back(4); 
  }
  foo(&v);
or

  void foo(std::vector &ref) {
    ref.push_back(4); 
  }
  foo(v);

Re: In Go, pointers (mostly) don't go with slices in practice

#27

Post by masklinn »

tsimionescu wrote: post #16
friendzis wrote: post #10

Earlier quoted context omitted.

> This is safe, but confuses some people. Rust has created a weird perception that memory safety equals safety. Language is a tool and it should work with me: it is extremely important that my understanding of what the program should do aligns with what it actually does. The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container". Taking a pointer/reference…

The problem here is identical to the problem of pointers to array elements in C after a 'realloc', except that Go at least guarantees that you're not going to modify some other object's memory. Of course, since append neither guarantees nor prevents a copy, the semantics of modifying a value through a pointer to a slice element after an append are unspecified, so it is not a useful construct.

> except that Go at least guarantees that you're not going to modify some other object's memory.

Or that you’re way off in UB (UAF) land.

Re: In Go, pointers (mostly) don't go with slices in practice

#28

Post by watt »

drran wrote: post #20
watt wrote: post #15

Earlier quoted context omitted.

the line `s = append(s, 50)` redefines what "s" actually is. And after this line `ps` points to some previous version of what "s" used to be.

No, ps still points to s: https://play.golang.org/p/iSjoqGTg20_O var s []int s = append(s, 10, 20, 30) pe := &s[0] ps := &s s = append(s, 50) s[0] = 100 pe2 := &s[0] fmt.Println("s: ", s, ", ps: ", ps, ", pe: ", pe, ", pe2: ", pe2) // s: [100 20 30 50] , ps: &[100 20 30 50] , pe: 0xc0000be000 , pe2: 0xc0000b8030

I don't get why address of slice returned from "append" does not change. Maybe in a trivial program like this the backing array can always be extended in-place, because there in memory fragmentation.

Is that still true in an app that has considerable memory pressure and has GC running now and then?

Re: In Go, pointers (mostly) don't go with slices in practice

#29

Post by masklinn »

vanderZwan wrote: post #12

I get the confusion (to people unfamiliar with pointers) about pointers to elements in reference types, but why would anyone want pointers to reference types? They're basically pointers with extra features

Go doesn’t have reference types, and slices certainly do not behave like them: because the slice is composed of the pointer, length, and capacity (on the stack) if you want to modify a slice on behalf of a caller you absolutely must get a pointer to a slice, unless your mutations consist solely of setting existing indices.

Go’s map is (AFAIK) a pointer to the hmap stucture where everything happens, so it does behave like a reference type (all updates in the callee will be visible to the caller) but even then it can be useful to have a pointer to one so you can reset it without mutating it in-place. While that can be a bit weirder, it is also much safer. Especially given Go’s maps are not thread-safe (and in fact not memory-safe under concurrent updates).

Re: In Go, pointers (mostly) don't go with slices in practice

#30

Post by tialaramex »

simiones wrote: post #23
pjmlp wrote: post #21

Earlier quoted context omitted.

Not really, if reallocation takes place. So if you got a pointer to a vector element, it now points to garbage.

I guess the difference is this: C++: std::vector v {1, 2, 3}; void foo(std::vector *ref) { ref.push_back(4); } foo(&v); //v[3] == 4 is true here Go: v := []int{1, 2, 3} func foo(ref *[]int) { append(ref, 4) } foo(&v) //v[3] == 4 may or may not be true here. Pointers to elements in the vector do indeed have the same problems both in Go and C++ (except for memory safety).

> //v[3] == 4 is true here

Note that C++ std::vector has an operator overload so this is actually just calling the method named operator[] on the object v and that method returns you a reference to the object in the backing array if in fact it is a suitable size (no checks are made). In particular if foo doesn't for any reason extend the vector then our program now has Undefined Behaviour.

This is not merely the C syntactic sugar array subscript operation v[3] == *(v+3) as the vector is not necessarily just a backing array pointer and some magic.

In contrast Go is really offering array syntax for this slice, that's the built-in array subscript operation and it cares whether v[3] exists when you try to compare it to 4.

Post reply on HN