Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

In Go, pointers (mostly) don't go with slices in practice

utcc.utoronto.ca

Post reply on HN
Print view View original

31–40 of 98 posts

Re: In Go, pointers (mostly) don't go with slices in practice

#31

Post by Zababa »

friendzis wrote: post #10
Animats wrote: post #3

Well, of course slices work that way. Think about what happens if you have a reference to a slice in an array and you shrank the array to 0. You've just created a dangling pointer. In Go, you get a stable version of the old data, and the garbage collector tracks that you still have a reference to it. This is safe, but confuses some people. In Rust, the borrow checker won't let you modify the array while you have a re…

> This is safe, but confuses some people. Rust has created a weird perception that memory safety equals safety. Language is a tool and it should work with me: it is extremely important that my understanding of what the program should do aligns with what it actually does. The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container". Taking a pointer/reference…

> Rust has created a weird perception that memory safety equals safety.

I think it's a bit more than that. They're also riding on the static typing trend that's happening right now, so type-safety is also part of the equation. From the website:

> A language empowering everyone to build reliable and efficient software.

> Reliability: Rust’s rich type system and ownership model guarantee memory-safety and thread-safety — enabling you to eliminate many classes of bugs at compile-time.

That means that people like me that still have a hard time with C and C++ can build efficient software using the same workflow as I'm used to in my usual "web languages" (Python and JS mostly).

Re: In Go, pointers (mostly) don't go with slices in practice

#32

Post by feffe »

friendzis wrote: post #10
Animats wrote: post #3

Well, of course slices work that way. Think about what happens if you have a reference to a slice in an array and you shrank the array to 0. You've just created a dangling pointer. In Go, you get a stable version of the old data, and the garbage collector tracks that you still have a reference to it. This is safe, but confuses some people. In Rust, the borrow checker won't let you modify the array while you have a re…

> This is safe, but confuses some people. Rust has created a weird perception that memory safety equals safety. Language is a tool and it should work with me: it is extremely important that my understanding of what the program should do aligns with what it actually does. The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container". Taking a pointer/reference…

This is how I think about Go slices (may help other understand them).

A slice itself is just a window into a backing array of fixed size. The slice carries three data members. The pointer to the backing array and its remaining capacity and the length of the slice data.

Typically slices are passed around by value but you can take their address and modify a "shared" slice.

The built-in append() returns a new slice by value.

What happens is simply that when appending data to a slice and there is no room in the backing array, a new backing array is allocated that the returned slice points into. The old "input" slice to append is still intact and if some code has access to it, it will look at data stored in the old backing array.

I've constructed similar utility types in C and find them quite convenient. It's very convenient to have the distinction between the backing memory (array) and a slice viewing a portion of it instead of just a dynamic array.

Re: In Go, pointers (mostly) don't go with slices in practice

#33

Post by pitkali »

watt wrote: post #28
drran wrote: post #20

Earlier quoted context omitted.

No, ps still points to s: https://play.golang.org/p/iSjoqGTg20_O var s []int s = append(s, 10, 20, 30) pe := &s[0] ps := &s s = append(s, 50) s[0] = 100 pe2 := &s[0] fmt.Println("s: ", s, ", ps: ", ps, ", pe: ", pe, ", pe2: ", pe2) // s: [100 20 30 50] , ps: &[100 20 30 50] , pe: 0xc0000be000 , pe2: 0xc0000b8030

I don't get why address of slice returned from "append" does not change. Maybe in a trivial program like this the backing array can always be extended in-place, because there in memory fragmentation. Is that still true in an app that has considerable memory pressure and has GC running now and then?

Even if slice is reallocated, the information about new reallocated slice is still stored in variable s. ps is merely pointing to that variable. The fact that the contents of the variable changed does not mean that its location has to change.

In other words, ps is a pointer to a pointer to array data. Append may change the inner pointer's value but that's about it.

Re: In Go, pointers (mostly) don't go with slices in practice

#34

Post by codeflo »

feffe wrote: post #32
friendzis wrote: post #10

Earlier quoted context omitted.

> This is safe, but confuses some people. Rust has created a weird perception that memory safety equals safety. Language is a tool and it should work with me: it is extremely important that my understanding of what the program should do aligns with what it actually does. The way you describe go's behavior is "takes snapshot of the underlying data", which usually means "deep copy container". Taking a pointer/reference…

This is how I think about Go slices (may help other understand them). A slice itself is just a window into a backing array of fixed size. The slice carries three data members. The pointer to the backing array and its remaining capacity and the length of the slice data. Typically slices are passed around by value but you can take their address and modify a "shared" slice. The built-in append() returns a new slice by v…

Does that mean that a) the pointer to the single element is only invalidated on “append” if the slice has no more capacity (this is how C++ vectors work), or b) does the fact that there’s an active reference into the slice always cause a reallocation (copy-on-write style)? If it’s the former, we’re literally in the C++ iterator invalidation nightmare, only without the debugging tools.

Re: In Go, pointers (mostly) don't go with slices in practice

#35

Post by Joker_vD »

simiones wrote: post #24
vanderZwan wrote: post #12

I get the confusion (to people unfamiliar with pointers) about pointers to elements in reference types, but why would anyone want pointers to reference types? They're basically pointers with extra features

There are no such things as "reference types" in Go, though slices do have the extremely odd behavior that they can take the value `nil`, similarly to interfaces and unlike any other non-pointer type in Go. Pointers to "fat pointer" types are sometimes needed, just like you sometimes need pointers-to-pointers.

What about maps? Non-nil maps sure seem like they're "reference types", look at [0].

[0] https://play.golang.org/p/xtY_ASExQzR

Re: In Go, pointers (mostly) don't go with slices in practice

#36

Post by ratorx »

masklinn wrote: post #29
vanderZwan wrote: post #12

I get the confusion (to people unfamiliar with pointers) about pointers to elements in reference types, but why would anyone want pointers to reference types? They're basically pointers with extra features

Go doesn’t have reference types, and slices certainly do not behave like them: because the slice is composed of the pointer, length, and capacity (on the stack) if you want to modify a slice on behalf of a caller you absolutely must get a pointer to a slice, unless your mutations consist solely of setting existing indices. Go’s map is (AFAIK) a pointer to the hmap stucture where everything happens, so it does behave…

Isn’t passing in pointers to slices considered an anti-pattern? I thought the go way would be to take the slice as argument by value and return possibly a different slice, like append does. I don’t think I’ve seen any (idiomatic) code that took slices by pointer.

Re: In Go, pointers (mostly) don't go with slices in practice

#37

Post by simiones »

pjmlp wrote: post #26
simiones wrote: post #23

Earlier quoted context omitted.

I guess the difference is this: C++: std::vector v {1, 2, 3}; void foo(std::vector *ref) { ref.push_back(4); } foo(&v); //v[3] == 4 is true here Go: v := []int{1, 2, 3} func foo(ref *[]int) { append(ref, 4) } foo(&v) //v[3] == 4 may or may not be true here. Pointers to elements in the vector do indeed have the same problems both in Go and C++ (except for memory safety).

Ah ok, although for the audience not versed in C++ the correct code is, void foo(std::vector *ref) { ref->push_back(4); } foo(&v); or void foo(std::vector &ref) { ref.push_back(4); } foo(v);

Oops, right, bit rusty on C++...

Re: In Go, pointers (mostly) don't go with slices in practice

#38

Post by simiones »

tialaramex wrote: post #30
simiones wrote: post #23

Earlier quoted context omitted.

I guess the difference is this: C++: std::vector v {1, 2, 3}; void foo(std::vector *ref) { ref.push_back(4); } foo(&v); //v[3] == 4 is true here Go: v := []int{1, 2, 3} func foo(ref *[]int) { append(ref, 4) } foo(&v) //v[3] == 4 may or may not be true here. Pointers to elements in the vector do indeed have the same problems both in Go and C++ (except for memory safety).

> //v[3] == 4 is true here Note that C++ std::vector has an operator overload so this is actually just calling the method named operator[] on the object v and that method returns you a reference to the object in the backing array if in fact it is a suitable size (no checks are made). In particular if foo doesn't for any reason extend the vector then our program now has Undefined Behaviour. This is not merely the C sy…

I'm not sure I understand how this is relevant. In C++, arr[non_existent_index] is UB both if arr is an std::vector and if it is a C-style array. In Go it's a runtime error instead.

Sure, std::vector::operator[]() is not just syntax sugar for *(v+i), but I don't think any of this is relevant for the discussion at hand, unless I'm missing something.

Re: In Go, pointers (mostly) don't go with slices in practice

#39

Post by ubercow13 »

watt wrote: post #28
drran wrote: post #20

Earlier quoted context omitted.

No, ps still points to s: https://play.golang.org/p/iSjoqGTg20_O var s []int s = append(s, 10, 20, 30) pe := &s[0] ps := &s s = append(s, 50) s[0] = 100 pe2 := &s[0] fmt.Println("s: ", s, ", ps: ", ps, ", pe: ", pe, ", pe2: ", pe2) // s: [100 20 30 50] , ps: &[100 20 30 50] , pe: 0xc0000be000 , pe2: 0xc0000b8030

I don't get why address of slice returned from "append" does not change. Maybe in a trivial program like this the backing array can always be extended in-place, because there in memory fragmentation. Is that still true in an app that has considerable memory pressure and has GC running now and then?

s changes but &s doesn't: https://play.golang.org/p/Xs1SYXqEl9i

Re: In Go, pointers (mostly) don't go with slices in practice

#40

Post by unrealhoang »

watt wrote: post #28
drran wrote: post #20

Earlier quoted context omitted.

No, ps still points to s: https://play.golang.org/p/iSjoqGTg20_O var s []int s = append(s, 10, 20, 30) pe := &s[0] ps := &s s = append(s, 50) s[0] = 100 pe2 := &s[0] fmt.Println("s: ", s, ", ps: ", ps, ", pe: ", pe, ", pe2: ", pe2) // s: [100 20 30 50] , ps: &[100 20 30 50] , pe: 0xc0000be000 , pe2: 0xc0000b8030

I don't get why address of slice returned from "append" does not change. Maybe in a trivial program like this the backing array can always be extended in-place, because there in memory fragmentation. Is that still true in an app that has considerable memory pressure and has GC running now and then?

Because there’s still space left in that slice (capacity > len), and the strategy of pre allocate capacity is to double the current (1,2,4,8,…), in this case 3 elements added => that slice was having a capacity of 4.
Post reply on HN