Live data from Hacker News

Go structs are copied on assignment (and other things about Go I'd missed)

jvns.ca

111–120 of 176 posts

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#111
post #51

Earlier quoted context omitted.

uh I'd not say it like that Python passes primitive types by value, out rather "as if by value", because it copies them on write. if you modify your experiment to pass around a dict or list and modify that in the 'y', you'll see y is happily modified. so Python passes by reference, however it either blocks updates (tuple) or copies on write (int, str, float) or updates in place (dict, list, class)

> if you modify your experiment to pass around a dict or list and modify that in the 'y', you'll see y is happily modified. No, you won't. x = {'a' : 1} foo(x) print(x) def foo(z): z = {'b' : 2} You'll see that this prints `{'a' : 1}`, not `{'b' : 2}`. Python always uses pass-by-value. It passes a copy of the pointer to a dict/list/etc in this case. Of course, if you modify the fields of the z variable, as in `z['b']…

Is it not pass-by-reference by some technicality? In the mutation example you suggest, if a reference to x isn't being passed into foo, how could foo modify x?

I would sooner believe the example is showing you shadowing the z argument to foo, than foo being able to modify the in-parameter sometimes even if it's pass by value.

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#112

func findThing(things []Thing, name string) *Thing { for i := range things { if things[i].Name == name { return &things[i] } } return nil } Also you could just return i or -1, and the consuming code would be clear about what it was doing. Find the index. Update the item at the index. if location := findThing(things, name); location != -1 { things[location].Name = "updated" }

Well, if you don't mind that it doesn't work correctly with slices: [0], then sure, you may return indices.

[0] https://go.dev/play/p/Q2ntuaugbGQ

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#114
post #94

Earlier quoted context omitted.

My pet peeve with slices and maps is that they hide a reference to the actual structure, and you are never sure of what you are modifying, or the performance impact when moving around big structures. Example with slices: https://go.dev/play/p/8arcUrGU4SU Example with maps: https://go.dev/play/p/eq8i6z8a4jN

No, there is no "hiding" of a "reference". There is copying a struct: s2 := s1 This copies the struct in `s1` into a new name `s2`. This struct contains, among other things, a pointer to the backing array. Therefore, when you assign to the slice s2[0] = "bye" You assign to the same backing array. Slices are not arrays. Copying a slice copies a struct containing a pointer to an array. A similar situation holds true fo…

Since a slice/map, internally, contains a pointer to the data, it looks like slices/maps have reference semantics: after you do "m2 := m1", all changes done through m1 are visible through m2, even though the type of m1 and m2 has no visible asterisk anywhere in it.

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#115

Earlier quoted context omitted.

Non-C# developer question: what use-case/situation would a `struct` make sense to use instead of a `class`? Just out of curiosity. [Edit] Well, there's a nice, special article for this very question: https://learn.microsoft.com/en-us/dotnet/standard/design-gui...

In principle, the answer should be (if we ignored the language community and the model of the stdlib, which we shouldn't do), that structs should be used for most things, and classes only when they are needed. There's nothing a class can do that a struct can't, and structs are not automatically allocated in the heap, so they take some pressure off the GC. Go showed that you can have a fully managed GC language where…

There is no free lunch in software :)

> using the syntax of C pointers (but without pointer arithmetic)

Go structs, when they have their address taken, act the same way object references do in C# or Java (that is, *MyStruct). Taking an address of a struct in Go and assigning it to a location likely turns it into a heap allocation. Even when not, it is important to understand what a stack in Go is and how it is implemented:

Go makes a different set of tradeoffs by using a virtual/managed stack. When more memory on stack is required - the request goes through an allocator and the new memory range is attached to a previous one in a linked-list fashion (if this has changed in recent versions - please correct me). This is rather effective but notably can suffer from locality issues where, for example, two adjacently accessed structs in a hot loop are placed in different memory locations. Modern CPUs can deal with such non-locality of data well, but it is a concern still.

It also, in a way, makes it somewhat similar to the role Gen0 / Nursery GC heaps play in .NET and OpenJDK GC implementations - heap allocations just bump a pointer within thread-local allocation context, and if a specific heap has ran out of free memory - more memory is requested from GC (or a collection is triggered, or both). By nature of both having generational heaps and moving garbage collector, the data in the heap has high locality and consecutively allocated objects are placed in a linear fashion. One of the main drawbacks of such approach is much higher implementation complexity.

Additionally, the Go choice comes at a tradeoff of making FFI (comparatively) expensive - you pay about 0.5-2ns for a call across interop into C in .NET (when statically linked, it's just a direct call + branch), where-as the price of FFI in GoGC is around 50ns (and also interacts worse with the executor if you block goroutines).

Overall, the design behind memory management in Go is interesting and makes a lot of sense for the scenarios Go was intended for (lightweight networked microservices, CLI tooling, background daemons, etc.).

However, it trades off smaller memory footprint for a significantly lower allocation throughput. Because GC in Go is, at least partially, write-barrier driven, it makes WBs much more expensive - something you pay for when you assign a Go struct pointer to a heap or not provably local location (I don't know if Go performs WB elision/cheap WB selection the way OpenJDK and .NET do).

To explore this further, I put together a small demonstration[0] based on BenchmarksGame's BinaryTrees suite which stresses this exact scenario. Both Go and C# there can be additionally inspected with e.g. dtrace on macOS or most other native profilers like Samply[1].

> Now, in working with real C# code and real C# programmers, all this is false...

> but if you want to store it in a field, you need to use some class type as a box,

This does not correspond to language specification or the kind of code that is being written outside of enterprise.

First of all, `ref T` syntax in C# is a much more powerful concept than a simple reference to a local variable. `ref T` in .NET is a 'byref' aka managed pointer. Byrefs can point to arbitrary memory and are GC-aware. This means that they can point to stack, unmanged memory (you can even mmap it directly) GC on NonGC heap object interiors, etc. If they point to GC heap object interiors, they are appropriately updated by GC when it moves the objects, and are ignored when they are not. They cannot be stored on heap, which does cause some tension, but if you have a pointer-rich deep data structure, then classes are a better choice almost every time.

Byrefs can be held by ref structs and the most common example used everywhere today is Span - internally it is (ref T _reference, int _length) and is used for interacting with and slicing of arbitrary contiguous memory. You can find additional details on low-level memory management techniques here: https://news.ycombinator.com/item?id=40963672

Byrefs, alongside regular C pointers in C#, support pointer arithmetics as well. Of course it is just as unsafe, but for targeted hot paths this is indispensable. You can use it for fancy things like vectorized byte pair count within a sequence without having to pin the memory: https://github.com/U8String/U8String/blob/split-refactor/Sou...

Last but not least, the average line-of-business code indeed rarely uses structs - in the past it was mostly classes, today it is a mix of classes, records, and sometimes record structs for single-field wrappers (still rarely). However, C# is a multi-paradigm language with strong low-level capabilities and there is a sea of projects beyond enterprise that make use of all the new and old low-level features. It's something that both C# and .NET were designed in mind from the very beginning. In this regard, they brings you much closer to the metal than Go unless significant changes are introduced to it.

If you're interested, feel free to explore Ryujinx[2] and Garnet[3] which are more recent examples of projects demonstrating suitability of C# in domains historically reserved for C, C++ and Rust.

[0]: https://gist.github.com/neon-sunset/72e6aa57c6a4c5eb0e2711e1...

[1]: https://github.com/mstange/samply

[2]: https://github.com/search?q=repo%3ARyujinx%2FRyujinx+ref+str...

[3]: https://github.com/search?q=repo%3Amicrosoft%2Fgarnet+struct...*

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#116
I always get surprised in the opposite direction by languages that work like that, fun to see it from the other side.

As for the second mistake listed, this is practically the reverse confusion itself... I remember one time in an interview, I got this bit of arcana about Go slices right and the interviewer insisted it was wrong, and despite the evidence being on the screen in the program output at the time, I just backed down. Not sure why I or anyone ever submits to the indignity of job interviews, but it also soured me on Go itself a bit!

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#117

func findThing(things []Thing, name string) *Thing { for i := range things { if things[i].Name == name { return &things[i] } } return nil } Also you could just return i or -1, and the consuming code would be clear about what it was doing. Find the index. Update the item at the index. if location := findThing(things, name); location != -1 { things[location].Name = "updated" }

Well, if you don't mind that it doesn't work correctly with slices: [0], then sure, you may return indices. [0] https://go.dev/play/p/Q2ntuaugbGQ

It does work with slices as long as you use the slice to update as well

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#118
post #78

Earlier quoted context omitted.

> There's a good argument for immutability by default, but many programmers dislike all the extra declarations required. That's one little reason why Rust is loved by many: immutability by default. Meanwhile, it's not even possible in Go to declare immutable variables!

Immutable by default is only really possible now that we have gobs of memory though. I'm not even sure it's likely to stay popular: the demands of data processing at scale mean we're all likely to be routinely handling gigantic datasets which we don't want to copy all over the place. The real problem is just visibility: am I editing a copy of the original? Who else can edit the original? Who's going to? I'd argue tho…

LLVM will transform your mutable program into immutable one anyway because otherwise it's much harder if not impossible to write a lot of optimizations or validations.

You can collapse a lot of the additional copies at compile time even for C

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#119

Earlier quoted context omitted.

No, there is no "hiding" of a "reference". There is copying a struct: s2 := s1 This copies the struct in `s1` into a new name `s2`. This struct contains, among other things, a pointer to the backing array. Therefore, when you assign to the slice s2[0] = "bye" You assign to the same backing array. Slices are not arrays. Copying a slice copies a struct containing a pointer to an array. A similar situation holds true fo…

Since a slice/map, internally, contains a pointer to the data, it looks like slices/maps have reference semantics: after you do "m2 := m1", all changes done through m1 are visible through m2, even though the type of m1 and m2 has no visible asterisk anywhere in it.

> , it looks like slices/maps have reference semantics

No, they don't. Pointers and references are fundamentally different concepts. A reference is a name, handled by the runtime, that is bound to an entity. A pointer is just a value of type `uintptr`.

When I "copy" a reference, I simply instruct the runtime to bind another name to the entity.

When I copy a struct containing a pointer, I actually allocate new memory to contain a new copy of that `uintptr`. And since that copy is a true copy of a pointer-value, I can change it.

That's why this:

    func main() {
        s1 := []int{1, 2}
        s2 := s1
        s2 = append(s2, 3)
        s1[0] = 42
        fmt.Println(s1)
        fmt.Println(s2)
    }
Will give you

    [42 2]
    [1 2 3]
as an output. s2 is not a "Reference" to the same entity as s1, it is a struct holding a pointer, and when we grow the slice this struct represents beyond the capacity of the backing array that pointer points to, by appending to it, we replace that pointer in s2.

Comparing that to a language that actually does have reference semantics (python):

    s1 = [1, 2]
    s2 = s1
    s2.append(3)
    s1[0] = 42
    print(s1)
    print(s2)
Gives me

    [42, 2, 3]
    [42, 2, 3]

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#120

Earlier quoted context omitted.

> There's a good argument for immutability by default, but many programmers dislike all the extra declarations required. That's one little reason why Rust is loved by many: immutability by default. Meanwhile, it's not even possible in Go to declare immutable variables!

I hope it gets added, it adds so much safety to the language. Mind you, Go is not a very safe language in general, its type system is pretty loose compared to e.g. Java or Typescript. I don't believe it wants to be though.

It is possible to add more immutability in Go. There are many proposals for this: https://github.com/go101/go101/wiki/Go-immutable-value-propo....

The main reason nothing happened in this direction is the core team think it is not important enough.

Post reply on HN