Go structs are copied on assignment (and other things about Go I'd missed) • Hacker News Board

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#121

Post by froh » Mon, Aug 12, 2024, 1:06 PM UTC

tsimionescu wrote: post #65

froh wrote: post #51

Earlier quoted context omitted.

uh I'd not say it like that Python passes primitive types by value, out rather "as if by value", because it copies them on write. if you modify your experiment to pass around a dict or list and modify that in the 'y', you'll see y is happily modified. so Python passes by reference, however it either blocks updates (tuple) or copies on write (int, str, float) or updates in place (dict, list, class)

> if you modify your experiment to pass around a dict or list and modify that in the 'y', you'll see y is happily modified. No, you won't. x = {'a' : 1} foo(x) print(x) def foo(z): z = {'b' : 2} You'll see that this prints `{'a' : 1}`, not `{'b' : 2}`. Python always uses pass-by-value. It passes a copy of the pointer to a dict/list/etc in this case. Of course, if you modify the fields of the z variable, as in `z['b']…

you replace the local binding z to the dict globally bound to x by a local dict in that z = ... assignment.

however if you do z['b'] = 2 in foo, then you'll see the global dict bound to x has been modified, as you have stated.

well, that's _exactly_ pass by reference.

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#122

Post by tialaramex » Mon, Aug 12, 2024, 1:30 PM UTC

foldr wrote: post #101

kzrdude wrote: post #99

Earlier quoted context omitted.

Well, that's another nice thing with Rust. Maybe it's not the saviour language but at least it brings clarity to ownership of values.

I think the same thing could happen in Rust. foo.x = bar will compile if foo is a copy or if it's a mutable reference. As in Go, you'd have to explicitly type 'foo' in order for any compiler error to show up.

Let's work through the example

1. There's a user-defined type named Thing with a String inside it.

2. We've got a function named findThing, which takes an array of Things and a string name, performs a linear search of the array and returns a mutable reference to the Thing with that name or if it's not found it doesn't. In Go, this function accidentally gives back a mutable reference to a copy of the Thing.

3. We make an array of things, use findThing to find one of them and try to mutate it.

We can translate 1 into Rust easily, it's just struct Thing { Name: String } and the compiler will moan because this is deeply unconventional naming in Rust and that warns by default.

We can translate 3 into Rust easily too, the compiler moans about our naming again.

But when we try to translate 2 by writing findThing we struggle. Rust wants to understand what lifetime to associate with this mutable reference we're returning. If we try to make a copy, where does the copy live? Rust doesn't have garbage collection, so if it just goes out of scope the lifetime ends and Rust will reject this function as nonsense - you can't return references which have expired, silly programmer, try again.

If we don't make a copy we can successfuly tie the lifetime to the array, but then we don't have the Go bug, so the thing you say "could happen in Rust" doesn't happen.

Here's a Godbolt link for a working example: https://rust.godbolt.org/z/shKj6rEz7

Now, try to adjust it to have the same bug, I expect it will be much harder to do this wrong.

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#123

Post by jerf » Mon, Aug 12, 2024, 2:01 PM UTC

dwattttt wrote: post #111

tsimionescu wrote: post #65

Earlier quoted context omitted.

> if you modify your experiment to pass around a dict or list and modify that in the 'y', you'll see y is happily modified. No, you won't. x = {'a' : 1} foo(x) print(x) def foo(z): z = {'b' : 2} You'll see that this prints `{'a' : 1}`, not `{'b' : 2}`. Python always uses pass-by-value. It passes a copy of the pointer to a dict/list/etc in this case. Of course, if you modify the fields of the z variable, as in `z['b']…

Is it not pass-by-reference by some technicality? In the mutation example you suggest, if a reference to x isn't being passed into foo, how could foo modify x? I would sooner believe the example is showing you shadowing the z argument to foo, than foo being able to modify the in-parameter sometimes even if it's pass by value.

"In the mutation example you suggest, if a reference to x isn't being passed into foo, how could foo modify x?"

Because it is passing a pointer by value under the hood.

This is the part that messes everyone up. Passing pointers by value is not what passing by reference used to mean.

And it matters, precisely because that is extremely realistic Python code that absolutely will mess you up if you don't understand exactly what is going on. You were passed a reference by value. If you go under the hood, you will find it is quite literally being copied and a ref count is being incremented. It's a new reference to the same stuff as the passed-in reference. But if you assign directly to the variable holding that reference, that variable will then be holding the new reference. This is base level, "I'd use it on an interview to see if you really know Python", level stuff.

Everything in a modern language involves passing things by value. Sometimes the language will gloss over it for you, but it's still a gloss. There were languages where things fundamentally, at the deepest level, were not passed by value. They're gone. Passing references by copy is not the same thing, and that Python code is precisely why it's not the same thing.

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#124

Post by tsimionescu » Mon, Aug 12, 2024, 2:41 PM UTC

froh wrote: post #121

tsimionescu wrote: post #65

Earlier quoted context omitted.

> if you modify your experiment to pass around a dict or list and modify that in the 'y', you'll see y is happily modified. No, you won't. x = {'a' : 1} foo(x) print(x) def foo(z): z = {'b' : 2} You'll see that this prints `{'a' : 1}`, not `{'b' : 2}`. Python always uses pass-by-value. It passes a copy of the pointer to a dict/list/etc in this case. Of course, if you modify the fields of the z variable, as in `z['b']…

you replace the local binding z to the dict globally bound to x by a local dict in that z = ... assignment. however if you do z['b'] = 2 in foo, then you'll see the global dict bound to x has been modified, as you have stated. well, that's _exactly_ pass by reference.

There is no notion of variable binding in Python, that's a different thing. z, like any Python variable, is a reference to something. Initially, it's a reference to the same dictionary that x references. If we modify the object referenced by z (e.g. by adding a new item), we of course also modify the object referenced by x, as they are referencing the same object initially. However, when we assign something to z, we change the object that z is referencing. This has no effect on x, because x was passed-by-value to foo().

Pass-by-reference doesn't exist in Python. Here's what it looks like in C#, which does suport it:

  auto x = Dictionary() ;
  x.Add("a", 1);
  foo(ref x);
  System.Println(x); //prints {b: 2}

  void foo(ref Dictionary z) {
    auto k = new Dictionary();
    k.Add("b", 2);
    z = k;
  }

Here z is just a new name for x. Any change you make to z, including changing its value, applies directly to x itself, not just to the object referenced by x.

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#125

Post by tsimionescu » Mon, Aug 12, 2024, 2:56 PM UTC

dwattttt wrote: post #111

tsimionescu wrote: post #65

Earlier quoted context omitted.

> if you modify your experiment to pass around a dict or list and modify that in the 'y', you'll see y is happily modified. No, you won't. x = {'a' : 1} foo(x) print(x) def foo(z): z = {'b' : 2} You'll see that this prints `{'a' : 1}`, not `{'b' : 2}`. Python always uses pass-by-value. It passes a copy of the pointer to a dict/list/etc in this case. Of course, if you modify the fields of the z variable, as in `z['b']…

Is it not pass-by-reference by some technicality? In the mutation example you suggest, if a reference to x isn't being passed into foo, how could foo modify x? I would sooner believe the example is showing you shadowing the z argument to foo, than foo being able to modify the in-parameter sometimes even if it's pass by value.

> In the mutation example you suggest, if a reference to x isn't being passed into foo, how could foo modify x

The important point is that it's not "a reference to x" that gets passed, it's a copy of x's value. x's value, like the value of all Python variables, is a reference to some object. The same thing applies to setting variables in Python in general:

  x = {1:2} # x is a new variable that references some dict 
  y = x # y is a new variable that references the same dict
  y[1] = 7 # the dict referenced by x and y was modified
  x = None # x no longer references the dict
  print(y) # y still references the dict, so this will print {1:7}
  y = None # now neither x nor y reference that dict; since y was the last reference to it, the dict's memory will be freed

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#126

Post by klyrs » Mon, Aug 12, 2024, 4:03 PM UTC

tsimionescu wrote: post #62

klyrs wrote: post #38

Earlier quoted context omitted.

Is copying huge blocks of data free in 2024? My benchmarks suggest otherwise, and the world still needs assembly programmers.

The way almost all programming languages work is that they explicitly pass a copy of a pointer to a function. That is, in almost all languages used today, whether GC or not, assigning to a function parameter doesn't modify the original variable in the calling function. Assigning to a field of that parameter will often modify the field of the caller's local variable, though. That is, in code like this: ReferenceType a…

Right. Passing pointers is much cheaper than passing values of large structures. And then references are an abstraction over pointers that allow further compile-time optimization in languages that support it. Pass-by-value, pass-by-pointer, and pass-by-reference are three distinct operational concepts that should be taught to programmers.

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#127

Post by [deleted] » Mon, Aug 12, 2024, 4:15 PM UTC

tialaramex wrote: post #122

foldr wrote: post #101

Earlier quoted context omitted.

I think the same thing could happen in Rust. foo.x = bar will compile if foo is a copy or if it's a mutable reference. As in Go, you'd have to explicitly type 'foo' in order for any compiler error to show up.

Let's work through the example 1. There's a user-defined type named Thing with a String inside it. 2. We've got a function named findThing, which takes an array of Things and a string name, performs a linear search of the array and returns a mutable reference to the Thing with that name or if it's not found it doesn't. In Go, this function accidentally gives back a mutable reference to a copy of the Thing. 3. We make…

[deleted]

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#128

Post by [deleted] » Mon, Aug 12, 2024, 4:21 PM UTC

Animats wrote: post #53

The semantics of when stuff is copied, moved, or passed by reference are all over the place in language design. C started with the idea that functions returned one int-sized value in a register. This led to classic bugs where the function returns a pointer to a local value. Compilers now usually catch this. C eventually got structure return by copy. Then C++ added return value by move, and automatic optimization for…

[deleted]

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#129

Post by foldr » Mon, Aug 12, 2024, 4:21 PM UTC

tialaramex wrote: post #122

foldr wrote: post #101

Earlier quoted context omitted.

I think the same thing could happen in Rust. foo.x = bar will compile if foo is a copy or if it's a mutable reference. As in Go, you'd have to explicitly type 'foo' in order for any compiler error to show up.

Let's work through the example 1. There's a user-defined type named Thing with a String inside it. 2. We've got a function named findThing, which takes an array of Things and a string name, performs a linear search of the array and returns a mutable reference to the Thing with that name or if it's not found it doesn't. In Go, this function accidentally gives back a mutable reference to a copy of the Thing. 3. We make…

Ah, I was working on a mistaken assumption about the Go example code (I thought the function returned Thing, whereas actually it returns *Thing). That will teach me to RTFA...

Re: Go structs are copied on assignment (and other things about Go I'd missed)

#130

Post by tsimionescu » Mon, Aug 12, 2024, 4:31 PM UTC

klyrs wrote: post #126

tsimionescu wrote: post #62

Earlier quoted context omitted.

The way almost all programming languages work is that they explicitly pass a copy of a pointer to a function. That is, in almost all languages used today, whether GC or not, assigning to a function parameter doesn't modify the original variable in the calling function. Assigning to a field of that parameter will often modify the field of the caller's local variable, though. That is, in code like this: ReferenceType a…

Right. Passing pointers is much cheaper than passing values of large structures. And then references are an abstraction over pointers that allow further compile-time optimization in languages that support it. Pass-by-value, pass-by-pointer, and pass-by-reference are three distinct operational concepts that should be taught to programmers.

I think the right mental model is pass-by-value for the first two. There is nothing different in the calling convention between sending a parameter of type int* vs a parameter of type int. They are both pass-by-value. The value of a pointer happens to be a reference to an object, while the value of an int is an int. In both cases, the semantics is that the value of the expression passed to the function is copied to a local variable in the function.

Depending on the language, that is very likely the whole picture of how function calls work. In a rare few modern languages, this is not true: in C# and C++, when you have a reference parameter, things get sonewhat more complicated. When you pass an expression to a reference parameter, instead of copying the value of evaluating that expression into the parameter of the function, the parameter is that value itself. It's probably easier to explain this as passing a pointer to the result of the expression + some extra syntax to auto-dereference the pointer.