Crafting Interpreters: Closures

This was a great time for this to be posted. I've been spending a lot of time with Racket, and closures are a big part of that. I believe (though maybe I'm wrong) that Racket's closures don't have the issue described in this chapter of closing over values vs. variables due to the functional nature of the language.

Out of scope, alas. One of the real challenges of the book was deciding which language features to include and which to omit. It's on track to an enormous book — it's over 200k words already. So I had to cut what I could. The language in the book doesn't even have arrays.

But it does have lexical scope, closures, objects, inheritance, a full operator precedence table, and a bunch of other fun stuff.

They do, actually. Scheme supports `(set! some-var)` to reassign values to variables, which means you can tell if a closure captures a value or variable.

So, there's one thing here that strikes me as really weird. And I think it's best summed up at the end, where it discusses closing over a loop variable.

It discusses how if you close over a loop variable, people expect that each function created will get a different value of the variable, even though there's only one variable, and closures close over variables, not values. It treats this as a special case. But to my mind this isn't a special case; it just highlights that having closures close over variables rather than values is, well, not right. Like, if I write, to use some sort of pseudocode,

  x = 1;
  f = function() { print(x) };
  x = 2;
  g = function() { print(x) };

...then I expect that f will print 1 and g will print 2. Because, well, I'm coming in from lambda calculus and Haskell where everything is values. Of course, there this problem doesn't come up because there's no mutability -- but that's still how I'd expect things to work once you introduce it. Closing over variables just seems horribly counterintuitive to me.

This came up earlier, too, when the was going through all the machinery with upvalues, and I was just like, why doesn't it just copy in the values? And after reading further, it's like, oh, it's because they want to be able to modify the variable outside the function! Which, like, whoa.

Like obviously what I said above about how I expect closures to work isn't workable if you want to be able to mutate the closed-over variables, outside the closure, from inside the closure. But maybe that just shouldn't be allowed? I think if I were writing a language from scratch, and it included lambdas, they'd close over values, not variables, and mutating the closed-over variables would have no effect on the world outside the closure (or perhaps be disallowed entirely).

So, basically... are you sure that people expect that behavior with loop variables because they are thinking of each iteration as a new variable? Or is it because they don't expect closures to close over variables at all, but rather values, and would be surprised that loop variables are being treated as a special case rather than closures just always working that way? Because in my case it's definitely the latter.

From a functional perspective, this of course doesn't make any sense. But from an imperative perspective it does.

A common use for closures in imperative languages is to have custom control structures similar to `if`, `for`, `while`, etc. And in imperative languages mutating values is very important, if you want a new control structure you have to be able to mutate your surrounding variables.

Smalltalk only uses this kind of control structures based on closures. Ruby also uses this a lot, it even changes the meaning of `return` in closures (blocks) to return not from the closure but the enclosing function.