Stop writing lambda expressions in Python

I also work in an enterprise situation and very many bugs that I deal with from legacy code and others’ code comes directly from in-lining anonymous functions.

Many bugs have to do with utilizing closures to access variables needed in the function body, which then make refactoring harder and make modifying unit tests harder.

This can be even worse in languages like Python where there is a distinction between early binding and late binding and you have to be aware if by closure you’re using a reference name that may be associated with different data during its lifetime, or a name whose value won’t change, because a change to the underlying value could make a difference in what is bound inside the anonymous function at different times when it’s called. The classic example is trying to define functions in a loop where functions use the loop variables by closure. Then being surprised when every one of the functions has a reference to only the final value of the loop variable.

Even in statically typed languages, this makes things much harder to reason about than they should be. On the other hand, making an anonymous function that accepts many arguments for all the data it would try to access by closure is stupid: those arguments need to be documented, and it’s just so much cleaner and maintainable to do that with a regular function definition, which also makes it much clearer what all the conditions are for calling the function.

What’s worse is that these things can be deeply welded into some coding context, like using a flatMap over some TypedPipe in Scala / Scalding, and can result in needing to make in-line functions that are hundreds of lines long with arbitrarily complex function bodies, which then become tied into assumptions about the runtime context you’re embedded in, and then nobody can figure out how to refactor it into a standalone function, so it just grows by attrition to an inline lambda over years and is extremely fragile. Change something seemingly unrelated about the outer context it’s defined in, and suddenly you get unexpected, cryptic compiler errors complaining something’s wrong with the TypedPipe, and you have to dig deeper to understand why it’s related to the anonymous function.

I would say many of the most serious closure-related bugs and bugs related to unrefactorable yet undocumented dependence on an enclosing context that I’ve seen have been largely a direct result of the programming style of in-lining anonymous functions inside functional programming constructs.

I sympathize with your claim of “not reading an essay” especially because people can be prone to try to use functional programming or overloading the Python data model and operator syntax with cutesy bullshit that they try to pass off as expressiveness.

But I think there’s a middle ground where you think about it not like an essay, but just basic modularity and separation of concerns, and write functions separately except when they are very short and really trivial.

That is a good point that whichever approach displays the most bugs in a given team is likely to just be whatever is the most common approach for that team, by simple base rates.

I’m not sure how we could objectively decide if either of these two approaches is definitively better, but in the specific, restricted case of a framework like Scalding, I’d strongly wager that avoiding in-lining ends up better in the long run. Those cases also have little connection to the weak module design issue you brought up, since it’s usually a module with de facto map reduce boiler plate and then just a few isolated places with any actual implementation, and when those parts are expressed as huge in-lined anonymous functions inside Scalding data type wrappers, I know right away it’s a bad code smell.

Sorry if this wasn't clear - I'm talking about lambdas. I'm basically complaining about a piece you pointed out, that you can't assign to variables in a lambda expression, this is very not ergonomic IMO. Having to create a def every time I want a function to assign to a variable is annoying. I pretty much want anonymous nested functions.

> Yes, that is a confusing way to write and read code, rather than a linear flow, like flattening the tree by extracting functions into separate definitions.

I don't understand. It's harder to flatten the tree: you have to unflatten it in your mind afterwards to understand what's happening. It's easier to just visualize, say, this lisp function as a tree directly.

    (defun good-enough-p (guess x)
      (format t "~% Guess =~7,4f     Guess^2 = ~7,4f    Error= ~7,4f" guess 
              (* guess guess) (abs (- (* guess guess) x)))
      (