Live data from Hacker News

The Problem with Macros

ianthehenry.com

41–50 of 70 posts

Re: The Problem with Macros

#41
Isn't this problem trivially solved by hygienic macros in Scheme?

At least I'm sure it is by fexpr because with them you have complete control over evaluation.

For those who don't know, fexpr are closures (i.e., lexically scoped functions) that are called with unevaluated arguments (just like macros) and are implicitly passed the dynamic environment (i.e., the call site environment). That way you have all the powers of macros and functions with fexpr, and no scoping problem.

But for the capturing problem mentioned in the article, I'm quite sure hygienic macros are enough.

Re: The Problem with Macros

#42
post #21

May be unrelated, but that's why I prefer the way JS approach this kind of problem: JS doesn't have macros. People use callback function to achieve almost the same thing function doTexture(texture, fn) { beginTextureMode(texture) fn() endTextureMode() } doTexture(myTexture, () => { drawCircle() drawRectagle() }) At the cost of being more verbose, we get the benefit of one thing less to learn; and simplicity is a powe…

> People use callback function to achieve almost the same thing

There's a QoL difference here with macros - writing out those lambdas can become annoying. That said, the "good style" rule in (Common) Lisp is to prefer lambda-forms in such cases - i.e. cases where the macro parameters are a block of code to be mostly run straight.

In fact, a common pattern for with-macros (of which doTexture would be an example), is the call-with pattern. Example from some random project of mine:

  (defmacro with-logging-conditions (&body forms)
    "Set up a passthrough condition handler that will log all signalled conditions."
    `(call-with-logging-conditions (lambda () ,@forms)))
Which is then used like:

  (with-logging-conditions
    (blah blah)
    (main game code))
All the macro does is, upon expansion, package the code block into a lambda, and passing it to a function call-with-logging-conditions, that does the actual work. So it's like your example, except I don't have to write the lambda myself. This is a trivial case; commonly, macros might accept additional arguments that they process, but eventually they'd still wrap their input body argument in a lambda and expand to a function call with said lambda as argument.

A better use of macros, which you can't replicate in JavaScript[0], would be if you wanted to do something like:

  (do-texture texture
    o O r R x2)
And have it expand - at compile time - to:

  ;; unwind-protect is Lisp's sorta-equivalent of try/finally in other languages.
  (unwind-protect
    (progn
      (begin-texture-mode texture)
      (draw-circle)
      (draw-circle :big)
      (draw-rectangle)
      (draw-rectangle :big)
      (draw-rectangle :big))
    (end-texture-mode))
However silly this looks, this kind of code generation is (one of the main reasons) why you need macros.

--

[0] - Well, you can if you have a toolchain. Babel is essentially a macro engine for JavaScript, but you can only use it at build time.

Re: The Problem with Macros

#43

As unsatisfying as it may sound, Common Lisp taught me that, truly, none of this function binding capture stuff really matters in practice. Millions of lines of Common Lisp code have been running for decades without running into problems with capture. So I purport solving this problem is akin to solving a 0.00000001% issue if we measure the frequency of encountering this error writing thousands of lines of Lisp per d…

Similar to variable capture with dynamic scope e.g. in classical Emacs Lisp? That's been the root cause of a bug for me exactly once in years of Emacs hacking, and having finally been bitten my radar detects that risk going forward. So much as I prefer lexical scope the alternative seems pretty tame in practice.

Re: The Problem with Macros

#44
post #43

As unsatisfying as it may sound, Common Lisp taught me that, truly, none of this function binding capture stuff really matters in practice. Millions of lines of Common Lisp code have been running for decades without running into problems with capture. So I purport solving this problem is akin to solving a 0.00000001% issue if we measure the frequency of encountering this error writing thousands of lines of Lisp per d…

Similar to variable capture with dynamic scope e.g. in classical Emacs Lisp? That's been the root cause of a bug for me exactly once in years of Emacs hacking, and having finally been bitten my radar detects that risk going forward. So much as I prefer lexical scope the alternative seems pretty tame in practice.

I miss dynamic scope in non-Lisp languages. Perhaps defaulting to it for general-purpose programming was a bad idea (one on which most Lisps backtracked), but it's arguably a good choice for an extensible application like Emacs, and it's definitely a tool you want to have available.

An easy way to see this is: dynamic binding is to environmental variables what lexical binding is to command line arguments.

Re: The Problem with Macros

#45

Earlier quoted context omitted.

My experience that a certain systems programming language of Unix origins featuring 1 namespace and an unhygienic token-based macro system also has few issues with function-name capture, even in code bases with seven digit LOCs. That makes me severely disinterested and skeptical about hygienic macros. They are too weird for the little benefit they provide. You can't look at a piece of code and know what it expands to…

All I know is that in Python, JS and Clojure is I’ve accidentally written code like: def foo(str): v = str(1) . . . Which resulted in head-scratching errors like “str is not a function”. EDIT: I agree that hygienic macros aren't the right way to solve this issue.

With Lisp-2 designs as discussed in the article this is not an issue, as variables and functions are in different namespaces:

  CL-USER> (defun foo (list) (list list))
  FOO
  CL-USER> (foo 42)
  (42)
In this case the function attached to the symbol LIST is applied to the argument with the same name, but that isn't a problem.

To further illustrate, in the above example the LIST symbol is imported from the package COMMON-LISP and has a function, plist etc. attached to it:

  CL-USER> (symbol-package 'list)
  #
  CL-USER> (symbol-function 'list)
  #
  CL-USER> (symbol-plist 'list)
  NIL

Re: The Problem with Macros

#46
post #43

Earlier quoted context omitted.

Similar to variable capture with dynamic scope e.g. in classical Emacs Lisp? That's been the root cause of a bug for me exactly once in years of Emacs hacking, and having finally been bitten my radar detects that risk going forward. So much as I prefer lexical scope the alternative seems pretty tame in practice.

I miss dynamic scope in non-Lisp languages. Perhaps defaulting to it for general-purpose programming was a bad idea (one on which most Lisps backtracked), but it's arguably a good choice for an extensible application like Emacs, and it's definitely a tool you want to have available . An easy way to see this is: dynamic binding is to environmental variables what lexical binding is to command line arguments.

Buffer-local variables are also underrated. There's another universe where this context-switching between objects is what we call "OO" instead of Smalltalk/Self style.

Re: The Problem with Macros

#47
"So we’re supposed to be writing a game, right? But in order to make progress, we have to fix a bug. And in order to fix the bug, we have to write a test. And in order to write a test, we have to write a test framework. And in order to write a test framework, we have to understand a thing or two about macros."

You could just fix the bug?

Game development is a different architecture to 'regular' system development (especially if it is a solo project).

If you are testing to ensure the bug isn't reintroduced then you haven't fixed the bug.

Re: The Problem with Macros

#48

"So we’re supposed to be writing a game, right? But in order to make progress, we have to fix a bug. And in order to fix the bug, we have to write a test. And in order to write a test, we have to write a test framework. And in order to write a test framework, we have to understand a thing or two about macros." You could just fix the bug? Game development is a different architecture to 'regular' system development (es…

> If you are testing to ensure the bug isn't reintroduced then you haven't fixed the bug.

It’s called regression testing, and it’s a (fairly) common thing.

Re: The Problem with Macros

#49

Earlier quoted context omitted.

> closures over lexical environments possibly shared by other functions The same problem exists with lists (or any other mutable object). If x and y point to the same list, and you print x and read it back in, and then modify the list x points to, then (naively) it won't modify y. If you did want to preserve such structure sharing, one approach would be to print the entire environment and make liberal use of #n= nota…

Functions with compiled code referring to address offsets in the closure environment are not printable and I’d hazard to say they can’t be without compromising something else. The function being purportedly serialized is already in a representation very far removed from its source code, and has mutable state that isn’t just from its closure environment. LOAD-TIME-VALUE (in Common Lisp) is another problem that allocat…

> Functions with compiled code referring to address offsets in the closure environment are not printable and I’d hazard to say they can’t be without compromising something else.

Surely these offsets refer to items saved in the lexical or global environment, which were originally named by variables? Then you serialize the variable references and the values they refer to. A shared lexical environment would get serialized like this:

  (let ((x 10) (y '(1 2)))
    (let ((f (lambda (z) (set! x z))))
      (let ((g (lambda (a) (list x y)))
            (h (lambda (b) (f b))))
        (list g h))))
  ; serializes to the following,
  ; with the notation #closure(args body env)
  ; where env is ((var1 val1) (var2 val2) ...)
  (#closure( (a)
             ((list x y))
             (#1=(f #closure( (z)
                              ((set! x z))
                              (#2=(x 10) #3=(y (1 2)))))
              #2#
              #3#))
   #closure( (b)
             ((f b))
             (#1#
              #2#
              #3#)))
The runtime system is presumably competent at converting these into machine code, and can do that either during the read, or JIT at execution.

Yes, most of the time, when you print lists, either there is no shared structure, or the program doesn't try to modify it, so it doesn't matter if you read a portion back in and bifurcate the identity. Likewise, for the majority of functions, either they don't share their lexical environment with others, or they do but not in a way where it matters. (This thread's original use case was printing globally defined functions, which usually have no lexical environment.) The common case should work fine like this:

  (let ((n 0)) (lambda (x) (incf n x)))
  ; serializing to
  #closure( (x) ((incf n x)) ((n 0)))
If you do need to print a list and read it back in, where it's important that this list shares structure with existing objects, and you're not printing and reloading those objects, then either (a b . #), or, if your runtime doesn't relocate objects, (a b . #do need to print a lambda sharing a lexenv with another one you're not printing, you'd go #(closure arglist body (# # ...)). Of course, this only works if you reify the object in the same running Lisp session from which you serialized it—but that applies equally to shared lists and shared closures.

The question upthread seemed to be whether it was possible to have a language that could serialize functions. I've argued yes. Now the counterargument seems to be "well, it's inconvenient and expensive to get full fidelity". This brings us to the question of what use cases we're talking about, and how often you want full fidelity (and what assumptions you can make).

I think one of the use cases was printing macroexpansions in the REPL? And then maybe the readability-requiring use case would be selecting a subexpression and saying (macroexpand-1 '([paste])). The literal functions in the expansion would generally be globally defined—the result of using ",+" instead of "+". Well, I think that's covered reasonably well by having the globally-defined function "foo" get printed as #f:foo or something like that. (Maybe we could use the syntax #'foo to mean that? Hah.) It's less nice than seeing the bare name, but macroexpansions already often contain gensyms and package prefixes.

Re: The Problem with Macros

#50
post #21

May be unrelated, but that's why I prefer the way JS approach this kind of problem: JS doesn't have macros. People use callback function to achieve almost the same thing function doTexture(texture, fn) { beginTextureMode(texture) fn() endTextureMode() } doTexture(myTexture, () => { drawCircle() drawRectagle() }) At the cost of being more verbose, we get the benefit of one thing less to learn; and simplicity is a powe…

> People use callback function to achieve almost the same thing There's a QoL difference here with macros - writing out those lambdas can become annoying. That said, the "good style" rule in (Common) Lisp is to prefer lambda-forms in such cases - i.e. cases where the macro parameters are a block of code to be mostly run straight. In fact, a common pattern for with-macros (of which doTexture would be an example), is t…

> (do-texture texture > o O r R x2)

This might be another reason why I find macros less appealing: macros introduce DSL in form of normal s-expression, but they don't actually behave like a function; macros introduce their own mini-language/syntax.

In the last example you provided, I bet the macro implementation would look like a little interpreter? If that's the case, having a function call like

  doTexture(myTexture, ['op1', 'op0', 'opR'])
and let doTexture handle each cases(ops), might be able to achieve the same behavior, right?

I'm not trying to argue that macros are unnecessary, I really want to like them! Just most of the time, I find functions are sufficient enough.

Post reply on HN