Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lisp Macros, Delayed Evaluation and the Evolution of Smalltalk

blog.metaobject.com

Post reply on HN
Print view View original

11–20 of 114 posts

Re: Lisp Macros, Delayed Evaluation and the Evolution of Smalltalk

#11

Post by lispm »

skybrian wrote: post #9
lispm wrote: post #4

Earlier quoted context omitted.

That doesn't matter. One can write IF as a function taking functions as THEN and ELSE forms: CL-USER 9 > (defun my-if (test then else) (if test (funcall then) (funcall else))) MY-IF CL-USER 10 > (let ((a 10)) (my-if (> a 5) (lambda () (print 'hello)) (lambda () (print 'world))) 'done) HELLO DONE The lambdas you see above serve the same purpose as blocks in Smalltalk.

It works, but the syntax isn't nice enough to use as-is, so in practice you need a macro. The innovation here is using better syntax for anonymous functions to make such things practical in languages without macros.

If you think brackets are the core innovation of Smalltalk to make lambdas practical, then Lisp has no problem providing it, too:

    CL-USER 19 > (defun bracket-reader (stream char)
                   (declare (ignore char))
                   `(lambda () ,@(read-delimited-list #\] stream t)))
    BRACKET-READER

    CL-USER 20 > (set-macro-character #\[ #'bracket-reader)
    T

    CL-USER 21 > (let ((a 10))
                  (my-if (> a 5)
                        [(print 'hello)]
                        [(print 'world)])
                  'done)

    HELLO 
    DONE
With a bit more effort, we could also parse a parameter list.

But Lisp does not go the route of making IF a function, because it typically provides three different types of language expressions and IF then is a special operator:

1) function calls

2) macro forms

3) a small set of special operators, which are implemented as built-in functionality. One of them is a core conditional operator. More complex conditional operators then are implemented as macros, which expand eventually into the core conditional operator. The interpreter and compiler will have to specially recognize and implement these special operators.

See IF: http://clhs.lisp.se/Body/s_if.htm

The usual Lisp view is that

  [(print 'hello)]
is no useful improvement over

  (lambda () (print 'hello))
Some disagree, but most of the time the pattern

  (  ... )
is preferred in Lisp over

    ...

Re: Lisp Macros, Delayed Evaluation and the Evolution of Smalltalk

#12

Post by bitwize »

jfoutz wrote: post #8
reitzensteinm wrote: post #5

Earlier quoted context omitted.

What are lisp macros for if not to delay evaluation? I assume you're making a distinction between purpose and mechanism here? I've always understood the semantics to be: fn: (a b c) => (call a (eval b) (eval c)) mac: (a b c) => (eval (call a b c))

i'll defer to lispm's expert opinion, but i always thought of macros as _conditional_ evaluation. `if` is a special form because it does not evaluate the branch not taken. it's tricky to implement `if` without using `if` or `cond`. macros make it easy to implement your own special forms. in your example, you might want to implement `first`. (first b c) -> (eval b) i can't think of a way (but it might be possible) to…

Macros are more general than that; they are generalized expression rewriters. `if` is tricky to implement in general because you need a primitive to select an alternative based on whether a given value is truthy or falsy. Let's pretend we have such a primitive that's not a special form called `call-if`. Then you can implement `if` in terms of `call-if` by writing it as a macro that transforms this:

    (if (condition? x) (do-a x) (do-b x))
into this:

    (call-if (condition? x) (lambda () (do-a x)) (lambda () (do-b x)))
(The semantics of `call-if`, by the way, are exactly the semantics exposed by the Smalltalk ifTrue:ifFalse: family of methods.)

But macros can do more than merely wrap things in lambdas, which alone is sufficient for delayed and conditional evaluation. They can quote identifiers that are passed into them, allowing you to write new defining forms and even introduce new bindings. A nontrivial, but understandable, example is the `define-record-type` special form from SRFI 9, which can define a new disjoint record type and a constructor, predicate, and field accessors for that type all in one form. If you are willing to break hygiene and have `defmacro` or something like `syntax-case` available, you can even introduce new identifiers from out of nowhere, and write something like Common Lisp's `defstruct` which lets you define a type and automatically derive names for the constructor, slot accessors, etc. from the names of the type and slots.

Macros are a tremendously powerful, general code generation and rewriting tool, built right into the language. Underestimate them at your own peril!

Re: Lisp Macros, Delayed Evaluation and the Evolution of Smalltalk

#13

Post by reitzensteinm »

lispm wrote: post #6
reitzensteinm wrote: post #5

Earlier quoted context omitted.

What are lisp macros for if not to delay evaluation? I assume you're making a distinction between purpose and mechanism here? I've always understood the semantics to be: fn: (a b c) => (call a (eval b) (eval c)) mac: (a b c) => (eval (call a b c))

That might be more like what was called an FEXPR mechanism in some early Lisps or some niche Lisps, or even in languages like R. https://en.wikipedia.org/wiki/Fexpr When an FEXPR is called, it gets unevaluated args and can then at runtime decide what to do. Macros OTOH are a different mechanism, where expressions get rewritten at macro expansion time, which can be for example at compile time. The macro then gets call…

I don't think what I wrote is a FEXPR, which doesn't evaluate the result of the function call. I'm having a hard time parsing what you wrote or any of the sources you linked in such a way that says the semantics of macros differs from what I wrote (certainly the implementation gets a lot more complex and there are subtleties).

Re: Lisp Macros, Delayed Evaluation and the Evolution of Smalltalk

#14

Post by jfoutz »

bitwize wrote: post #12
jfoutz wrote: post #8

Earlier quoted context omitted.

i'll defer to lispm's expert opinion, but i always thought of macros as _conditional_ evaluation. `if` is a special form because it does not evaluate the branch not taken. it's tricky to implement `if` without using `if` or `cond`. macros make it easy to implement your own special forms. in your example, you might want to implement `first`. (first b c) -> (eval b) i can't think of a way (but it might be possible) to…

Macros are more general than that; they are generalized expression rewriters. `if` is tricky to implement in general because you need a primitive to select an alternative based on whether a given value is truthy or falsy. Let's pretend we have such a primitive that's not a special form called `call-if`. Then you can implement `if` in terms of `call-if` by writing it as a macro that transforms this: (if (condition? x)…

aww, now i'm embarrassed about my lack of imagination. I have to admit i forgot about the magic you can do with define.

Excellent point. i kinda sorta think defun in common lisp injects the new function at top level regardless, but in scheme it's scoped. So in CL you could probably get away with a simple lambda (or function) to make your struct syntax, scheme would need the macro to put all of that stuff at the top level.

Yeah, macros are crazy powerful

Re: Lisp Macros, Delayed Evaluation and the Evolution of Smalltalk

#15

Post by kazinator »

jfoutz wrote: post #8
reitzensteinm wrote: post #5

Earlier quoted context omitted.

What are lisp macros for if not to delay evaluation? I assume you're making a distinction between purpose and mechanism here? I've always understood the semantics to be: fn: (a b c) => (call a (eval b) (eval c)) mac: (a b c) => (eval (call a b c))

i'll defer to lispm's expert opinion, but i always thought of macros as _conditional_ evaluation. `if` is a special form because it does not evaluate the branch not taken. it's tricky to implement `if` without using `if` or `cond`. macros make it easy to implement your own special forms. in your example, you might want to implement `first`. (first b c) -> (eval b) i can't think of a way (but it might be possible) to…

It's not exactly conditional evaluation that the macro

  (defstruct point x y)
defines a structure type, with name point and slots x and y.

Well, sure; in a sense it is unconditional (which is a condition) that point, x and y are not evaluated (which is an evaluation choice).

Re: Lisp Macros, Delayed Evaluation and the Evolution of Smalltalk

#16

Post by lispm »

reitzensteinm wrote: post #13
lispm wrote: post #6

Earlier quoted context omitted.

That might be more like what was called an FEXPR mechanism in some early Lisps or some niche Lisps, or even in languages like R. https://en.wikipedia.org/wiki/Fexpr When an FEXPR is called, it gets unevaluated args and can then at runtime decide what to do. Macros OTOH are a different mechanism, where expressions get rewritten at macro expansion time, which can be for example at compile time. The macro then gets call…

I don't think what I wrote is a FEXPR, which doesn't evaluate the result of the function call. I'm having a hard time parsing what you wrote or any of the sources you linked in such a way that says the semantics of macros differs from what I wrote (certainly the implementation gets a lot more complex and there are subtleties).

That's why I wrote 'more like'. Given that you haven't defined any semantics of your operators, it's more like a guess.

Not sure if this helps you. But let's define a macro A:

    CL-USER 32 > (defmacro a (b c)
                   (print (list :macro-expansion b c))
                   (list 'print (list 'quote (list :runtime :b b :c c))))
    A
This macro does two things: it prints something at macro expansion time and then generates some code it returns as a value.

Now we can use this macro in some code:

    CL-USER 33 > (defun test ()
                   (a 21 42))
    TEST
If we now compile the function, the macro gets executed and prints something:

    CL-USER 34 > (compile 'test)

    (:MACRO-EXPANSION 21 42) 
    TEST
    NIL
    NIL
We can also call the macroexpander independent of the compiler. The we a) get the side effect of the print statement and we see the result:

    CL-USER 35 > (macroexpand '(a 20 30))

    (:MACRO-EXPANSION 20 30) 
    (PRINT (QUOTE (:RUNTIME :B 20 :C 30)))
    T
At runtime we call the test function:

  CL-USER 36 > (test)

  (:RUNTIME :B 21 :C 42)  ;
Thus all the macro expansions have been done at compile time and we have generated some code there. No macro expansion at runtime.

Thus it has to do with code generation and code execution at macro expansion - nothing about 'delaying' something.

The example isn't useful, but imagine a macro INFIX

   (infix a * b + c)
which rewrites the expression to the Lisp expression:

   (+ (* a b) c)
There is nothing about 'delaying' -> it's just rewriting the form. Ideally at compile time. There are many other examples which do something different.

Re: Lisp Macros, Delayed Evaluation and the Evolution of Smalltalk

#17

Post by lispm »

jfoutz wrote: post #14
bitwize wrote: post #12

Earlier quoted context omitted.

Macros are more general than that; they are generalized expression rewriters. `if` is tricky to implement in general because you need a primitive to select an alternative based on whether a given value is truthy or falsy. Let's pretend we have such a primitive that's not a special form called `call-if`. Then you can implement `if` in terms of `call-if` by writing it as a macro that transforms this: (if (condition? x)…

aww, now i'm embarrassed about my lack of imagination. I have to admit i forgot about the magic you can do with define. Excellent point. i kinda sorta think defun in common lisp injects the new function at top level regardless, but in scheme it's scoped. So in CL you could probably get away with a simple lambda (or function) to make your struct syntax, scheme would need the macro to put all of that stuff at the top l…

In Common Lisp, DEFUN runs side effect code at compile time. For example it notes the function in the compile-time environment, it may record its definition/the location of the definition, ... It also may rewrite the code it defines: for example by adding some declarations, etc.

Re: Lisp Macros, Delayed Evaluation and the Evolution of Smalltalk

#18

Post by skybrian »

lispm wrote: post #11
skybrian wrote: post #9

Earlier quoted context omitted.

It works, but the syntax isn't nice enough to use as-is, so in practice you need a macro. The innovation here is using better syntax for anonymous functions to make such things practical in languages without macros.

If you think brackets are the core innovation of Smalltalk to make lambdas practical, then Lisp has no problem providing it, too: CL-USER 19 > (defun bracket-reader (stream char) (declare (ignore char)) `(lambda () ,@(read-delimited-list #\] stream t))) BRACKET-READER CL-USER 20 > (set-macro-character #\[ #'bracket-reader) T CL-USER 21 > (let ((a 10)) (my-if (> a 5) [(print 'hello)] [(print 'world)]) 'done) HELLO DON…

Okay, but if you have macros, you might as well use them. The idea is more about exploring what can be done when designing a less powerful language, without macros.

Re: Lisp Macros, Delayed Evaluation and the Evolution of Smalltalk

#19

Post by lispm »

skybrian wrote: post #18
lispm wrote: post #11

Earlier quoted context omitted.

If you think brackets are the core innovation of Smalltalk to make lambdas practical, then Lisp has no problem providing it, too: CL-USER 19 > (defun bracket-reader (stream char) (declare (ignore char)) `(lambda () ,@(read-delimited-list #\] stream t))) BRACKET-READER CL-USER 20 > (set-macro-character #\[ #'bracket-reader) T CL-USER 21 > (let ((a 10)) (my-if (> a 5) [(print 'hello)] [(print 'world)]) 'done) HELLO DON…

Okay, but if you have macros, you might as well use them. The idea is more about exploring what can be done when designing a less powerful language, without macros.

> The idea is more about exploring what can be done when designing a less powerful language, without macros.

Lot's of things. But the language won't be able to easily compute code at compile time, which a main purpose of macros. Lisp has a simple data format for source code and thus code transformations are relatively simple to do and even integrated into the language.

Re: Lisp Macros, Delayed Evaluation and the Evolution of Smalltalk

#20

Post by mpweiher »

lispm wrote: post #2

That's a common misconception that Lisp macros are mostly used to 'delay' evaluation. What Smalltalk calls 'blocks' are just (anonymous) functions in Lisp. Books like SICP explain in detail how to use that for delayed evaluation in Lisp/Scheme: https://mitpress.mit.edu/sites/default/files/sicp/full-text/...

> misconception that Lisp macros are mostly used to 'delay' evaluation.

It was in the examples. No generalization to all of LISP was made or intended, though it would be interesting to make a survey study of actual macro usage by developers.

It is also clear that you actually need the delay-evaluation mechanism in order to get something like "if" out of the language and into the library.

> what Smalltalk calls 'blocks' are just (anonymous) functions in Lisp

"but blocks/anonymous functions seem quite different from alternate argument-passing mechanisms."

I think the history also explains why Smalltalk blocks were a bit odd as anonymous functions go: they did not start out as a "function" mechanism, they started out as just a delayed evaluation mechanism, and Smalltalk has no ordinary functions, just the anonymous kind. Which is also odd.

The Smalltalk developers were well aware of LISP, cite it as a major source of inspiration in fact. They initially had a more macro-ish mechanism (the unevaluated token-stream), then moved to a pure delayed-evaluation mechanism which then turned into blocks. They never added a macro mechanism again. I think that's interesting. YMMV.

Post reply on HN