How Lisp is Going to Save the World

As presented, the example macro doesn't demonstrate referential transparency, but then again, the example appears to be incorrect.

The issue is invoking "(swapargs (mod 7 5))". I tried this in the Scheme REPL (Chicken to be precise), in which the macro was defined:

  (define-syntax swapargs
    (syntax-rules ()
      ((_ ls) (list (list-ref ls 0) (list-ref ls 2)
                    (list-ref ls 1)))))

  (swapargs '(a b c)) => (a c b)
  (swapargs (swapargs '(a b c))) => (a b c)

In other words, the macro does show referential transparency.

However, the following doesn't work:

  (swapargs (modulo 7 5)) => Error: (list-tail) bad   
                             argument type: 2

The problem is the argument is evaluated first, and "2" is not a list. (The macro requires a list-of-3 argument.)

  (modulo 7 5) => 2

Probably, what was intended was like this:

  (swapargs '(module 7 5)) => (modulo 5 7)

And again:

  (swapargs (swapargs '(module 7 5))) => (module 7 5)
  (eval (swapargs '(module 7 5))) => 5
  (eval (swapargs (swapargs '(module 7 5)))) => 2
  (eval (swapargs (swapargs '(module 10 8)))) => 2

It looks like confusion between literal and evaluable lists prompted the wrong conclusion, but in this case it's simple to rectify.

Of course, macros in Scheme/Lisp can easily become convoluted and bug-ridden as much as any code, even aside from arguments about the virtues of "hygienic" vs. "unhygienic" systems. Properly constructed, macros remain an essential feature of Lisp/Scheme languages.

BTW, if we're comparing qualities of programming languages, here's a real-life example showing the particular merit of Scheme. I took on the task of creating a complex application (a web server supporting multiple hosts) and decided to write it primarily in Scheme (and some C). The first version was up and running in less than half a year.

Inevitably, months after the project was deployed changes were necessary. Despite the length of time since last seen, the code wasn't obscure to me, it was easy to understand and pick up where I'd left off before. Definitely different from prior experiences.

The crux is getting a good grasp on its core, macrology perhaps among the harder parts. But understood, Scheme allows enhanced productivity, as I've known it more so than other languages "under load" in parallel situations.

I admit I didn't have enough time to try your code, so excuse me if I made some oversight, but it seems to me that the basic oversight that you made is that there are two steps that clojure compiler does with the code. In the first step, all macros are expanded, replacing all macro calls with the actual code they generate. Then, the resulting, "macro-free" code is compiled to bytecode. So, you can try macro calls as many times as you want: all calls with the same arguments (which are clojure forms, that is - code, and not the actual values of that code in the runtime) result in exactly the same code. 5s 8s and all the data from your example does not exist yet in that moment. Then, the transparency of functional calls is as advertised: it is referentially transparent if your code is pure (no Java calls etc.). I think the misunderstanding was in forgetting that macros are expanded and gone long before your code starts to run...

I am more interested in what the human sees than the compiler. And to a human, it does not look referentially transparent.

OK, I'll try to be more precise this time, so I hope that you reconsider at least a few things:

First, I think that the main misunderstanding is, as you mentioned, your somewhat incomplete experience of Clojure.

Second, perhaps there is a tiny bit of a strawman fallacy in your argument. Why? Well, def is not for defining "variables". It does get mentioned many times in Clojure literature: these are vars, you can change them, but they are NOT there for the regular data, but to enable you to give names to your functions and the rest of clojure artefacts. The programmer does not need to use metaprogramming or macros to shoot himself in the foot using vars as "variables". How does that relate to the wikipedia definition of referential transparency? Well, only PURE functions are referentially transparent. Def is not pure, and it is not even a function - it is a special form. Of course, it could be confusing for a novice, but it has been clearly stated many times in the literature how and why, so to anyone ho has learned the basics well and is not trying to recreate Java/python/Ruby/C coding style in Clojure, that should not be a source of problems.

As for your macro example, the first part of your argument, "First, I'll show that the argument that it takes must be the code itself, rather than the result of evaluating the code" - that is something macros are for, so the programmer expects exactly what you described, although you did not need a macro for swapping arguments in a function call.

I hope that we agree that, if we know that there are two phases in Cloure compilation process, a "macro" phase and a "regular code" phase, then everything is fine and clear. I understand that your complaint is that the programmer might forget that so she might be confused in the examples that you stated, so I will continue with that assumption in mind.

Let's try referential transparency with the function foo:

1) Is the function foo referentially transparent?

user=> (defn foo [a b] (swapargs (mod a b))) #'user/foo

user=> (foo 7 5)

5

user=> (foo 10 8)

8

user=> (foo 10 8)

8

user=> (foo 10 8)

8

user=> (foo 7 5)

5

As we can see, "the same function (foo) with the same arguments will always produce the same result, and that you can call it more (throwing away the result) or fewer (replacing calls with the result) times without affecting the meaning of the program".

2) Is swapargs macro referentially transparent?

user=> (swapargs (mod 7 5))

5

user=> (swapargs (mod 7 5))

5

user=> (swapargs (mod 7 5))

5

user=> (swapargs (mod 10 8))

8

user=> (swapargs (mod 10 8))

8

Is "the same function (swapargs) with the same arguments will always produce the same result, and that you can call it more (throwing away the result) or fewer (replacing calls with the result) times without affecting the meaning of the program"? Well, is swapargs a function? -No, but let's forget that for a sake of being fair. The proper way to resolve this is to read the documentation of swapargs. And the documentation would say that the argument to swapargs is a clojure form (something like (mod 7 5)), not a number (something like the result of calling (mod 7 5)). If we having that in mind, it is clear that swapargs is also referentially transparent, with regards to its argumens. We CAN replace swapargs with its result, s you can see in the aforementioned code. If we want our argument so be the results of (mod 7 5) we would use a function, not a macro!

With the rest of your post, I agree. Macros are not superpowerful, you can shoot yourself in the foot with them (and every lisp book warns you about that in many ways), and haskell is awasome in many ways. There could be many pitfals with clojure and macros, but I think these pitfalls are not of the kind that your examples show :)

I was (re-)reading On Lisp yesterday and came across two further variations by pg:

In preface:

As well as writing their programs down toward the language, experienced Lisp programmers build the language up toward their programs.

Then later in 1.2 "Programming Bottom-up", pg rephrases that statement:

In Lisp, you don’t just write your program down toward the language, you also build the language up toward your program