They Called It LISP for a Reason: List Processing (2005)

Generally speaking this isn’t the case. There are probably exceptions besides a repl, but they’ll almost all be repl-like because macros are effectively functions of pre-compiled code, represented as lists as written in the code.

In other words, generally speaking, a lisp program’s life cycle goes something like this:

1. Developer writes lists.

2. Some of these lists are macros, compile them and execute them with their arguments. Many of these are themselves lists and symbols, which will be preserved according to various rules depending on which lisp you’re using and how they isolate code lists from program lists. It’s complicated.

3. Recurse step two until there are no more macro calls.

4. Now you have the actual program, and now you compile that.

5. Now you can execute the actual program.

— ⁂ —

The first thing to understand is that Lisp invented functional programming. (Seibel's chapter mentions this, of course.)

The functional-programming approach is to define your procedures recursively rather than iteratively; this began in Lisp but is now central to a number of other languages, like Haskell, OCaml, and F#. For certain fields, like symbolic algebra and compilers, this approach makes many difficult problems trivial. Even outside those fields, pervasive immutability often has many benefits, eliminating large classes of bugs, greatly simplifying problems like undo and thread-safety, and dramatically reducing the runtime cost of garbage collection. However, recursively defined linked data structures tend to use more space, and they're not very cache-friendly, and sometimes those are more important considerations.

To define your procedures recursively, it's very helpful to define your data types recursively. Recursively-defined lists (for example, "a list of Ts is either the empty list of Ts or a T followed by a list of Ts") support this recursive, immutable approach to programming.

Other kinds of recursive structures do, too; Haskell and ML dialects tend to use application-specific sum types at least as much as general-purpose lists, and I like that style better. It has most of the same advantages and disadvantages.

STL-style vectors do not support a recursive style of programming. They support an iterative, imperative approach to programming. If you use that other approach to programming exclusively, you will not understand why anyone would want to primarily use linked lists. And occasionally you will be confronted with problems that are very difficult for you to solve, problems which would have been trivial with a recursive approach, and you will not realize that you are doing a hundred times more work to solve them than you need to. SICP is full of examples.

— ⁂ —

The second thing about Lisp is that it has orthogonal serialization and deserialization: PRINT and READ. This mechanism is sometimes, though not always, adequate for debug logging, saving and loading application state, configuration files, and networking. Modern Lisps like Clojure generally extend those to arbitrary data structures, not just lists and atoms, but it's especially easy to implement if you only have lists and atoms; it's about 30 lines of Forth, for example¹. This is of course not unique to Lisp anymore; Java, Python, Tcl, Golang, and many other languages have ways to do it.

Orthogonal deserialization of lists is not really an optional extra, since you use it to parse Lisp programs, too. And you need the serialization to print lists in the REPL, anyway.

— ⁂ —

The third thing about Lisp is that it supports metaprogramming very well. This takes lots of forms; compile-time macros are a common one, and they give you enormous flexibility to extend Lisps into domain-specific languages, though they're probably used even more often to hack around inadequately optimized compilers. EVAL answers many requirements for runtime flexibility, though there are times when it is too powerful.

As with serialization and deserialization, these metaprogramming facilities mostly just fall out of the Lisp design; they require minimal or no extra code in a Lisp interpreter.

— ⁂ —

Historically speaking Lisp had a lot of other advantages: for a long time it was the only garbage-collected language, the only dynamically-typed language, the only language with EVAL, the only language with higher-order functions, and so on, and so for many years it was by far the best language for the things that you would do in JS, Python, Ruby, Lua, or OCaml today. JS, Python, Ruby, Lua, and OCaml are better for some of those things, and for many purposes they adequately support the functional, recursive, immutable approach to programming that Lisp pioneered.

Still, it might be easier to learn it in Lisp.

— ⁂ —

Lisp is sort of a minimal core of functional programming. In a decent low-level language like C or Forth, you can build a Lisp with an interactive functional programming environment with dynamic typing, orthogonal serialization and deserialization, an interactive interpreter, garbage collection, and extensive metaprogramming capabilities, in under 1000 lines of code, and the only data structure you need to do it is cons.

(It's not quite as simple as you might think from reading the "Maxwell's Equations of Software" in the Lisp 1.5 manual; those gloss over READ, PRINT, decimal conversion, arithmetic, symbol interning, the garbage collector, and user interaction, so you end up with significantly more code in a low-level language² where you have to implement those. But it's still days of work, not weeks.)

That doesn't mean Lisp is the only way to do functional programming, or even the best one. Quite apart from implementation questions, you might reasonably prefer the ML approach, with its strong static type checking; or the Haskell approach, which also features laziness; or the Clojure approach, which includes first-class persistent finite maps and integer-indexed "vectors"; or again the Clojure approach, where every operation supports ad-hoc polymorphism; or the Tcl approach, where everything is a string; or the Q approach, where your code defines term-rewriting rules rather than functions; or the Prolog approach, where you have not only list processing but lists that can contain uninstantiated logic variables; or the KANREN approach, which generalizes functional programming to full-on relational programming; or the Bicicleta approach, where you program directly in a side-effect-free ς-calculus, overriding methods instead of passing parameters; and many other approaches that haven't been thought of yet.

But if you're thinking of Forth as an alternative to Lisp, or STL vectors as an alternative to Lisp lists, and in general rather than in a particular case, you just haven't understood the Lisp approach to problem solving at all.

And when you do, it's going to be awesome.

______

¹ http://canonical.org/~kragen/sw/dev3/readprint.fs

² https://www.mail-archive.com/kragen-hacks@canonical.org/msg0...

Not sure what you mean by this but perhaps you're referring to how Lisp allows redefining functions on the fly without relinking? That's done by indirect function calls. Every function call in Lisp jumps indirectly through the function's symbol name (if it has one). This incurs a runtime penalty of one extra memory access per function call, but it enables functions to be redefined on the fly without changing any of their callers. Optimization switches exist to get rid of this extra overhead if you need maximum speed in deployed code.