Lisp Machine Manual (1984)

>However, many day-to-day developers (...) believe that Lisp's syntax creates nigh-unreadable programs (...) Very smart people will disagree with that opinion (...) it's a fact many people believe (...) trying to convince people that Lisp is "advanced", even though it doesn't support basic capabilities like infix out of the box, is a losing battle.

But note the use of the "believe" word. As you point out, it's all beliefs. There are also programmers that know Javascript but believe that programming in Python must be hard (true story). Beliefs.

Then there are people, like myself, that once they use Lisp enough, feel that almost any other language would be torture compared to used Lisp. And i love the parentheses.

The problem, and this is a problem in all aspects of programming, is that people like to "believe" things about something they don't know enough to have a reliable opinion of.

So the parent question was something like "if Lisp is so good, why isn't popular?". Are good things always popular? Are popular things always good?

If really really good music must be popular as well, then you would turn on the radio (or MTV, or VH1 on the TV) and all music would be Bach, Beethoven, Debussy, Stravinsky... you get the idea. Or if you are a rock or jazz fan, include in that list the very best of the genre (i.e. John Coltrane, etc.)

If popular music must be good as well then perhaps Justin Bieber and Shakira are masters of music and we should revere them.

In my view, the reason Lisp isn't popular today starts with the Unix becoming popular and mainstream. Unix ran fast on cheap hardware (minicomputers, PDP-x). Little by little C became accepted as a standard for systems programming.

So by early 1980s C becomes mainstream; operating system APIs are then majorly written in C, even more so in the 90s (i.e. Windows API), thus the next popular languages are modeled on C: C++, and then Java which holds the same syntax and is quite influenced by it. Afterwards cue all the other languages made for overcoming some of Java's drawbacks, like C# and others.

Meanwhile Lisp was simply let behind in terms of popularity. But at the same time during the 90s and 00s it was used for important, complex stuff: air travel reservation, aerodynamics simulation, credit card transaction checking, auto-piloting the deep space one spaceship for days...

It wasn't forgotten, since Lisp never stalled from evolution, it has kept evolving since 1958, it has never stopped; the enthusiasts are still creating more good tooling, more libraries, even really nice stuff like Portacle that makes installing a complete Lisp programming environment a one-click operation.

So, it all depends on beliefs of the people versus actual usage. Some of the beliefs and myths held by people, besides the parentheses thing, are for example (my example is biased to Common Lisp)

    1. Lisp is slow

-> Lisp is generally at the same speed level as Java on current Oracle JVM, and can be done, with tricks, to perform in the same speed as C or Fortran. In any case it is still 10x to 100x faster than most other popular languages out there like Ruby or Python.

    2. There's no good IDE available

-> You will find that many Lispers love using SLIME on Emacs for hacking on Lisp. It provides high productivity.

    3. Since there are few users, there is little 
    support (forum topics covering your issues) or manuals or guides.

-> There are a ton of books on Lisp and most of them are very good, some of them outstandingly good. The reference documentation (CLHS) is really well written. Since this is a language that has endured for decades, the amount of articles or posts written about doing X in Lisp is enormous.

    4. There aren't enough libraries

-> Not many libraries if we compare with say, Java libs; however so far I've found everything i need for my purposes. And while i wouldn't bother to dive into a Java or C++ library to modify it or understand how it works, usually Lisp libraries (actually "systems" made of "packages") are easy to understand and modify at your will. It also helps that usually Lisp requires few lines of code for achieving something, compared to C++ or Java.

    5. Lisp doesn't have static typing.

It does have it. But it is optional and depends on the implementation. However the most popular implementation, SBCL, has very good support by it. On the other hand, many Lispers use it for performance reasons rather than safety reasons. Lisp is a very strongly typed language.

    6. Lisp doesn't support 

In theory, Lisp can support any programming paradigm, and in practice it supports most programming paradigms out there.

    7. Lisp produces unreadable, garbled code.

Some of the cleanest code i've seen in my whole 20+ years of programming was Lisp libraries.

    8. Lisp can't work for complex or large codebases
    since it's so hard to read

As pointed out, very complex, millions-of-LOC Lisp systems have been created successfully, some of them used today (like aero simulation software).

-----

Now, something that is true is that Lisp is hard to learn: If you use Lisp, a very powerful language, then you ought to use the full power of Lisp. And for this, you would need to be familiar and handy with all of the most popular programming paradigms (procedural, imperative, functional, OOP, declarative), AND you would also need to understand and know how to take advantage of metaprogramming (macro programming). For many programmers, this represents a challenge. Even if they already know OOP, they don't really know OOP the Lisp way, since the object system (CLOS) is different and more powerful than the typical OOP facilities provided by Java or C++.

But those "pre-requisites" are in fact the "pre-requisites" for any programmer to become a really good programmer, regardless of the language....

Once these "pre-requisites" are met, learning Lisp itself is rather easy, since the syntax is easy and most features are orthogonal to the language.

A not-so-close approximation to using a Lisp Machine these days is to use the GuixSD distro. It features a package manager, init system, and initial RAM disk all written in Scheme. Combine that with Emacs, the guix-emacs extension, and, if you are so inclined, a Lispy WM like StumpWM and you have a system where a large number of critical components are implemented in Lisp.

If an excellent language doomed to obscurity is one that offers "a lot of features which require a lot of discipline by the programmer," and for the mediocre-only mainstream languages "all their features are comprehensible by mediocre developers," then I think C++ is pretty clearly a counter-example to that, not a supporting example.

I am well aware of the architecture of the MIT CADR, LMI Lambda and TI Explorer, Symbolics lispms less so but they are not the subject of this thread. I have written CADR microcode recently.

None of the features you list are constrained by the architecture of the hardware, they are just conventions of the software VM running on it. Would you suggest that the X86 is a tagged CPU architecture just because SBCL or a JVM use tags ?

>Page 5: 1 GC bit, 1 User bit, 2 cdr code bits, 5 bits data type, 23 bit pointer.

This is not true for the software that matches this version of the manual. System 99 used 25 bit pointers, there wasn't a GC or user bit. The change from the earlier word format was possible because this was not fixed in hardware.

The CADR microinstruction set is load/store with regular opcode fields, it is very much like an early RISC.

If the X86 would provide SBCL with such instructions and data, it would be a tagged architecture, but it doesn't. The SBCL compiler outputs conventional X86 instructions.

The Lisp Machine compiler OTOH generates instructions for a mostly stack machine, which runs on the CPU in microcode.