How to write Common Lisp in 2017 – an initiation manual

Better exception handling (conditions and restarts), advanced OOP capabilities, proper macros which are convenient to use (pretty much requires a language to be homoiconic), image-based development with ability to hot-swap any code in a running program, including but not limited to full class redefinitions without losing data. All that with resulting code able to achieve close-to-C++ performance thanks to very good (commercial and open-source) compilers. Not to mention the stability of the language thanks to the ANSI standard.

To recap: when you load a (dynamically-linked) C program in Linux or OS X, the OS will create a new process, and map the dynamic linker (ld-linux.so or dyld) into that process's memory. It then transfers control to the dynamic linker. The dynamic linker then loads the rest of the program into memory (from ".so" or ".dylib" or ".dll" files) and links everything together (i.e. resolves symbol addresses) before passing control to the application entry point.

A compiled Lisp system like SBCL works much the same way, except it uses its own linking/compilation machinery. A new process is bootstrapped by running a C program, which loads the Lisp image into memory. The image is compiled code/data containing the Lisp runtime, compiler, and standard library. After the Lisp image is loaded, you've got two things to work with: COMPILE and LOAD. COMPILE takes Lisp source code and compiles it into a FASL (compiled code and data, equivalent to a ".o" file). LOAD functions much like dyld or ld-linux, and loads compiled code and data into the running process. Indeed, in some implementations like ECL, LOAD is just built on dlopen(). The "eventual program" is ultimately built by LOAD-ing the FASL files comprising the program.

In practice, you don't do this manually. Instead, you use something like ASDF: https://common-lisp.net/project/asdf/asdf.html#Defining-syst.... ASDF functions much like 'make' in that you've got a system definition listing the files belonging to your program. Whenever you call ASDF from the REPL, it'll look at what source files need to be (re)-built, compile them to FASLs, and (re)-load them into the running process. In contrast with C (but similarly to Python), Lisp allows code to be loaded into a running image. In the Lisp workflow, you don't restart the program you're writing each time you recompile (although you can, if you want). Instead, you keep it running, and load or re-load code into it as you work on it.

* Why hasn't anyone made a more eye-frendly version of the Common Lisp Hyper Spec ? Having good, easily-browsable documentation is a core-problem.

* The relation between the various native data-types were quite unclear to me.

* dealing with the external world was quite a mess. Many project/libraries implementing only half of something and then got abandoned.

* some libraries had a compatibility matrix... with common lisp implementations. that seemed weird to me.

A friend of mine is working on that as we speak. The CLUS, or Common Lisp UltraSpec:

https://phoe.tymoon.eu/clus/doku.php

That said, CLHS isn't bad - it's lightweight, available off-line, and on-line you can pretty much always find what you're looking for by searching for "clhs [term of interest]".

> some libraries had a compatibility matrix... with common lisp implementations. that seemed weird to me.

Common Lisp has a pretty good standard, but it's a bit old so it didn't predict some things we're currently using, and also left some other things to the implementers. So some features are only available via vendor-specific extensions, which makes it necessary for some projects - especially compatibility libraries - to be tested across plethora of CL implementations.

The flip side is that you have a few commercial and open source implementations to choose from, which is valuable if your project can exploit strengths of a particular one.