> Lisp has had decades to break out of its niche if it delivered a really advantageous solution, but somehow that never happened.
I think a huge part of it is that it is not immediately obvious that one needs what Lisp offers, and by the time the system has grown to the extent that the need is obvious, it has also grown to the extent that one no longer sees the fores for the trees. One doesn’t think ‘oh man, I need garbage collection’; one thinks, ‘oh man, I need to manage malloc and free better!’. One doesn’t think, ‘oh man, dynamic scope would really fit this problem well’; one thinks ‘oh man, I need dependency injection.’ Peter Norvig famously noted that 16 of the original 23 design patterns were invisible or simpler in dynamic languages such as Lisp†. Heck, there was a time when one couldn’t rely on recursion, or even conditionals! But the programmer who has managed to get stuff done without recursion, or without conditionals, or without macros doesn’t really see the point. He’s even worried: those things may add too much expressivity to the language. Why, folks could write unmaintainable code with them!
Of course, folks write unmaintainable code without them, too …
Anyway, I think a huge issue is one of education and experience. Ours is a massively growing field. The vast majority of folks are juniors, and don’t know any better; a portion of their education was miseducation. The seniors often have one year of experience, twenty times (rather than twenty years of experience). Objective standards are rare to nonexistent. Norms and standards are absent.
But yeah, when I’m working on a large project in a language other than Lisp, I often think, ‘man, this would be so much easier in Lisp!’ or even ‘man, this would be practical in Lisp!’ (because anything is possible in a Turing-complete language …).
†: https://norvig.com/design-patterns/ppframe.htm