I had AI write Haskell for me and it did that beautifully. I am not sure why would LISP be any sudden.
Writing Lisp is AI resistant and I'm sad
11–20 of 104 posts
Re: Writing Lisp is AI resistant and I'm sad
#12> I wonder what adaptations will be necessary to make AIs work better on Lisp. Some are going to nitpick that Clojure isn't as lispy as, say, Common Lisp but I did experiment with Claude Code CLI and my paid Anthropic subscription (Sonnet 4.6 mostly) and Clojure. It is okay'ish. I got it to write a topological sort and pure (no side effect) functions taking in and returning non-totally-trivial data structures (maps i…
I think you're right. Try asking GPT-5 this:
> Are the parentheses in ((((()))))) balanced?
There was a thread about this the other day [1]. It's the same issue as "count the r's in strawberry." Tokenization makes it hard to count characters. If you put that string into OpenAI's tokenizer, [2] this is how they are grouped:
Token 1: ((((
Token 2: ()))
Token 3: )))
Which of course isn't at all how our minds would group them together in order to keep track of them.
[1] https://news.ycombinator.com/item?id=47615876 [2] https://platform.openai.com/tokenizer
Re: Writing Lisp is AI resistant and I'm sad
#13> I wonder what adaptations will be necessary to make AIs work better on Lisp. Some are going to nitpick that Clojure isn't as lispy as, say, Common Lisp but I did experiment with Claude Code CLI and my paid Anthropic subscription (Sonnet 4.6 mostly) and Clojure. It is okay'ish. I got it to write a topological sort and pure (no side effect) functions taking in and returning non-totally-trivial data structures (maps i…
Sometimes LLMs astonish me with what the code they can write. Other times I have to laugh or cry. As an example, I asked claude 3.5 back when that was the latest to indent all the code in my file by four more spaces. The file was about 700 lines long. I got a busy spinner for two minutes then it said, "OK, first 50 lines done, now I'll do the rest" and got another busy spinner and it said, "this is taking too long. I…
That's you at the time not knowing LLM fundamentals with regards to context management.
Re: Writing Lisp is AI resistant and I'm sad
#14Amusingly, some of the earliest AI research was using Lisp which beget AI winter. Now we’ve come full circle with LLMs that struggles to write valid Lisp. Almost poetic.
Re: Writing Lisp is AI resistant and I'm sad
#15How many closing parentheses are in strawberry.
Re: Writing Lisp is AI resistant and I'm sad
#16I enjoyed reading this. Thank you for sharing.
I learned Common Lisp years ago while working in the AI lab at the University of Toronto, and parts of this article resonated strongly with me.
However, if you abandon the idea of REPL-driven development, then the frontier models from Anthropic and OpenAI are actually very capable of writing Lisp code. They struggle sometimes editing it (messing up parens)), but usually the first pass is pretty good.
I've been on an LLM kick the past few months, and two of my favorite AI-coded (mostly) projects are, interestingly, REPL-focused. icl (https://github.com/atgreen/icl) is a TUI and browser-based front end for your CL REPL designed to make REPL programming for humans more fun, whether you use it stand-alone, or as an Emacs companion. Even more fun is whistler (https://github.com/atgreen/whistler), which allows you to write/compile/load eBPF code in lisp right from your REPL. In this case, the AI wrote the highly optimizing SSA-based compiler from scratch, and it is competitive against (and sometimes beating) clang -O2. I mean... I say the AI wrote it... but I had to tell it what I wanted in some detail. I start every project by generating a PRD, and then having multiple AIs review that until we all agree that it makes sense, is complete enough, and is the right approach to whatever I'm doing.
Re: Writing Lisp is AI resistant and I'm sad
#17Wildly speculating here, but if you buy that human brains have innate / evolved syntactic knowledge, and that this knowledge projects itself as the common syntactic forms across the bulk of human languages, then it’s no surprise that LLMs don’t have particularly deep grooves for s-expressions, regardless of the programming language distribution of the training set.
Is Java or Haskell any closer to human language?
Re: Writing Lisp is AI resistant and I'm sad
#18Amusingly, some of the earliest AI research was using Lisp which beget AI winter. Now we’ve come full circle with LLMs that struggles to write valid Lisp. Almost poetic.
I have a feeling we'll care less about untyped languages going forward as LLMs prototype faster than we do, and fast prototyping was a big reason why we cared about untyped languages.
Re: Writing Lisp is AI resistant and I'm sad
#19Interesting, and not quite my experience. While I do get better agentic coding results for Python projects, I also get good results working with Common Lisp projects. I do have a habit of opening an Emacs buffer and writing a huge prompt with documentation details, sometimes sample code in other languages or if I am hitting APIs I add a working CURL example. For Common Lisp my initial prompts are often huge, but I fi…
Yes, I've also found llms can generate working common lisp code quite well, albeit I've only been solving simple problems.
I haven't tried integrating it into a repl or even command line tools though. The llm can't experience the benefit of a repl so it makes sense it struggled with it and preferred feeing entire programs into sbcl each time.
Re: Writing Lisp is AI resistant and I'm sad
#20Claude has really helped me improve my Emacs config (elisp) substantially, and sometimes even fix issues I've found in packages. My emacs setup is best it has ever been. Can't say it just works and produces the best solution and sometimes it would f** up with closing parens or even make things up (e.g. it suggest load-theme-hook which doesn't exist). But overall, changing things in Emacs and learning elisp is definitely much easier for me (I'm not good with elisp, but pretty good Racket programmer).