Live data from Hacker News

Writing Lisp is AI resistant and I'm sad

blog.djhaskin.com

71–80 of 104 posts

Re: Writing Lisp is AI resistant and I'm sad

#71

> I wonder what adaptations will be necessary to make AIs work better on Lisp. Some are going to nitpick that Clojure isn't as lispy as, say, Common Lisp but I did experiment with Claude Code CLI and my paid Anthropic subscription (Sonnet 4.6 mostly) and Clojure. It is okay'ish. I got it to write a topological sort and pure (no side effect) functions taking in and returning non-totally-trivial data structures (maps i…

I am lazy: when an LLM messes up parenthesis when working with any Lisp language I just quickly fix the mismatch myself rather than try to fix the tooling.

Re: Writing Lisp is AI resistant and I'm sad

#72
post #56

Earlier quoted context omitted.

Don’t ask the LLM to do that directly: ask it to write a program to answer the question, then have it run the program. It works much better that way.

If the LLM is intelligent, why can’t it figure out on its own that it needs to write a program?

The answer is self-evident.

Re: Writing Lisp is AI resistant and I'm sad

#74
My own experience over the last few months is quite the opposite so it's heartening to see some reputable Lispers reporting the same in the comments here.

Everything in this area is moving so quickly that I haven't yet crystallized my thinking or settled on a working methodology but I am getting a lot of value out of running Claude Code with MCP servers for Common Lisp and Emacs (cl-mcp & emacs-mcp-server). Among other things this certainly helps with the unbalanced parentheses rabbit hole.

Along with that I am showing it plenty of my own Lisp code and encouraging it to adopt my preferred coding style and libraries. It takes a little coaching and reinforcement (recalcitrant intern syndrome) but it learns as it goes. It's really quite a pleasant experience to see it write Lisp as I might have written it.

Re: Writing Lisp is AI resistant and I'm sad

#75
post #65

Earlier quoted context omitted.

I bet it would do much better at hcl (or Starlark, maybe even yaml, something that it has seen plenty of examples of in the wild). This is a weird moment in time where proprietary technology can hurt more than it can help, even if it's superior to what's available in public in principle.

Depends if the AI masters also own said proprietary technology.

Well, GCL is (afaik) a Google technology, and they do have some kind of internal, fine-tuned models just for their stack.

Who owns the tech doesn't matter, what matters is whether there's a set of diverse examples of its use spread around the internet.

Re: Writing Lisp is AI resistant and I'm sad

#76
post #65

Earlier quoted context omitted.

Depends if the AI masters also own said proprietary technology.

Well, GCL is (afaik) a Google technology, and they do have some kind of internal, fine-tuned models just for their stack. Who owns the tech doesn't matter, what matters is whether there's a set of diverse examples of its use spread around the internet.

Yeah it's internal, and we have fine tuned models and more lines of it than you can imagine.

That's the reason I think it honestly depends more on the complexity to understand and the necessity of having a mental model of the code.

Re: Writing Lisp is AI resistant and I'm sad

#77

Earlier quoted context omitted.

Sometimes LLMs astonish me with what the code they can write. Other times I have to laugh or cry. As an example, I asked claude 3.5 back when that was the latest to indent all the code in my file by four more spaces. The file was about 700 lines long. I got a busy spinner for two minutes then it said, "OK, first 50 lines done, now I'll do the rest" and got another busy spinner and it said, "this is taking too long. I…

That's you at the time not knowing LLM fundamentals with regards to context management.

That was me at the time kicking the tires to understand what it was good at or not. If I actually wanted to indent a file by four spaces it would take me less time in my editor than to prompt the LLM to do it, even if the LLM had been capable of it.

Re: Writing Lisp is AI resistant and I'm sad

#78
I leaned on Claude Code quite a bit resurrecting Clojure on Android[0] and got good results with it. Using the Clojure REPL MCP works especially well for about the same reasons I find developing with a REPL myself important: it can query the running program to see how things work, and test implementations with rapid turnaround.

I wasn't sure if I should expect great results relative to more popular languages with more code for the LLM to train on, but it looks like that's either not a big issue, or Clojure is over the popularity threshold for good results. I also previously expected languages with a lot of static guarantees like Rust to lead to consistently better results with LLM coding agents than languages like Clojure which have few, but that's untrue to the point that "bad AI rewrite in Rust" is a meme.

[0] https://github.com/clj-android

Re: Writing Lisp is AI resistant and I'm sad

#79
post #7

Wildly speculating here, but if you buy that human brains have innate / evolved syntactic knowledge, and that this knowledge projects itself as the common syntactic forms across the bulk of human languages, then it’s no surprise that LLMs don’t have particularly deep grooves for s-expressions, regardless of the programming language distribution of the training set.

OK, I'll bite. I want to know more of the reasoning behind this, because I think it implies that S-expressions are alien to the innate/evolved syntactic knowledge in human languages. A lot of American linguistics, like Chomsky's gropings for how to construct universal grammar and deep syntax trees, or the lambda calculus of semantic functions, looks like S-expressions, and I think that's because there was some coordi…

S-expressions are just lists and trees. That’s it. If a language has groups of words and any hierarchy, you can use s-expressions to represent it. Sure, some human languages might be more or less flat and the groups might represent different things, but I don’t see how that prevents s-expressions from being suitable. Greek doesn’t rely on word order nearly as much as English (it does more with suffixes to indicate subject and object, for instance), but all of that can still be represented in s-expressions.

Re: Writing Lisp is AI resistant and I'm sad

#80
I don’t find many issues with Clojure. The main problem is that it sometimes gets the paren balance wrong when it’s proposing an edit. Sometimes it will spin for a bit on that. A harness can help there, I’ve heard, but thus far I’ve just done a quick hand edit each time. I think this has something to do with how Lisps are typically written with all the closing parens on the last line, as opposed to on separate lines like with C. It might also have something to do with how parens and groups of parens are tokenized in the LLM and how edits are communicated (typically line oriented diffs). Regardless, it’s a problem but not a major one.
Post reply on HN