91–100 of 154 posts
Earlier quoted context omitted.
Parsing expression grammars (pegs) are usually IME more maintainable long term, partially just because of much more testable and composable they are
I suspect but am not sure that PEGs cannot do negative nor positive lookbehind, but it is not a very used feature.
Earlier quoted context omitted.
Especially in Python, where it can be hard to tell if something is being imported purely for side effects.
That does happen, but not frequently in the common libraries that are going to be in public training data. Is there a top 100 package that does something funny on import?
import matplotlib
import PIL
im = PIL.Image.open("foo.png")Languages that are difficult for LLM to read & write are also difficult for the general public. These languages have always had poor uptake and never reach critical mass, or are eventually replaced by better languages. Language designers would be smart to recognize this fact and favor making their languages more LLM friendly. This should also make them more human friendly.
I don't find this to be true. There are languages that are difficult to wrap your head around initially, but that turn out to be delightfully productive with a few adjustments to the mental model. Adjustments that LLMs don't have the training data for. That says nothing about the language at all, actually. Just that it's small and easily confused for something more idiomatic to a newbie.
Methinks if you want job-security in a post—LLM-zero-shot-app-generator world, get into Lisp or Haskell; People that know only Node+React from YouTube learn-2-code tutorials are going to be slaughtered.
I just had an idea: an app/GUI/backend framework for Lisp or Haskell (with an S-expression lib) where everything is structurally inverted so it must be manually run through foldr - behold: an LLM-resistant (if not LLM-proof?) dev environment!
Earlier quoted context omitted.
A human can deal with right-to-left evaluation by moving the cursor around to write in that direction. An LLM can’t do that on its own. A human given an editor that can only append would struggle too.
Idea: feed the language model the parse tree instead of the textual sequence.
Cognitive load in LLMs: When LLMs are faced with syntactic complexity (Lisp/J parentheses/RL-NOP), distractors (cat facts), or unfamiliar paradigms (right-to-left evaluation), the model’s performance degrades because its "attention bandwidth" is split or overwhelmed. This mirrors human cognitive overload. My question: is there a way to reduce cognitive load in LLMs?, one solution seems to be process the input and out…
Isn't the whole idea of Lisp that there is _no_ syntactic complexity? Lisp programs are roughly a serialized AST.
That's like reading binary for humans. 1s and 0s may be the simplest possible representation of information, but not the one your wet neural network recognizes.
Earlier quoted context omitted.
> I did some experiments to find what a programming language would look like, instead of e.g. python, if it were designed to be written and edited by an LLM. Did your experiment consist of asking an LLM to design a programming language for itself?
Yes. ChatGPT 4 and Claude 3.7. They led me to similar conclusions, but they produced very different syntax, which led me to believe that they were not just regurgitating from a common source.
Earlier quoted context omitted.
That does happen, but not frequently in the common libraries that are going to be in public training data. Is there a top 100 package that does something funny on import?
I'd be surprised. That kind of thing was en vogue for a little while in the early 2000s before cooler heads prevailed, but now people will understandably shout at you for changing behavior in someone else's code. My guess is that nearly all packages that did this sort of thing were left behind in the 2-to-3 migration, which a lot of us used as the excuse for a clean break.
But I agree that observable side effects are generally pretty rare. And apparently, both libraries are not even in the top 100 packages, depending on how you count. It looks like those spots are all taken by libraries used in uncached, wasteful CI workflows: https://hugovk.github.io/top-pypi-packages/
Earlier quoted context omitted.
[deleted]
I'm not disputing that LLMs are bad for Lisp code, I'm just saying I don't think "syntactic complexity" is a correct explanation for why that is.
Earlier quoted context omitted.
I think long term LLMs should directly generate Abstract Syntax Trees. But this is hard now because all the training data is text code.
There's a fair amount of experimental work happening trying different parsing and resolution procedures such that the training data reflects an AST and or predicts nodes in an AST as an in-filling capability.