Live data from Hacker News

LLMs pose an interesting problem for DSL designers

kirancodes.me

51–60 of 157 posts

Re: LLMs pose an interesting problem for DSL designers

#51
post #21

Programming languages researchers and designers labor under the mistaken assumption that programming practitioners--people who are writing programs to solve problems--actually want "a language with a syntax and semantics tailored for a specific domain", or any really fancy language features at all. I say this from the perspective of someone who nearly became a PL researcher myself. I could easily have decided to stud…

Hmm. Go was written to be easier for junior developers to use, and to be maintainable "in the large".

I wonder if we need a language designed to be easier for an AI to reason about, or easier for a human to see the AI's mistakes.

Re: LLMs pose an interesting problem for DSL designers

#52
post #50
post #21

Programming languages researchers and designers labor under the mistaken assumption that programming practitioners--people who are writing programs to solve problems--actually want "a language with a syntax and semantics tailored for a specific domain", or any really fancy language features at all. I say this from the perspective of someone who nearly became a PL researcher myself. I could easily have decided to stud…

30 lines are always going to be easier to read/write/debug than 3000 lines, so it'll probably remain easier (for both humans and machines) to write correct code in languages that make it possible to express ideas concisely and elegantly.

From elsewhere on the front page: "given choice between complexity or one on one against t-rex, grug take t-rex: at least grug see t-rex".

Re: LLMs pose an interesting problem for DSL designers

#53
I've been drafting a blog post on this as well. My take is that programming langauges largely evolve around "human" ergonomics and solve for "humans writing the code", but that can result in code that is too abstract and non-performant. I think where LLMs will succeed (more) is in writing _very dumb verbose code_ that can be easily optimized by the compiler.

What humans look at and what an AI looks at right now are similar only by circumstance, and what I sort of expect is that you start seeing something more like a "structure editor" that expresses underlying "dumb" code in a more abstract way such that humans can refactor it effectively, but what the human sees/edits isn't literally what the code "is".

IDK it's not written yet but when it is it will be here: https://kylekukshtel.com/llms-programming-language-design

Re: LLMs pose an interesting problem for DSL designers

#54
Coincidentally, I released a DSL last week called Hypershell [1], a Rust-based domain-specific language for shell scripting at the type level. While writing the blog post, I found myself wondering: will this kind of DSL be easier for LLMs to use than for humans?

In an initial experiment, I found that LLMs could translate familiar shell scripting concepts into Hypershell syntax reasonably well. More interestingly, they were able to fix common issues like type mismatches, especially when given light guidance or examples. That’s a big deal, because, like many embedded DSLs, Hypershell produces verbose and noisy compiler errors. Surprisingly, the LLM could often identify the underlying cause hidden in that mess and make the right correction.

This opens up a compelling possibility: LLMs could help bridge the usability gap that often prevents embedded DSLs from being more widely adopted. Debuggability is often the Achilles' heel of such languages, and LLMs seem capable of mitigating that, at least in simple cases.

More broadly, I think DSLs are poised to play a much larger role in AI-assisted development. They can be designed to sit closer to natural language while retaining strong domain-specific semantics. And LLMs appear to pick them up quickly, as long as they're given the right examples or docs to work with.

[1] https://contextgeneric.dev/blog/hypershell-release/

Re: LLMs pose an interesting problem for DSL designers

#55
I've been working on a programming language for about a year which aims to replace Bash for scripting but is far closer to Python. It's something I hope to see used by many other people in the future, but a very common objection I hear from people I pitch it to is "yeah but an LLM could just generate me a Python script to do this, sure it might be uglier, twice as long, and not work quite as well, but it saved me from learning a new language and is probably fine", to which I have lots of counters on why that's a flawed argument, but it still demonstrates what I think is an increase in people's skepticism towards new languages which will contribute to the stagnation the author is talking about. If you're writing a new language, it's demotivating to see people's receptiveness to something new diminish.

I don't blame anyone in the picture, I don't disagree that time saved with LLMs can be well worth it, but it still is a topic I think we in the PL community need to wrestle more with.

Re: LLMs pose an interesting problem for DSL designers

#56
post #21

Programming languages researchers and designers labor under the mistaken assumption that programming practitioners--people who are writing programs to solve problems--actually want "a language with a syntax and semantics tailored for a specific domain", or any really fancy language features at all. I say this from the perspective of someone who nearly became a PL researcher myself. I could easily have decided to stud…

That sounds grim, but not implausible. Domain-specific languages seemed like a significant improvement over general-purpose languages plus libraries. But now that we have a tool that lets you make Jazz Hands at your computer and have it spit out something that does most of what you want, do they really help?

Maybe some boring, kind-of-consistent language like C, Python, or Go is good enough. An LLM spits out a pile of code in one or more of them that does most of what you want, and you can fix it because it's less opaque than assembly. It doesn't sound like a job I'd want, but maybe that's just the way things will go.

Re: LLMs pose an interesting problem for DSL designers

#57
post #46

People often use the analogy of LLMs being to high-level languages what compilers were for assembly languages, and despite being a terrible analogy there's no guarantee it won't eventually be largely true in practice. And if it does come true, consider how the advent of the compiler completely eliminated any incentive to improve the ergonomics or usability of assembly code, which has been and continues to be absolute…

I suspect the important sticking point will be reliability. The "incentive" exists because of an high degree of trust, so much so that "junior dev thinks it's a compiler bug" is a kind of joke.

If compilers had significant non-deterministic error rates with no reliable fix, that would probably be a rather different timeline.

Re: LLMs pose an interesting problem for DSL designers

#58

Good to see more people talking about this. I wrote about this about 6 months ago, when I first noticed how LLM usage is pushing a lot of people back towards older programming languages, older frameworks, and more basic designs: https://nathanpeck.com/how-llms-of-today-are-secretly-shapin... To be honest I don't think this is necessarily a bad thing, but it does mean that there is a stifling effect on fresh new DSL's…

Skynet will be run on C

This is an interesting idea actually, if we assume a couple things:

- There likely won't be one Skynet, but rather multiple AI's, produced by various sponsors, starting out as relatively harmless autonomous agents in corporate competition with each other

- AI agents can only inference, then read and write output tokens at a limited rate based on how fast the infrastructure that powers the agent can run

In this scenario a "Skynet" AI writing code in C might lose to an AI writing code in a higher level language, just because of the lost time spent writing the tokens for all that verbose C boilerplate and memory management bits for C. The AI agent that is capable of "thinking" in a higher level DSL is able to take shortcuts that let it implement things faster, with fewer tokens.

Re: LLMs pose an interesting problem for DSL designers

#59

I recently had to work with the robot framework DSL. Not a fan. I hardly think it's any more readable to a business user than imperative code either. Every DSL is another API to learn and usually full of gotchas. Intuitiveness is in the eye of the beholder . The approach I would take is transpiling from imperative code to a natural language explanation of what is being tested, with configuration around aliases and th…

I'm not skeptical about DSLs in general, but I agree with you on robot framework. I think it has a few good points like how it formats its HTML output is mostly nice, but how I'm not happy with how tags on test cases work and actually writing anything that's non-trivial is frustrating. It's easy to write Python extensions though so that's where I ended put basically all of logic that wasn't the "business logic" of the tests. I think that's generally what you're supposed to do, but at that point, it seems better to write it all in Python or the language of your choice.

Re: LLMs pose an interesting problem for DSL designers

#60
I notice I am confused.

> Suddenly the opportunity cost for a DSL has just doubled: in the land of LLMs, a DSL requires not only the investment of build and design the language and tooling itself, but the end users will have to sacrifice the use of LLMs to generate any code for your DSL.

I don't think they will. Provide a concise description + examples for your DSL and the LLM will excel at writing within your DSL. Agents even moreso if you can provide errors. I mean, I guess the article kinda goes in that direction.

But also authoring DSLs is something LLMs can assist with better than most programming tasks. LLMs are pretty great at producing code that's largely just a data pipeline.

Post reply on HN