LLMs pose an interesting problem for DSL designers • Hacker News Board

Re: LLMs pose an interesting problem for DSL designers

#31

Post by scelerat » Tue, Jun 17, 2025, 9:03 PM UTC

NathanKP wrote: post #12

Good to see more people talking about this. I wrote about this about 6 months ago, when I first noticed how LLM usage is pushing a lot of people back towards older programming languages, older frameworks, and more basic designs: https://nathanpeck.com/how-llms-of-today-are-secretly-shapin... To be honest I don't think this is necessarily a bad thing, but it does mean that there is a stifling effect on fresh new DSL's…

Linguistics and history of language folk: isn't there an observed slowdown of evolution of spoken language as the printing press becomes widespread? Also, "international english"?

Is this an observation of a similar phenomenon?

Re: LLMs pose an interesting problem for DSL designers

#32

Post by NathanKP » Tue, Jun 17, 2025, 9:05 PM UTC

gopiandcode wrote: post #23

NathanKP wrote: post #12

Good to see more people talking about this. I wrote about this about 6 months ago, when I first noticed how LLM usage is pushing a lot of people back towards older programming languages, older frameworks, and more basic designs: https://nathanpeck.com/how-llms-of-today-are-secretly-shapin... To be honest I don't think this is necessarily a bad thing, but it does mean that there is a stifling effect on fresh new DSL's…

Oh that's a great blog post and a very interesting point. Yep, I hadn't considered how LLMs would affect frameworks in existing languages, but it makes sense that there's a very similar effect of reinforcing the incumbents and stifling innovation. I'd argue that the problem of solving this effect in DSLs might be a bit harder than for frameworks, because DSLs can have wildly different semantics (imagine for example a…

I think there is a lot of overlap between DSL's and frameworks, and most frameworks contain some form of DSL in them.

What matters most of all is whether the DSL is written in semantically meaningful tokens. Two extremes as examples:

Regex is a DSL that is not written in tokens that have inherent semantic meaning. LLM's can only understand Regex by virtue of the fact that it has been around for a long time and there are millions of examples for the LLM to work from. And even then LLM's still struggle with reading and writing Regex.

Tailwind is an example of a DSL is that is very semantically rich. When an LLM sees: `class="text-3xl font-bold underline"` it pretty much knows what that means out of the box, just like a human does.

Basically, a fresh new DSL can succeed much faster if it is closer to Tailwind than it is to Regex. The other side of DSL's is that they tend to be concise, and that can actually be a great thing for LLM's: more concise, equals less tokens, equals faster coding agents and faster responses from prompts. But too much conciseness (in the manner of Regex), leads to semantically confusing syntax, and then LLM's struggle.

Re: LLMs pose an interesting problem for DSL designers

#33

Post by oleks » Tue, Jun 17, 2025, 9:06 PM UTC

Using an LLM to generate code is not an easily traceable and explainable process. Using a DSL to same ends is. PL research has yet to meet explainability in AI head on.

Re: LLMs pose an interesting problem for DSL designers

#34

Post by mplanchard » Tue, Jun 17, 2025, 9:14 PM UTC

jbellis wrote: post #11

I guess if you love writing DSLs this is an unfortunate development, but for me it's more of a glass half full: I can have the AI spit out boilerplate I need to solve a problem instead of spending a week building a one-off DSL compiler.

The benefit of (some) DSLs is that they make invalid states unrepresentable, which isn't possible with the entire surface-area of a programming language at your (or the LLM's) disposal.

Re: LLMs pose an interesting problem for DSL designers

#35

Post by guywithahat » Tue, Jun 17, 2025, 9:15 PM UTC

NathanKP wrote: post #12

Good to see more people talking about this. I wrote about this about 6 months ago, when I first noticed how LLM usage is pushing a lot of people back towards older programming languages, older frameworks, and more basic designs: https://nathanpeck.com/how-llms-of-today-are-secretly-shapin... To be honest I don't think this is necessarily a bad thing, but it does mean that there is a stifling effect on fresh new DSL's…

Skynet will be run on C

Re: LLMs pose an interesting problem for DSL designers

#36

Post by duskwuff » Tue, Jun 17, 2025, 9:17 PM UTC

arciini wrote: post #10

averkepasa wrote: post #3

Python increase in Tiobe index is scary: https://www.tiobe.com/tiobe-index/

I was pretty convinced by this article to not use TIOBE as a mark of a language's popularity: https://nindalf.com/posts/stop-citing-tiobe/ Its primary point is that TIOBE is based on *number* of search results on a weighted list of search engines, not actual usage in Github, search volume, job listings, or any of the other number of signals you'd expect a popularity index to use. It could easily be indicating that Py…

It's even worse than "Stop Citing TIOBE" makes it sound. The TIOBE rank is based on the number of hits reported from "25 search engines", which amount to:

  1) Google, on nine different TLDs
  2) Amazon, on seven TLDs
  3) EBay, on two TLDs
  4) wikipedia.org (which ends up defaulting to the English Wikipedia)
  5) microsoft.com (which only searches Microsoft documentation)
  6) sharepoint.com (similarly, Microsoft 365 documentation)
  7) rakuten.co.jp
  8) walmart.com

Only one of these is actually a web search engine; there are actually more shopping web sites included than search engines. Bing, and its various mirrors, were apparently all excluded because they don't display the number of hits on the result page.

And yes, this only adds up to 23. The TIOBE web site doesn't explain the discrepancy.

Re: LLMs pose an interesting problem for DSL designers

#37

Post by prats226 » Tue, Jun 17, 2025, 9:18 PM UTC

For LLMs, programming languages are basically additional languages that we speak. So how it handles low-resource programming languages is same as how it handles speaking languages with less contribution in training data?

DSL's would be even harder for LLM's to get right in that case compared to the low-resource language itself

Re: LLMs pose an interesting problem for DSL designers

#38

Post by winter_blue » Tue, Jun 17, 2025, 9:23 PM UTC

NathanKP wrote: post #12

Good to see more people talking about this. I wrote about this about 6 months ago, when I first noticed how LLM usage is pushing a lot of people back towards older programming languages, older frameworks, and more basic designs: https://nathanpeck.com/how-llms-of-today-are-secretly-shapin... To be honest I don't think this is necessarily a bad thing, but it does mean that there is a stifling effect on fresh new DSL's…

I think perhaps automatic translators might help mitigate some of this.

Even perhaps training a separate new neural network to translate from Python/Java/etc to your new language.

Re: LLMs pose an interesting problem for DSL designers

#39

Post by rramon » Tue, Jun 17, 2025, 9:23 PM UTC

Sooner or later languages are going to become 3d diagram to code IDEs where you just zoom in to get to lower levels of detail.

Re: LLMs pose an interesting problem for DSL designers

#40

Post by Lerc » Tue, Jun 17, 2025, 9:24 PM UTC

TimTheTinker wrote: post #5

darepublic wrote: post #2

I recently had to work with the robot framework DSL. Not a fan. I hardly think it's any more readable to a business user than imperative code either. Every DSL is another API to learn and usually full of gotchas. Intuitiveness is in the eye of the beholder . The approach I would take is transpiling from imperative code to a natural language explanation of what is being tested, with configuration around aliases and th…

DSLs are not all created equal. Consider MiniZinc. This DSL is super cool and useful for writing constraint-solving problems once and running them through any number of different backend solvers. A lot of intermediate languages and bytecode (including LLVM itself) are very useful DSLs for representing low-level operations using a well-defined set of primitives. Codegen DSLs are also amazing for some applications, esp…

I think Sturgeon's law might be the problem for DSLs. Enabling a proliferation of languages targeted at a specific purpose creates a lot of new things with a 50% chance of the quality being below the median. Using a general purpose language involves selecting (or more usually relying on someone else's earlier selection) one of many, that selection process is inherently biased towards the better languages.

Put differently, The languages people actually use had people who decided to use them, they picked the best ones. Making something new, you compete against the best, not the average. That's not to say that it can't be done, but it's not easy.