Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs pose an interesting problem for DSL designers

kirancodes.me

Post reply on HN
Print view View original

61–70 of 157 posts

Re: LLMs pose an interesting problem for DSL designers

#61

Post by guelo »

TeMPOraL wrote: post #22

On the one hand, this sucks. On the other hand, we're already vacillating along the Pareto frontier of how much we can stuff into code; in fact, most of the criticisms of DSLs are indirectly stating just that. So with LLMs making it easier to project back and forth between how programmer sees the task at hand, and the underlying dumb/straightforward code they ain't gonna read anyway, maybe we'll finally get to the po…

This sounds insightful but I can't make heads or tails of "you cannot optimize for every task and cross-cutting concern at the same time and expect improvement across the board".

Can someone help me out?

Re: LLMs pose an interesting problem for DSL designers

#62

Post by daxfohl »

kibwen wrote: post #46

People often use the analogy of LLMs being to high-level languages what compilers were for assembly languages, and despite being a terrible analogy there's no guarantee it won't eventually be largely true in practice. And if it does come true, consider how the advent of the compiler completely eliminated any incentive to improve the ergonomics or usability of assembly code, which has been and continues to be absolute…

A big difference is that compilers are deterministic, and coders generally don't review and patch the generated assembly. There's little reason to expect that LLMs will ever function like that. It's always going to be a back-and-forth of, "hey LLM code this up", "no, function f isn't quite right; do this instead", etc.

This mimics what you see in, say, Photoshop. You can edit pixels manually, you can use deterministic tools, and you can use AI. If you care about the final result, you're probably going to use all three together.

I don't think we'll ever get to the point where we a-priori present a spec to an LLM and then not even look at the code, i.e. "English as a higher-level coding language". The reason is, code is simply more concise and explicit than trying to explain the logic in English in totality up-front.

For some things where you truly don't care about the details and have lots of flexibility, maybe English-as-code could be used like that, similar to image generation from a description. But I expect for most business-related use cases, the world is going to revolve around actual code for a long time.

Re: LLMs pose an interesting problem for DSL designers

#63

Post by guywithahat »

NathanKP wrote: post #58
guywithahat wrote: post #35

Earlier quoted context omitted.

Skynet will be run on C

This is an interesting idea actually, if we assume a couple things: - There likely won't be one Skynet, but rather multiple AI's, produced by various sponsors, starting out as relatively harmless autonomous agents in corporate competition with each other - AI agents can only inference, then read and write output tokens at a limited rate based on how fast the infrastructure that powers the agent can run In this scenar…

But then we return to the premise of the article, which is that LLM's struggle with higher level code that has fewer examples. C is much more repetitive and has millions of examples from decades of use.

Plus if we assume the bottle neck on skynet is physical materials and not processing power, a system written in C can theoretically always be superior to a system written in another language if we assume infinite time can be spent building it.

Re: LLMs pose an interesting problem for DSL designers

#64

Post by gopiandcode »

furyofantares wrote: post #60

I notice I am confused. > Suddenly the opportunity cost for a DSL has just doubled: in the land of LLMs, a DSL requires not only the investment of build and design the language and tooling itself, but the end users will have to sacrifice the use of LLMs to generate any code for your DSL. I don't think they will. Provide a concise description + examples for your DSL and the LLM will excel at writing within your DSL. A…

Arguably it really depends on your DSL right? If it has a semantics that already lies close to existing programming languages, then I'd agree that a few examples might be sufficient, but what if your particular domain doesn't match as closely?

Examples of domains that might be more challenging to design DSLs for: languages for knitting, non-deterministic languages to represent streaming etc. (i.e https://pldi25.sigplan.org/details/pldi-2025-papers/50/Funct... )

My main concern is that LLMs might excel at the mundane tasks, but struggle at the more exciting advances, and so now the activation energy for coming up with advances DSLs is going to increase and as a result, the field might stagnate.

Re: LLMs pose an interesting problem for DSL designers

#65

Post by outofpaper »

Depending on the size of a DSL all the more recent LLMs can be employed to work with them. LoRA/Finetuning are a heavier option, followed by RAG, and just setting them up as a big system prompt and caching. And once a model is able to work with a DSL tokens used in valuable code creation can dramatically drop.

Re: LLMs pose an interesting problem for DSL designers

#66

Post by kibwen »

Terr_ wrote: post #57
kibwen wrote: post #46

People often use the analogy of LLMs being to high-level languages what compilers were for assembly languages, and despite being a terrible analogy there's no guarantee it won't eventually be largely true in practice. And if it does come true, consider how the advent of the compiler completely eliminated any incentive to improve the ergonomics or usability of assembly code, which has been and continues to be absolute…

I suspect the important sticking point will be reliability. The "incentive" exists because of an high degree of trust, so much so that "junior dev thinks it's a compiler bug" is a kind of joke. If compilers had significant non-deterministic error rates with no reliable fix, that would probably be a rather different timeline.

Yes, that's why I say it's a terrible analogy, but we live in a crap world with a fetish for waste and no incentive for quality, so I fear for a future where every night we just have the server farm spin up a thousand batch jobs to regenerate the entire codebase from scratch and pick out the first candidate that passes today's test suite.

Re: LLMs pose an interesting problem for DSL designers

#67

Post by daxfohl »

furyofantares wrote: post #60

I notice I am confused. > Suddenly the opportunity cost for a DSL has just doubled: in the land of LLMs, a DSL requires not only the investment of build and design the language and tooling itself, but the end users will have to sacrifice the use of LLMs to generate any code for your DSL. I don't think they will. Provide a concise description + examples for your DSL and the LLM will excel at writing within your DSL. A…

This is somewhat my take too. The way most vibe coding happens right now does create a lot of duplication because it's cheap and easy for LLMs to do. But eventually as the things we do with coding assistants become more complex, they're not necessarily going to be able to deal with huge swaths of duplicate code any better than humans are. Given their limited context size, having a DSL that allows them to fit more logic into their context with fewer tokens, we could conceivably see the importance of DSLs start to increase rather than decrease.

Re: LLMs pose an interesting problem for DSL designers

#68

Post by neilv »

furyofantares wrote: post #60

I notice I am confused. > Suddenly the opportunity cost for a DSL has just doubled: in the land of LLMs, a DSL requires not only the investment of build and design the language and tooling itself, but the end users will have to sacrifice the use of LLMs to generate any code for your DSL. I don't think they will. Provide a concise description + examples for your DSL and the LLM will excel at writing within your DSL. A…

I was about to paste the same sentence, and say much the same thing in response.

To add to that... One limitation of LLM for a new DSL is that the LLM may be less likely to directly plagiarize from open source code. That could be a feature.

Another feature could be users doing their own work, and doing a better job of it, instead of "cheating on their homework" with AI slop and plagiarism, whether for school or in the workplace.

Re: LLMs pose an interesting problem for DSL designers

#69

Post by TimTheTinker »

NathanKP wrote: post #32
gopiandcode wrote: post #23

Earlier quoted context omitted.

Oh that's a great blog post and a very interesting point. Yep, I hadn't considered how LLMs would affect frameworks in existing languages, but it makes sense that there's a very similar effect of reinforcing the incumbents and stifling innovation. I'd argue that the problem of solving this effect in DSLs might be a bit harder than for frameworks, because DSLs can have wildly different semantics (imagine for example a…

I think there is a lot of overlap between DSL's and frameworks, and most frameworks contain some form of DSL in them. What matters most of all is whether the DSL is written in semantically meaningful tokens. Two extremes as examples: Regex is a DSL that is not written in tokens that have inherent semantic meaning. LLM's can only understand Regex by virtue of the fact that it has been around for a long time and there…

> there is a lot of overlap between DSL's and frameworks

Not just frameworks, but libraries also. Interacting with some of the most expressive libraries is often akin to working with a DSL.

In fact, the paradigms of some libraries required such expressiveness that they spawned their own in-language DSLs, like JSX for React, or LINQ expressions in C#. These are arguably the most successful DSLs out there.

Re: LLMs pose an interesting problem for DSL designers

#70

Post by nylonstrung »

amterp wrote: post #55

I've been working on a programming language for about a year which aims to replace Bash for scripting but is far closer to Python. It's something I hope to see used by many other people in the future, but a very common objection I hear from people I pitch it to is "yeah but an LLM could just generate me a Python script to do this, sure it might be uglier, twice as long, and not work quite as well, but it saved me fro…

I love your idea.

LLMs are surprisingly bad at bash and apparently very bad at Powershell

Pythonic shell scripting is well suited to their language biases right now

Post reply on HN