Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs pose an interesting problem for DSL designers

kirancodes.me

Post reply on HN
Print view View original

71–80 of 157 posts

Re: LLMs pose an interesting problem for DSL designers

#71

Post by crq-yml »

NathanKP wrote: post #12

Good to see more people talking about this. I wrote about this about 6 months ago, when I first noticed how LLM usage is pushing a lot of people back towards older programming languages, older frameworks, and more basic designs: https://nathanpeck.com/how-llms-of-today-are-secretly-shapin... To be honest I don't think this is necessarily a bad thing, but it does mean that there is a stifling effect on fresh new DSL's…

I think it's healthy, because it creates an undercurrent against building a higher abstraction tower. That's been a major issue: we make the stack deeper and build more of a "Swiss Army Knife" language because it lets us address something local to us, and in exchange it creates a Conway's Law problem for someone else later when they have to decipher generational "lava layers" as the trends of the marketplace shift and one new thing is abandoned for another.

The new way would be to build a disposable jig instead of a Swiss Army Knife: The LLM can be prompted into being enough of a DSL that you can stand up some placeholder code with it, supplemented with key elements that need a senior dev's touch.

The resulting code will look primitive and behave in primitive ways, which at the outset creates a myriad of inconsistency, but is OK for maintenance over the long run: primitive code is easy to "harvest" into abstract code, the reverse is not so simple.

Re: LLMs pose an interesting problem for DSL designers

#72

Post by AlotOfReading »

NathanKP wrote: post #32
gopiandcode wrote: post #23

Earlier quoted context omitted.

Oh that's a great blog post and a very interesting point. Yep, I hadn't considered how LLMs would affect frameworks in existing languages, but it makes sense that there's a very similar effect of reinforcing the incumbents and stifling innovation. I'd argue that the problem of solving this effect in DSLs might be a bit harder than for frameworks, because DSLs can have wildly different semantics (imagine for example a…

I think there is a lot of overlap between DSL's and frameworks, and most frameworks contain some form of DSL in them. What matters most of all is whether the DSL is written in semantically meaningful tokens. Two extremes as examples: Regex is a DSL that is not written in tokens that have inherent semantic meaning. LLM's can only understand Regex by virtue of the fact that it has been around for a long time and there…

Knowing just what's going on in the existing text isn't the whole problem in navigating a DSL. You have to be able to predict new things based on the patterns in existing text.

Let's say you want to generate differently sized text here. An LLM will have ingested lots of text talking about clothing size and tailwind text sizes vaguely follow that pattern. Maybe it generates text-medium as a guess instead of the irregular text-base, or extends the numeric pattern down into text-2xs.

Re: LLMs pose an interesting problem for DSL designers

#73

Post by aaronvg »

We're making a prompting DSL (BAML https://github.com/BoundaryML/baml) and what we've found is that all the syntax rules can easily be encoded into a Cursor Rules file, which we find LLMs can follow nicely. DSLs are simple by nature so there's not too many rules to define.

Here's the cursor rules file we give folks: gist.github.com/aaronvg/b4f590f59b13dcfd79721239128ec208

Re: LLMs pose an interesting problem for DSL designers

#74

Post by wolpoli »

I think designers will end up needing to work with an existing LLM or even provide their own LLM to get developers to adopt a new language/library/feature. The market has been heading in that direction for a while now, with developers expecting official docs and tutorials.

Re: LLMs pose an interesting problem for DSL designers

#75

Post by freedomben »

crq-yml wrote: post #71
NathanKP wrote: post #12

Good to see more people talking about this. I wrote about this about 6 months ago, when I first noticed how LLM usage is pushing a lot of people back towards older programming languages, older frameworks, and more basic designs: https://nathanpeck.com/how-llms-of-today-are-secretly-shapin... To be honest I don't think this is necessarily a bad thing, but it does mean that there is a stifling effect on fresh new DSL's…

I think it's healthy, because it creates an undercurrent against building a higher abstraction tower. That's been a major issue: we make the stack deeper and build more of a "Swiss Army Knife" language because it lets us address something local to us, and in exchange it creates a Conway's Law problem for someone else later when they have to decipher generational "lava layers" as the trends of the marketplace shift an…

I think this depends a lot on the stack. for stacks like elixir and Phoenix, imho the extraction layer is about perfect. For anyone in the Java world, however, what you say is absolutely true. Having worked in a number of different stacks, I think that some ecosystems have a huge tolerance for abstraction layers, which is a net negative for them. I would sure hate to see AI decimate something like elixir and Phoenix though

Re: LLMs pose an interesting problem for DSL designers

#76

Post by freedomben »

guywithahat wrote: post #35
NathanKP wrote: post #12

Good to see more people talking about this. I wrote about this about 6 months ago, when I first noticed how LLM usage is pushing a lot of people back towards older programming languages, older frameworks, and more basic designs: https://nathanpeck.com/how-llms-of-today-are-secretly-shapin... To be honest I don't think this is necessarily a bad thing, but it does mean that there is a stifling effect on fresh new DSL's…

Skynet will be run on C

Because skynet knows what's up. Viva la skynet

Re: LLMs pose an interesting problem for DSL designers

#77

Post by freedomben »

wolpoli wrote: post #74

I think designers will end up needing to work with an existing LLM or even provide their own LLM to get developers to adopt a new language/library/feature. The market has been heading in that direction for a while now, with developers expecting official docs and tutorials.

Interesting, I tend to think design is a dying job now. It's not that I don't see value in designers, but if I am being honest, when they aren't advocating for a complete UI refresh, there isn't a whole lot for them to do. IMHO this is why we see all the major apps refreshing their UIs every 9 to 12 months. It is unnecessary and aggravating to users, and if we can get away from that I think we would be better off.

Re: LLMs pose an interesting problem for DSL designers

#78

Post by freedomben »

ackfoobar wrote: post #49
jp57 wrote: post #21

Programming languages researchers and designers labor under the mistaken assumption that programming practitioners--people who are writing programs to solve problems--actually want "a language with a syntax and semantics tailored for a specific domain", or any really fancy language features at all. I say this from the perspective of someone who nearly became a PL researcher myself. I could easily have decided to stud…

> Will the promised productivity gains allow me to recoup the cost of the time spent learning. Some deep PL stuff I doubt there is productivity gain to begin with. But many ideas in the ML language family are simple and reduce debugging pain. Time lost from one encounter with muddy JS/Python semantics is more than the time learning about sum types.

Do you have any opinions on elixir? Have you played around with reasonML?

Re: LLMs pose an interesting problem for DSL designers

#79

Post by freedomben »

AnimalMuppet wrote: post #51
jp57 wrote: post #21

Programming languages researchers and designers labor under the mistaken assumption that programming practitioners--people who are writing programs to solve problems--actually want "a language with a syntax and semantics tailored for a specific domain", or any really fancy language features at all. I say this from the perspective of someone who nearly became a PL researcher myself. I could easily have decided to stud…

Hmm. Go was written to be easier for junior developers to use, and to be maintainable "in the large". I wonder if we need a language designed to be easier for an AI to reason about, or easier for a human to see the AI's mistakes.

I think in 5 years you are absolutely right, but for now we desperately need a language which is easy for ai and easy for humans. The stuff that AI has suggested to me is about 50% genius in about 50% idiocy. I don't know what that language is, if I could pick it personally, I would pick elixir, but I recognize that we're probably looking at typescript or something like that

Re: LLMs pose an interesting problem for DSL designers

#80

Post by demosthanos »

gopiandcode wrote: post #64
furyofantares wrote: post #60

I notice I am confused. > Suddenly the opportunity cost for a DSL has just doubled: in the land of LLMs, a DSL requires not only the investment of build and design the language and tooling itself, but the end users will have to sacrifice the use of LLMs to generate any code for your DSL. I don't think they will. Provide a concise description + examples for your DSL and the LLM will excel at writing within your DSL. A…

Arguably it really depends on your DSL right? If it has a semantics that already lies close to existing programming languages, then I'd agree that a few examples might be sufficient, but what if your particular domain doesn't match as closely? Examples of domains that might be more challenging to design DSLs for: languages for knitting, non-deterministic languages to represent streaming etc. (i.e https://pldi25.sigpl…

Remember that LLMs aren't trained on all existing programming languages, they're trained on all text on the internet. They encode information about knitting or streaming or whatever other topic you want a DSL for.

So it's not just a question of the semantics matching existing programming languages, the question is if your semantics are intelligible given the vast array of semantic constructs that are encoded in any part of the model's weights.

Post reply on HN