Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs pose an interesting problem for DSL designers

kirancodes.me

Post reply on HN
Print view View original

101–110 of 157 posts

Re: LLMs pose an interesting problem for DSL designers

#101

Post by seanmcdirmid »

TimTheTinker wrote: post #69
NathanKP wrote: post #32

Earlier quoted context omitted.

I think there is a lot of overlap between DSL's and frameworks, and most frameworks contain some form of DSL in them. What matters most of all is whether the DSL is written in semantically meaningful tokens. Two extremes as examples: Regex is a DSL that is not written in tokens that have inherent semantic meaning. LLM's can only understand Regex by virtue of the fact that it has been around for a long time and there…

> there is a lot of overlap between DSL's and frameworks Not just frameworks, but libraries also. Interacting with some of the most expressive libraries is often akin to working with a DSL. In fact, the paradigms of some libraries required such expressiveness that they spawned their own in-language DSLs, like JSX for React, or LINQ expressions in C#. These are arguably the most successful DSLs out there.

Embedded DSLs have their own challenge, since the LLM can easily move out of the DSL into the host language in ways that aren’t valid for the eDSL. You really need to narrow the focus with more context to get anything useful out of it in my experience.

Re: LLMs pose an interesting problem for DSL designers

#102

Post by [deleted] »

Izkata wrote: post #100
NathanKP wrote: post #58

Earlier quoted context omitted.

This is an interesting idea actually, if we assume a couple things: - There likely won't be one Skynet, but rather multiple AI's, produced by various sponsors, starting out as relatively harmless autonomous agents in corporate competition with each other - AI agents can only inference, then read and write output tokens at a limited rate based on how fast the infrastructure that powers the agent can run In this scenar…

> There likely won't be one Skynet, but rather multiple AI's, produced by various sponsors, starting out as relatively harmless autonomous agents in corporate competition with each other This is kind of how Skynet begins in the TV series Terminator: The Sarah Connor Chronicles. It takes place after the 2nd movie in an alternate timeline from the 3rd movie (and establishes itself how that's possible without contradict…

[deleted]

Re: LLMs pose an interesting problem for DSL designers

#103

Post by seanmcdirmid »

daxfohl wrote: post #62
kibwen wrote: post #46

People often use the analogy of LLMs being to high-level languages what compilers were for assembly languages, and despite being a terrible analogy there's no guarantee it won't eventually be largely true in practice. And if it does come true, consider how the advent of the compiler completely eliminated any incentive to improve the ergonomics or usability of assembly code, which has been and continues to be absolute…

A big difference is that compilers are deterministic, and coders generally don't review and patch the generated assembly. There's little reason to expect that LLMs will ever function like that. It's always going to be a back-and-forth of, "hey LLM code this up", "no, function f isn't quite right; do this instead", etc. This mimics what you see in, say, Photoshop. You can edit pixels manually, you can use deterministi…

If LLMs can ever produce implementation and tests opaquely, then perhaps we can instruct at the level of requirements. They will never be good enough at the beginning and we will just provide feedback over the working prototype the LLM generates and demonstrates?

I haven’t seen technology move this fast before, so I wouldn’t make any hard predictions about how long actual code written by humans survives. We don’t really need AGI at this point to have opaque coding solutions, even if the LLMs should still be better.

Re: LLMs pose an interesting problem for DSL designers

#104

Post by noobermin »

kibwen wrote: post #96
noobermin wrote: post #84

Earlier quoted context omitted.

An ignorant perspective, from someone likely hasn't coded assembly ever. Assembly is tied to the system you target and it can't really be "improved". You can however improve ergonomics greatly via macros and everyone does this.

> Assembly is tied to the system you target and it can't really be "improved". Of course it can. There's no reason for modern extensions to keep pumping out instructions named things like "VCTTPS2DQ" other than an adherence to cryptic tradition and a confidence that the people who read assembly code are poor saps who don't matter in the grand scheme of the industry, which is precisely my point. And even if x86 was se…

>poor saps who don't matter in the grand scheme of the industry

Someone is mad. Just because you don't have the patience for it doesn't mean everyone has the same preferences as you do.

Anyway, one could argue macros in assembly are part and parcel of the process if you develop things in assembly of significant complexity. If you don't like a particular instruction name, you could always relabel it in a macro to something you like. "Redesigning assembly" of an existing target otherwise makes no sense as a concept as assembly languages are usually specified by ISA designers as a target to meet the developers of compilers.

You can of course write your own assembler for yourself if you really want. That's the beauty of asm, you don't have to meet ideological targets for this or that PL school. You just need to admit the right byte codes, and that's sufficient. It doesn't guarantee things will work how you want though easily in a higher level sense, which is why most people use programming languages.

Re: LLMs pose an interesting problem for DSL designers

#105

Post by jeroenhd »

jo32 wrote: post #98

In the LLM era, building a brand-new DSL feels unnecessary. DSLs used to make sense because they gave you a compact, domain-specific syntax that simple parsers could handle. But modern language models can already read, write, and explain mainstream languages effortlessly, and the tooling around those languages—REPLs, compilers, debuggers, libraries—is miles ahead of anything you’d roll on your own. So rather than inv…

I can't even trust an LLM to write working Java code, let alone trust it to convert whatever a DSL is supposed to express into another form. Sure, maybe there's not enough Java 23 in its training set to effectively copy into my application, but Java 11 combined with 10 year old libraries shouldn't be a problem if these coding LLMs are worth their salt.

Until LLMs stop making up language features, methods, and operators out of convenience, DSLs are here to stay.

Re: LLMs pose an interesting problem for DSL designers

#106

Post by alganet »

> a DSL requires not only the investment of build and design the language and tooling itself

Not necessarily true. There are two kinds of DSLs: external and internal.

An external DSL has its own tooling, parser, etc. The nix language, for example.

An internal DSL is like a small parasite that lives inside an existing language, reusing some of its syntax and tools. It's almost like intentional pareidolia. Like jQuery, for example.

Internal DSLs reduce the cognitive load, and in my opinion, they're the best kind of DSL.

Re: LLMs pose an interesting problem for DSL designers

#107

Post by KaiserPro »

jeroenhd wrote: post #105
jo32 wrote: post #98

In the LLM era, building a brand-new DSL feels unnecessary. DSLs used to make sense because they gave you a compact, domain-specific syntax that simple parsers could handle. But modern language models can already read, write, and explain mainstream languages effortlessly, and the tooling around those languages—REPLs, compilers, debuggers, libraries—is miles ahead of anything you’d roll on your own. So rather than inv…

I can't even trust an LLM to write working Java code, let alone trust it to convert whatever a DSL is supposed to express into another form. Sure, maybe there's not enough Java 23 in its training set to effectively copy into my application, but Java 11 combined with 10 year old libraries shouldn't be a problem if these coding LLMs are worth their salt. Until LLMs stop making up language features, methods, and operato…

I fucking hate DSLs however I know they need to exist, but nowhere near as many should exist as they do now.

However LLMs are actually quite useful for translating concepts in DSLs that you don't understand. They don't so it error free, of course, but allows one to ask enough questions to work out why your attempt to translate concepts into this new fucking stupid ontological pile of wank isn't working

Re: LLMs pose an interesting problem for DSL designers

#108

Post by noelwelsh »

jp57 wrote: post #21

Programming languages researchers and designers labor under the mistaken assumption that programming practitioners--people who are writing programs to solve problems--actually want "a language with a syntax and semantics tailored for a specific domain", or any really fancy language features at all. I say this from the perspective of someone who nearly became a PL researcher myself. I could easily have decided to stud…

Embedded DSLs (e.g. PyTorch) have been hugely successful in the field of machine learning, so I think there is a bit of nuance here that you're not considering.

I also take issue with the idea that Python is simple. Python's semantics are anything but. The biggest issue the language has, performance, is a consequence of these poorly thought out semantics. If the language was actually simple it would be a lot easier to build a faster implementation.

Re: LLMs pose an interesting problem for DSL designers

#109

Post by noelwelsh »

tptacek wrote: post #52
izabera wrote: post #50

Earlier quoted context omitted.

30 lines are always going to be easier to read/write/debug than 3000 lines, so it'll probably remain easier (for both humans and machines) to write correct code in languages that make it possible to express ideas concisely and elegantly.

From elsewhere on the front page: "given choice between complexity or one on one against t-rex, grug take t-rex: at least grug see t-rex".

So you write everything in assembly then?

Re: LLMs pose an interesting problem for DSL designers

#110

Post by le_marton »

guelo wrote: post #61
TeMPOraL wrote: post #22

On the one hand, this sucks. On the other hand, we're already vacillating along the Pareto frontier of how much we can stuff into code; in fact, most of the criticisms of DSLs are indirectly stating just that. So with LLMs making it easier to project back and forth between how programmer sees the task at hand, and the underlying dumb/straightforward code they ain't gonna read anyway, maybe we'll finally get to the po…

This sounds insightful but I can't make heads or tails of "you cannot optimize for every task and cross-cutting concern at the same time and expect improvement across the board". Can someone help me out?

I understand it that way:

The general programming languages we have now are about as good as they are ever going to get. Balance-wise. One can solve problems across a broad range of topics equally well.

Tailoring a language to make some tasks especially easy / straightforward will likely make some other problems a lot harder / more cumbersome to express.

So further improvements in abstraction and expressiveness have to come from elsewhere. Not better programming languages but partnering up with an LLM?

Post reply on HN