Live data from Hacker News

DSLs Enable Reliable Use of LLMs

martinfowler.com

71–80 of 88 posts

Re: DSLs Enable Reliable Use of LLMs

#71
post #68

Earlier quoted context omitted.

all languages are embedded in some domain, therefore having a domain does not make a language "a specific type of language"

People have settled on the term DSL to identify those languages whose domains are highly constrained as compared to more general purpose or broad domain languages. This usage exists whether or not you think the term itself is stupid.

I know what they meant to do, but they've drawn a line where no line exists and the result is confusion.

Re: DSLs Enable Reliable Use of LLMs

#72

I'm really starting to tire of people making broad, general claims about how LLMs work or how to use them with N = 1 or 2. An LLM is a statistics machine for goodness sake. Basically any general claim about them needs to exploit the law of large numbers to be even remotely sensible. You cannot extrapolate from one-off behavioral successes. LLMs are not understanding anything in the way humans do. If they did, yeah, m…

you have a premise at the heart of that:

> understanding anything in the way humans do

i'm not sure it's clearly established LLMs can't be a model of some part of “the way humans do”?

to be more specific, i'd argue LLMs “understand” awfully similarly to a brilliant (polymath) with early dementia or Alzheimer's

no executive function, no short term memory, and absent both of those, conversing with that person about the past or with an LLM about topics that had been "in their training sets before a cutoff date" is surprisingly similar, right down to: introing a topic precisely the same way, you'll experience the same conversation; convo loops if bits are too quantized (looping and lossiness / recall / context-length are correlated); and ofc opening a new session is like the first one never happened

Re: DSLs Enable Reliable Use of LLMs

#73
post #68

Earlier quoted context omitted.

People have settled on the term DSL to identify those languages whose domains are highly constrained as compared to more general purpose or broad domain languages. This usage exists whether or not you think the term itself is stupid.

I know what they meant to do, but they've drawn a line where no line exists and the result is confusion.

I don't see much confusion

Re: DSLs Enable Reliable Use of LLMs

#74
post #56

Earlier quoted context omitted.

I would say any JSON or YAML that carries semantics with the syntax is a language. It may be a configuration language rather than a programming language, but it’s a language rather than just generic data for some program to slurp.

> any JSON or YAML that carries semantics with the syntax is a language semantics are defined by the converter/compiler/interpreter, and that is the process which is going to consume the said json/yaml. if the json/yaml is going to be consumed by any process then the semantics are inherently defined, so by your definition all jsons/yamls are in their own a "language" (or they are not being consumed at all), which jus…

At that point you can say C has no semantics, because the compiler defines them.

Re: DSLs Enable Reliable Use of LLMs

#75
post #74

Earlier quoted context omitted.

> any JSON or YAML that carries semantics with the syntax is a language semantics are defined by the converter/compiler/interpreter, and that is the process which is going to consume the said json/yaml. if the json/yaml is going to be consumed by any process then the semantics are inherently defined, so by your definition all jsons/yamls are in their own a "language" (or they are not being consumed at all), which jus…

At that point you can say C has no semantics, because the compiler defines them.

yeah and so llm cannot write C reliably

Re: DSLs Enable Reliable Use of LLMs

#77
This article takes me back to the Martin Fowler book many years ago that for some reason was far less popular than most of his were at the time on DSLs. Reading the DDD book around the same time plus Novak’s Learning How to Learn was the conglomerate inspiration for creating the simplest possible DSL for capturing any domain’s ubiquitous language. it’s simply concepts connected by predicates, one sentence per line in a plain text file, and while it works well to keep LLMs conceptually aligned, they do get big quickly and need a tool to visualise and edit them so I built one: https://thinkingtools.software/concepticon/

Re: DSLs Enable Reliable Use of LLMs

#78
Anecdotal, but I have built from scratch a fairly large OpenAPI-compliant service using the Goa dsl (https://goa.design) and a coding agent and I've suspected for a while that the reason I've been as productive as I have with few blind alleys is that Goa kept the mistake surface pretty constrained. I also like not having the model generate boilerplate, as Goa's tooling does all that so much more cheaply. YMMV.

Re: DSLs Enable Reliable Use of LLMs

#79

This reminds me of this Bjarne Stroustrup's Rule (creator of C++): - For new features, people insist on loud, explicit syntax. - For established features, people want terse notation Hillel Wayne [1] argues that the same applies for the differences between what beginners and experts desire from a language: Beginners need explicit syntax, experts want terse syntax. In my mind, DSLs are related to that – a short notatio…

People mainly want loud, explicit syntax for new features that other people will start using, and that they don't like or want.

Re: DSLs Enable Reliable Use of LLMs

#80

> The advantage holds while the DSL stays small and constrained enough that a few in-context examples can convey its usage. There is also a real upfront cost in designing and maintaining the language and its semantic model. The payoff is therefore concentrated in well-factored, genuinely constrained DSLs backed by a validator. dsl stays small is doing all the heavy lifting here the premise is that because of these fe…

> the premise is that because of these few existing dsls (like PlantUML mentioned) my "new dsl" will be equally effective. PlantUML has millions of examples in the training data, my new dsls are not (*specially if its not json/yaml* or just function chain based) I can confirm that having a DSL that is json/yaml helps a ton . Kind of like static type checking, it eliminates entire swaths of syntactical errors, allowin…

the best language by a mile for dsl-as-data-structure is kdl2

it reads like code and you can easily define a schema too

well that's what i am using

https://kdl.dev/

and https://crates.io/crates/knus/ to define the schema

actually more than that. it would be awesome to have a programming language whose code is also valid as kdl 2. like lisp code being composed of s-expressions

Post reply on HN