Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DSLs Enable Reliable Use of LLMs

martinfowler.com

Post reply on HN
Print view View original

61–70 of 88 posts

Re: DSLs Enable Reliable Use of LLMs

#61

Post by codegladiator »

cestith wrote: post #56
codegladiator wrote: post #16

Earlier quoted context omitted.

> DSL that is json/yaml helps a ton it definitely does, and i would say json/yaml is not a dsl. this example of json/yaml keeps coming in the form of "DSL". i would say your configuration is not a dsl, it a declaration. llms are better at declarative stuff ? maybe but there are hardly that many of complex declarative frameworks. PlantUML is a real dsl. not just declarative yaml.

I would say any JSON or YAML that carries semantics with the syntax is a language. It may be a configuration language rather than a programming language, but it’s a language rather than just generic data for some program to slurp.

> any JSON or YAML that carries semantics with the syntax is a language

semantics are defined by the converter/compiler/interpreter, and that is the process which is going to consume the said json/yaml. if the json/yaml is going to be consumed by any process then the semantics are inherently defined, so by your definition all jsons/yamls are in their own a "language" (or they are not being consumed at all), which just defeats the purpose of calling it a language at that point.

Re: DSLs Enable Reliable Use of LLMs

#62

Post by dominotw »

> PlantUML, Mermaid, and Graphviz are domain specific languages for visual modeling; SQL is a DSL for querying databases; Kubernetes YAML is a DSL for describing cloud infrastructure.

Arent these all in training data too?

Re: DSLs Enable Reliable Use of LLMs

#63

Post by dominotw »

andy_ppp wrote: post #13

What is the general consensus on Martin Fowler - I worked with Thought Works and they were obsessed with overcomplicating everything, but maybe that is just agency in general? I think it goes without saying that the biggest fight we have as developers is keeping things as simple as possible when most external factors encourage complexity, especially LLMs.

Ironically Martin fowler was among the wave of influencers that were trying to get away from complexity of enterprise software. remember EJBs lol.

That wave included

1. TDD by beck 2. spring framework 3. rails ( later ) 4. Agile manifesto 5. refactoring by fowler 6. gang of four design patterns

That said. Thoughtworks is a moneygrab that tried to cash in on fowler brand. I worked with them at sears ( worked closely with author of article) and siemens. they were no different from any other consulting firms that try to overcomplicate things so they can deploy more warm bodies to the project.

Re: DSLs Enable Reliable Use of LLMs

#64

Post by brrrrrm »

codegladiator wrote: post #6

> The advantage holds while the DSL stays small and constrained enough that a few in-context examples can convey its usage. There is also a real upfront cost in designing and maintaining the language and its semantic model. The payoff is therefore concentrated in well-factored, genuinely constrained DSLs backed by a validator. dsl stays small is doing all the heavy lifting here the premise is that because of these fe…

> you are basically looking at a whole system prompt just describing the new language

whats wrong with this? You may be over-indexing on the need for large quantities of examples. These days self-play through RL is far more effective and data (not compute) efficient.

Re: DSLs Enable Reliable Use of LLMs

#65

Post by ako »

tclancy wrote: post #25
bob1029 wrote: post #22

The only thing which enables reliable use of LLMs are statistical techniques. Even the most constrained and well-designed Disney world ride will break down in some embarrassing way every now and again. As you increase the # of parallel rides, the chances that at least one of them will touch the desired parts of the search space go up dramatically. The fact that the major model providers keep publishing nano/mini/luna…

This is a statistical technique, effectively: it shrinks the problem space the LLM faces at each decision point. Ideally, it would be a way to turn a request from an open-world game into a game on rails. Given the DSL, my options are only X or Y at this point in the solution. Admittedly, that’s just improving the likelihood of getting a successful result, but if you mean 100% when you say “reliable”, that’s a false e…

Exactly, and a dsl gives you many options and tools to make the llm more reliable: a grammar enables deterministic syntax checker, error messages injecting correct syntax into the context, linting, terse examples in the skills for token optimization, and even the option to add the grammar to the llm to help it decide what next tokens are acceptable.

Re: DSLs Enable Reliable Use of LLMs

#66

Post by juancn »

I already had a DSL and with a proper prompt and a checker tool, even tiny LLMs could build really good scripts in that language. Gemma 12B QAT was excellent.

Actually the tool-calling convention is a small DSL too.

I suspect the DSL ideally has to be similar to something in the LLM's training set though.

It doesn't have to feel too alien, otherwise the description of it has to be thorough and will eat up a lot of context.

Frontier models don't suffer as much of this limitation since they can grab onto a larger corpus of knowledge, but they're expensive.

Re: DSLs Enable Reliable Use of LLMs

#67

Post by codegladiator »

brrrrrm wrote: post #64
codegladiator wrote: post #6

> The advantage holds while the DSL stays small and constrained enough that a few in-context examples can convey its usage. There is also a real upfront cost in designing and maintaining the language and its semantic model. The payoff is therefore concentrated in well-factored, genuinely constrained DSLs backed by a validator. dsl stays small is doing all the heavy lifting here the premise is that because of these fe…

> you are basically looking at a whole system prompt just describing the new language whats wrong with this? You may be over-indexing on the need for large quantities of examples. These days self-play through RL is far more effective and data (not compute) efficient.

nothing wrong if your dsl stays small and you have deterministic validators/compilers to your actual target.

if dsl gets large, the number of potential interactions your dsl allow will grow exponentially (unless you are building an one dimensional action layer). and there will be semantic issues unless your dsl is "clear and intuitive", also comprehensive enough to accomodate your ongoing changes, else every change is now 2 changes.

so you now you need a comprehensive manual for your agent which needs to be sent in every /completion request.

Re: DSLs Enable Reliable Use of LLMs

#68

Post by dare944 »

conartist6 wrote: post #34
systems wrote: post #31

Earlier quoted context omitted.

its a specific type of language, not all language are the same , the nature of that specificity is the main point of the the claim made in this article

all languages are embedded in some domain, therefore having a domain does not make a language "a specific type of language"

People have settled on the term DSL to identify those languages whose domains are highly constrained as compared to more general purpose or broad domain languages. This usage exists whether or not you think the term itself is stupid.

Re: DSLs Enable Reliable Use of LLMs

#69

Post by marssaxman »

codegladiator wrote: post #6

> The advantage holds while the DSL stays small and constrained enough that a few in-context examples can convey its usage. There is also a real upfront cost in designing and maintaining the language and its semantic model. The payoff is therefore concentrated in well-factored, genuinely constrained DSLs backed by a validator. dsl stays small is doing all the heavy lifting here the premise is that because of these fe…

The situation is much better than you think. I actually have a little DSL project along these lines: it's meant to be a declarative component in a larger application which enforces certain data guarantees.

It is not based on JSON or any other out-of-the-box format. The only source material for this DSL anywhere in the world is on my laptop: a little documentation, a few examples, a partial implementation.

This is enough. Codex can not only give me arbitrary examples in this novel DSL on demand, it can see what I'm trying to do with the compiler and extend it for me.

Re: DSLs Enable Reliable Use of LLMs

#70

Post by Terretta »

rudedogg wrote: post #53
sheepscreek wrote: post #50

Earlier quoted context omitted.

I’ve had great luck building SwiftUI apps with GPT-5.5 (now GPT-5.6 Sol), Opus 4.8 and Fable 5. I’m just offering another data point, not suggesting on the effectiveness of DSLs for this. One line of thinking can be that frontier models are already powerful enough to brute force through it, but this might be a stronger indicator for success for smaller models.

No argument that it works, it definitely does. I just think the LLMs have a hard time with tweaks/modifications of any kind, more so than what I see even in a custom GUI toolkit I’ve built that it has zero training data on. Asking for simple changes to SwiftUI interfaces I’ve one-shot is always a frustrating experience for me

> simple changes to SwiftUI interfaces I’ve one-shot is always a frustrating experience for me

1) Have you exported Apple docs from Xcode and provided those to your LLM and told your CLAUDE.md to read them as applies?

2) Have you added https://github.com/dpearson2699/swift-ios-skills to your agent?

I'd also recommend Xcode 27 beta unless you teach your agent the Xcode MCP. Xcode 27 beta works on MacOS 26.x.

Post reply on HN