"DSL" is a stupid word. If it's a language, it's a language.
its a specific type of language, not all language are the same , the nature of that specificity is the main point of the the claim made in this article
DSLs Enable Reliable Use of LLMs
31–40 of 88 posts
Re: DSLs Enable Reliable Use of LLMs
#32Earlier quoted context omitted.
> DSL that is json/yaml helps a ton it definitely does, and i would say json/yaml is not a dsl. this example of json/yaml keeps coming in the form of "DSL". i would say your configuration is not a dsl, it a declaration. llms are better at declarative stuff ? maybe but there are hardly that many of complex declarative frameworks. PlantUML is a real dsl. not just declarative yaml.
> and i would say json/yaml is not a dsl But you can have DSLs that are json/yaml, is my point. > PlantUML is a real dsl. PlantUML is a DSL that isn't json/yaml. That doesn't make it better, and you can make the argument that it is worse because the tooling around it won't be as good.
> you can have DSLs that are json/yaml, is my point
i disagree on the semantics of "DSL" vs "config in json". these are different.
and anyways both of them just pretend to be "reliable" by throwing the responsibility to an upper layer of validator/compiler/interpreter
> it is worse because the tooling around it won't be as good
plantuml is good because of the the tooling around it. not sure if we are agreeing/disagreeing there, confused by the wording
Re: DSLs Enable Reliable Use of LLMs
#33[flagged]
Re: DSLs Enable Reliable Use of LLMs
#34"DSL" is a stupid word. If it's a language, it's a language.
its a specific type of language, not all language are the same , the nature of that specificity is the main point of the the claim made in this article
all languages are embedded in some domain, therefore having a domain does not make a language "a specific type of language"
Re: DSLs Enable Reliable Use of LLMs
#35> The advantage holds while the DSL stays small and constrained enough that a few in-context examples can convey its usage. There is also a real upfront cost in designing and maintaining the language and its semantic model. The payoff is therefore concentrated in well-factored, genuinely constrained DSLs backed by a validator. dsl stays small is doing all the heavy lifting here the premise is that because of these fe…
DSLs are a great middle ground for 'use LLM to turn ambiguous spec into something well defined', with the caveat that without discipline they'll inevitably expand until you should just have the agent write whatever the final language is.
There's a context tax up front (which will hopefully be less relevant over time) and then you really need a compiler/linter with helpful errors to keep it on the rails, because there is no corrective context in pretraining for something novel.
A purely descriptive DSL is just a convention, which is useful, but doesn't inject reliability the same way an enforced syntactic contract does.
Re: DSLs Enable Reliable Use of LLMs
#36> The advantage holds while the DSL stays small and constrained enough that a few in-context examples can convey its usage. There is also a real upfront cost in designing and maintaining the language and its semantic model. The payoff is therefore concentrated in well-factored, genuinely constrained DSLs backed by a validator. dsl stays small is doing all the heavy lifting here the premise is that because of these fe…
> the premise is that because of these few existing dsls (like PlantUML mentioned) my "new dsl" will be equally effective. PlantUML has millions of examples in the training data, my new dsls are not (*specially if its not json/yaml* or just function chain based) I can confirm that having a DSL that is json/yaml helps a ton . Kind of like static type checking, it eliminates entire swaths of syntactical errors, allowin…
> it eliminates entire swaths of syntactical errors, allowing the LLM to focus on the semantics
As I understood it, the idea the article presents is basically the opposite: The LLM is completely clueless about the semantics, but by constraining the language so that semantic errors are syntax errors, we can catch the LLM's bullshit with a simple validation step and make it try again until it produces valid syntax, which limits how wrong it can get the semantics.
It does apply to JSON in a way: An LLM doesn't always produce valid JSON, but it's easy to do a syntax check with a JSON parser and make the LLM try again until the JSON is valid. The JSON might still be semantically wrong on the level of the application domain (have the wrong values), but at least you can be sure that the generated data consists of lists, dictionaries, strings and numbers.
Re: DSLs Enable Reliable Use of LLMs
#37As much as I dislike them [1], DSLs are an easy way to provide domain specific backpressure to your llm which might not easily be available in your language.
1- https://mikehadlow.blogspot.com/2012/05/configuration-comple...
Re: DSLs Enable Reliable Use of LLMs
#38What I do for my one framework (in dotnet, strong typing & compiler) is to create claude skills (bash scripts and dotnet console apps) that claude can call to do various things within the framework. I have a template engine (for classes/db scripts/view engine templates), build tools, deployment tools, backup tools, infra docs - all of it works amazingly well. I built them in a standardized way so it's easy to chain them together and claude just 'gets it'. It feels like a DSL on steroids.
edit: this is all on linux + posgresql.
Re: DSLs Enable Reliable Use of LLMs
#39I frequently blur the line between ad-hoc DSL and pseudocode, and just hand it off to the LLM. I want to get the thoughts out of my head as fast as possible, using whatever structure makes sense to me. Even if you know all of the code to be written, I think this is a huge win with LLMs, where your intent is more important than syntax.
This is exactly it.
Typing was always just the interface. The keyboard was never the scarce resource. Judgment was.
Architecture scales. Implementation accumulates.
Re: DSLs Enable Reliable Use of LLMs
#40[flagged]