Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs pose an interesting problem for DSL designers

kirancodes.me

Post reply on HN
Print view View original

141–150 of 157 posts

Re: LLMs pose an interesting problem for DSL designers

#141

Post by daxfohl »

daxfohl wrote: post #138
seanmcdirmid wrote: post #103

Earlier quoted context omitted.

If LLMs can ever produce implementation and tests opaquely, then perhaps we can instruct at the level of requirements. They will never be good enough at the beginning and we will just provide feedback over the working prototype the LLM generates and demonstrates? I haven’t seen technology move this fast before, so I wouldn’t make any hard predictions about how long actual code written by humans survives. We don’t rea…

I won't say "never", but I guess I left out the caveat that I tried this for a hack week demo once. Working in compliance and onboarding, there's a whole lot of logic around what you have to collect and check, what you have to do around user experience, what steps you can fall back to if something fails, etc. I tried to take all our code and translate it to English to feed to an LLM and see what it would do. But the…

Though now that I say that, I wonder if maybe the approach should be, instead of specifying the onboarding procedure in terms of steps, maybe it would be better to specify a goal, the constraints, and some optimization function, and allow the AI to figure out what the workflow steps should be. So it would act higher level, more as a constraint solver than a code generator. That could be easier to interact with, allow constraints to be read and updated more easily, and not require diving much into code at all.

Hmm, I don't think LLMs are quite there yet, but I could see this being a potential way to do things, that may fit better with their strengths, and allow more to be done without digging into actual code.

Now I just need to dust off my old Prolog books and figure out how to use coding assistants in a similar fashion.

Re: LLMs pose an interesting problem for DSL designers

#142

Post by TeMPOraL »

le_marton wrote: post #110
guelo wrote: post #61

Earlier quoted context omitted.

This sounds insightful but I can't make heads or tails of "you cannot optimize for every task and cross-cutting concern at the same time and expect improvement across the board". Can someone help me out?

I understand it that way: The general programming languages we have now are about as good as they are ever going to get. Balance-wise. One can solve problems across a broad range of topics equally well. Tailoring a language to make some tasks especially easy / straightforward will likely make some other problems a lot harder / more cumbersome to express. So further improvements in abstraction and expressiveness have…

> I understand it that way: (...)

You understood and explained what I meant well.

> So further improvements in abstraction and expressiveness have to come from elsewhere. Not better programming languages but partnering up with an LLM?

In practice, probably yes; but the point I've been making here and in bringing this up over the years is, further improvements can come if we stop insisting on directly editing "single source of truth" code representation. Then, you no longer need to make trade-offs up front - for task A, you may view code through lens (or "in language") that makes tasks like A especially easy, then for task B you switch to a view that makes tasks like B especially easy.

Examples:

- Switching between "point free style" and explicit temporary variables;

- For a given function, inlining its entire call subtree, turning a tiny function calling tiny functions into a single block you can read top to bottom; super useful for debugging;

- Switching between "exceptions" and "sum types" styles of error handling (they're pretty much equivalent), or more generally, switching between showing all vs. hiding error handling code vs. handling everything except error handling/propagating code;

- A view of code that shows only types, or one showing async, or one entirely hiding it;

- A database-style view for code, that lets you query it and edit in bulk;

- An editable state machine diagram that corresponds to code (maybe needs a little help by manually identifying which set of classes is conceptually a state machine, which methods are transitions, etc.);

- Hide/show logging, telemetry, or any other cross-cutting concern that you don't strictly care about at the moment (superset of earlier hiding of error handling code);

And so on, and so on.

Point being, all those views/perspectives operate on the same underlying artifact - the codebase. It can be plaintext, but because no one is actually reading it directly, it doesn't have to be optimized for anything in particular (or it could just be simple and straightforward, at a price of being verbose; say Python). Meanwhile, the views/perspectives could each use syntax or format best suited for the specific task it helps with. All the trade-offs would be made at the point of use, instead of baked in up front when the project starts.

Re: LLMs pose an interesting problem for DSL designers

#143

Post by gylterud »

cap11235 wrote: post #127
gylterud wrote: post #120

Earlier quoted context omitted.

I love writing shaders and manually shovelling arrays into the graphics card as much as anyone, and I know first hand how this will give the game very much your own style. But is that the direction LLM coding goes? My experience is that LLM produces code which is much more generic and boring than what skilled programmers make.

But do users care that your code is boring?

Since games are all about artistic expression and entertainment, I would say yes, it matters to the end users. The thing is, if you don't have your hand in the code details you might not know the directions it can be taken (and which it cannot). Seeing the possibility of the code is one way to get the creativity juices flowing for a game programmer. Just look at old games, which demonstrate extreme creativity even on the limits put on the software by the old hardware. But being stuck into the code, seeing the technical possibilities allowed this.

I think this is what the comment above was lamenting about abstractions. I am all for abstraction when it comes to being productive. And I think new abstractions open new possibilities some times! But some abstractions which makes standard stuff easy, makes non-standard stuff impossible.

Re: LLMs pose an interesting problem for DSL designers

#144

Post by doug_durham »

izabera wrote: post #50
jp57 wrote: post #21

Programming languages researchers and designers labor under the mistaken assumption that programming practitioners--people who are writing programs to solve problems--actually want "a language with a syntax and semantics tailored for a specific domain", or any really fancy language features at all. I say this from the perspective of someone who nearly became a PL researcher myself. I could easily have decided to stud…

30 lines are always going to be easier to read/write/debug than 3000 lines, so it'll probably remain easier (for both humans and machines) to write correct code in languages that make it possible to express ideas concisely and elegantly.

That is a false dichotomy. It is not 3000 versus 30. It's 3000 versus 2578 where the DSL is poorly documented by someone who long ago lost interest in maintaining it. I prefer the initial example in the article to the DSL equivalent. I can immediately read the non-DSL code and be productive. The DSL looks like a vanity project.

Re: LLMs pose an interesting problem for DSL designers

#145

Post by doug_durham »

noelwelsh wrote: post #108
jp57 wrote: post #21

Programming languages researchers and designers labor under the mistaken assumption that programming practitioners--people who are writing programs to solve problems--actually want "a language with a syntax and semantics tailored for a specific domain", or any really fancy language features at all. I say this from the perspective of someone who nearly became a PL researcher myself. I could easily have decided to stud…

Embedded DSLs (e.g. PyTorch) have been hugely successful in the field of machine learning, so I think there is a bit of nuance here that you're not considering. I also take issue with the idea that Python is simple. Python's semantics are anything but. The biggest issue the language has, performance, is a consequence of these poorly thought out semantics. If the language was actually simple it would be a lot easier t…

I teach beginner Python classes. If the semantics were so horribly convoluted we wouldn't be teaching it to beginners.

Re: LLMs pose an interesting problem for DSL designers

#146

Post by doug_durham »

AnimalMuppet wrote: post #51
jp57 wrote: post #21

Programming languages researchers and designers labor under the mistaken assumption that programming practitioners--people who are writing programs to solve problems--actually want "a language with a syntax and semantics tailored for a specific domain", or any really fancy language features at all. I say this from the perspective of someone who nearly became a PL researcher myself. I could easily have decided to stud…

Hmm. Go was written to be easier for junior developers to use, and to be maintainable "in the large". I wonder if we need a language designed to be easier for an AI to reason about, or easier for a human to see the AI's mistakes.

I totally see "read-only" language emerging in the future. A language that is never written by humans yet has unambiguous syntax and semantics.

Re: LLMs pose an interesting problem for DSL designers

#148

Post by guelo »

TeMPOraL wrote: post #142
le_marton wrote: post #110

Earlier quoted context omitted.

I understand it that way: The general programming languages we have now are about as good as they are ever going to get. Balance-wise. One can solve problems across a broad range of topics equally well. Tailoring a language to make some tasks especially easy / straightforward will likely make some other problems a lot harder / more cumbersome to express. So further improvements in abstraction and expressiveness have…

> I understand it that way: (...) You understood and explained what I meant well. > So further improvements in abstraction and expressiveness have to come from elsewhere. Not better programming languages but partnering up with an LLM? In practice, probably yes; but the point I've been making here and in bringing this up over the years is, further improvements can come if we stop insisting on directly editing "single…

That does sound useful but it seems like it should be fancy features in the IDE, not the language.

Re: LLMs pose an interesting problem for DSL designers

#149

Post by lamuswawir »

jrmg wrote: post #123
NathanKP wrote: post #12

Good to see more people talking about this. I wrote about this about 6 months ago, when I first noticed how LLM usage is pushing a lot of people back towards older programming languages, older frameworks, and more basic designs: https://nathanpeck.com/how-llms-of-today-are-secretly-shapin... To be honest I don't think this is necessarily a bad thing, but it does mean that there is a stifling effect on fresh new DSL's…

It’s not just new frameworks, it’s new features. Good luck getting a LLM to write code that uses iOS 26 features, for example. I’m not convinced simply getting the LLM to inject documentation about the features will work well (perhaps someone has studied this?) because the reason they’re good at doing ‘well known’ things is the plethora of actual examples they’re trained on.

I now put documentation of my DSLs and macros in a folder in the repo. I give the agent an instruction about consulting the documentation and it is working so far, but not that well. I am considering switching to all generics.

Re: LLMs pose an interesting problem for DSL designers

#150

Post by seanmcdirmid »

daxfohl wrote: post #141
daxfohl wrote: post #138

Earlier quoted context omitted.

I won't say "never", but I guess I left out the caveat that I tried this for a hack week demo once. Working in compliance and onboarding, there's a whole lot of logic around what you have to collect and check, what you have to do around user experience, what steps you can fall back to if something fails, etc. I tried to take all our code and translate it to English to feed to an LLM and see what it would do. But the…

Though now that I say that, I wonder if maybe the approach should be, instead of specifying the onboarding procedure in terms of steps, maybe it would be better to specify a goal, the constraints, and some optimization function, and allow the AI to figure out what the workflow steps should be. So it would act higher level, more as a constraint solver than a code generator. That could be easier to interact with, allow…

I think the magic key might be LLMs that can build and test code, and then revise it and test it again, revise tests, etc...all while accepting feedback from human users about what the requirements really are (because they aren't going to be complete at the beginning). We might already be there with LLMs and we just haven't figured out the workflow yet (it isn't just one LLM activation, for sure, it has to be a series, with feedback from test execution and reviews...like real SWEs).
Post reply on HN