Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Malleable Software in the Age of LLMs (2023)

geoffreylitt.com

Post reply on HN
Print view View original

11–20 of 39 posts

Re: Malleable Software in the Age of LLMs (2023)

#11

Post by dspillett »

greenavocado wrote: post #3
chiasson wrote: post #2

> Until now, that vision has been bottlenecked on turning fuzzy informal intent into formal, executable code Software has been and always will be fundamentally a problem of communication. What many don't realize is that the true challenge of communication is not one of transmission, but clarity of thought and understanding. Any tool, software language, or AI coder will still be limited by the clarity and completeness…

Agree. Even though I only have twelve years of experience writing software professionally I started to use LLMs more and more in the past few months to write my code for me given a specification of the task. The top commercial LLMS usually do a good job but often fail catastrophically, especially on esoteric projects. I feel like I am slowly developing into a lawyer because of how thoroughly I have to spec out the pr…

> because of how thoroughly I have to spec out the problem at hand to give enough context to get exactly what I want

This is why LLMs in their current iterations are not the danger to our jobs that some fear them to be: end users and other sake holders lack the precision to properly spec what they want (and often don't even accurately know what they want) a tool/service to do, and they don't have the time (or if they do, don't have the patience) to go back and forth iterating over the wording of the spec to get things adequately defined.

I'm currently an LLM refusenik⁰ so might be missing some context, but from my outside view I get the impression that they do a good job a simple boiler-plate-y stuff but don't save time/effort/thinking on anything much more complex. I'm sure most devs beyond the beginner are happy to have those boilerplate tasks taken off them so they can do the fun stuff, but equally end users aren't going to spend the time working with the tools to get anything more interesting than very simple programming/automation tasks done.

--

[0] I'm on the “is that really morally right?” side of the fence on how the training materials are sourced, particularly with regard to code covered by licences like the GPL family¹, and I'm anal enough to not use something I have that sort of question about even if it makes my life a little harder.

[1] If the assurances that chunks of code can't be regurgitated and that makes it all fine both morally and legally, why are none of the publicly useable LLMs such as MS's copilot trained using, say, Microsoft's Office/Windows/other code as well as public repositories? Surely they should be assured that isn't a problem as much as they want everyone else to be assured it isn't a problem?

Re: Malleable Software in the Age of LLMs (2023)

#13

Post by layer8 »

surfingdino wrote: post #9
sublinear wrote: post #5

Earlier quoted context omitted.

The same points you made make me wonder if end users even want to modify or write a program. Even if it's possible, it's still a lot of work. I just don't see that as the typical consumer behavior.

I have yet to meet a non-dev who wants to write or modify software themselves. Quite a few people still don't know that it's the software that makes appliances and devices do things and that it is written. They just don't know what software is so how would they know where to begin writing or modifying code?

Start with macro recorders [0] that let users record and replay sequences of operations. This already exists in certain household appliances, or when making a music playlist. Give them the ability to view and modify the macro code. Add loops and variables etc., and some portion of people will create more complex macros, even if they are no software developers and have no interest in being one. It’s like writing a cooking recipe, people understand that. Apple iOS Shortcuts are also a form of that.

[0] https://en.wikipedia.org/wiki/Macro_recorder

Re: Malleable Software in the Age of LLMs (2023)

#14

Post by michaelbrave »

philipswood wrote: post #8
sublinear wrote: post #5

Earlier quoted context omitted.

The same points you made make me wonder if end users even want to modify or write a program. Even if it's possible, it's still a lot of work. I just don't see that as the typical consumer behavior.

Software often doesn't do the thing you want to do. It almost does it. Or it can just-just do it with contortions and a lot of repetitive toil on your side. I think writing little plugins and drivers to do the thing you want with an LLM is something that could be built into a lot of software. I don't think LLM can architect and build whole systems yet, but this niche is something that can be done.

I've heard rumors of people that don't know how to code using LLMs to make blender addons in python, so this has already started I think.

Re: Malleable Software in the Age of LLMs (2023)

#15

Post by karma_pharmer »

chiasson wrote: post #2

> Until now, that vision has been bottlenecked on turning fuzzy informal intent into formal, executable code Software has been and always will be fundamentally a problem of communication. What many don't realize is that the true challenge of communication is not one of transmission, but clarity of thought and understanding. Any tool, software language, or AI coder will still be limited by the clarity and completeness…

If you don't have clarity of thought, no intelligence in the world can help you.

I disagree.

Moronic people in positions of power are often in this situation. So far this has worked out well for the moronic powerful people: as long as they can convince intelligent people to work for them, they get to be moronic, powerful, and successful. This strategy has been working for them (as a group) for several millenia.

I think this is why the "LLMs will evolve into God In My Pocket" story gets so much traction. To a moronic person in power this sounds a lot like everyday life, but with computers cleaning up their messes instead of humans. It is totally plausible to them.

Re: Malleable Software in the Age of LLMs (2023)

#16

Post by svieira »

surfingdino wrote: post #9
sublinear wrote: post #5

Earlier quoted context omitted.

The same points you made make me wonder if end users even want to modify or write a program. Even if it's possible, it's still a lot of work. I just don't see that as the typical consumer behavior.

I have yet to meet a non-dev who wants to write or modify software themselves. Quite a few people still don't know that it's the software that makes appliances and devices do things and that it is written. They just don't know what software is so how would they know where to begin writing or modifying code?

Yet I'm sure many of those non-devs have used Excel / LibreOffice / Lotus 1-2-3 to build something that is more-than-a-list-or-grid (anytime you put `=` into a cell, you are programming).

Re: Malleable Software in the Age of LLMs (2023)

#17

Post by alphazard »

chiasson wrote: post #2

> Until now, that vision has been bottlenecked on turning fuzzy informal intent into formal, executable code Software has been and always will be fundamentally a problem of communication. What many don't realize is that the true challenge of communication is not one of transmission, but clarity of thought and understanding. Any tool, software language, or AI coder will still be limited by the clarity and completeness…

> If you don't have clarity of thought, no intelligence in the world can help you.

This is clearly not true, just look at the tech industry. We have product managers, who in the most literal sense, do not know what they mean when they ask for something. They are paired up with people much smarter then them, who try to figure out what they mean, and then deliver that.

While this is far from ideal, it's proof that if you don't have clarity of thought, external (human level) intelligence can help you.

Re: Malleable Software in the Age of LLMs (2023)

#18

Post by drdrek »

chiasson wrote: post #2

> Until now, that vision has been bottlenecked on turning fuzzy informal intent into formal, executable code Software has been and always will be fundamentally a problem of communication. What many don't realize is that the true challenge of communication is not one of transmission, but clarity of thought and understanding. Any tool, software language, or AI coder will still be limited by the clarity and completeness…

Not only this but the author lacks enough empathy and imagination to understand how most people are wildly different than him and would not want any of this complexity in the slightest.

Re: Malleable Software in the Age of LLMs (2023)

#19

Post by williamcotton »

dspillett wrote: post #11
greenavocado wrote: post #3

Earlier quoted context omitted.

Agree. Even though I only have twelve years of experience writing software professionally I started to use LLMs more and more in the past few months to write my code for me given a specification of the task. The top commercial LLMS usually do a good job but often fail catastrophically, especially on esoteric projects. I feel like I am slowly developing into a lawyer because of how thoroughly I have to spec out the pr…

> because of how thoroughly I have to spec out the problem at hand to give enough context to get exactly what I want This is why LLMs in their current iterations are not the danger to our jobs that some fear them to be: end users and other sake holders lack the precision to properly spec what they want (and often don't even accurately know what they want) a tool/service to do, and they don't have the time (or if they…

Projects of mine that LLMs assisted on:

A web framework for C using clang blocks with request-bases memory arenas for easy memory management.

A CNN trained on video stills of playing various chords on guitar hosted using PyTorch in a SwiftUI iOS app

A DSL using Python Lex Yacc for piping CSV data into a runtime that spits out matplotlib graphs in PNG bytes ready for imgcat.

Right now I’m working on language server in written F# using Fparsec with an intermediary AST for capturing multiple parser errors.

I’d say these are relatively advanced endeavors and that there’s a lot more than boilerplate going on.

And yes, I can explain how these things work as LLMs require quite a bit of guidance.

Re: Malleable Software in the Age of LLMs (2023)

#20

Post by ickelbawd »

I’m quite sure I don’t want a different bespoke UI for everything. And given the non-deterministic nature of these models it’s going to be different even for the same task. Now instead of hunting for the right buttons after a software “update” with some genius redesign: I’ll be hunting every time I use it? Sounds like a nightmare.

I do agree with the author though as to the deficiencies with a purely chat based interface especially for power users or technical interfaces. It’s simply a non-starter for me. That said LLMs will complement power user interfaces nicely by helping the new user learn them.

Post reply on HN