Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Some thoughts on LLMs and software development

martinfowler.com

Post reply on HN
Print view View original

281–290 of 422 posts

Re: Some thoughts on LLMs and software development

#281

Post by dapperdrake »

jeppester wrote: post #107

In my company I feel that we getting totally overrun with code that's 90% good, 10% broken and almost exactly what was needed. We are producing more code, but quality is definitely taking a hit now that no-one is able to keep up. So instead of slowly inching towards the result we are getting 90% there in no time, and then spending lots and lots of time on getting to know the code and fixing and fine-tuning everything…

Perlis, epigram 7:

7. It is easier to write an incorrect program than understand a correct one.

Link: http://cs.yale.edu/homes/perlis-alan/quotes.html

Re: Some thoughts on LLMs and software development

#282

Post by ljm »

dragonwriter wrote: post #265

> My former colleague Rebecca Parsons, has been saying for a long time that hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful. This is an example of my least favorite style of feigned insight: redefining a term into meaninglessness just so you can say something that sounds different while not actu…

If everything is a hallucination then nothing is a hallucination.

Thanks for listening to my Ted Talk.

Re: Some thoughts on LLMs and software development

#283

Post by discreteevent »

bluefirebrand wrote: post #213
x0x0 wrote: post #176

Earlier quoted context omitted.

ime this is the problem. When I have to deeply understand what an llm created, I don't see much of a speed improvement vs writing it myself. With an engineer you can hand off work and trust that it works, whereas I find code reviewing llm output something that I have to treat as hostile. It will comment out auth or delete failing tests.

> When I have to deeply understand what an llm created Which should be always in my opinion Are people really pushing code to production that they don't understand?

They are, because in fairness in a lot of cases it just doesn't matter. It's some website to get clicks for ads and as long as you can vibe use it it's good enough to vibe code it.

Re: Some thoughts on LLMs and software development

#284

Post by ljm »

tugberkk wrote: post #254
oo0shiny wrote: post #122

> My former colleague Rebecca Parsons, has been saying for a long time that hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful. What a great way of framing it. I've been trying to explain this to people, but this is a succinct version of what I was stumbling to convey.

Yes, can't remember who said it but LLM's always hallucinate, it is just that they are 90 something percent right.

If I was to drop acid and hallucinate an alien invasion, and then suddenly a xenomorph runs loose around the city while I’m tripping balls, does being right in that one instance mean the rest of my reality is also a hallucination?

Because it seems the point being made multiple times that a perceptual error isn’t a key component of hallucinating, the whole thing is instead just a convincing illusion that could theoretically apply to all perception, not just the psychoactively augmented kind.

Re: Some thoughts on LLMs and software development

#285

Post by belter »

dragonwriter wrote: post #265

> My former colleague Rebecca Parsons, has been saying for a long time that hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful. This is an example of my least favorite style of feigned insight: redefining a term into meaninglessness just so you can say something that sounds different while not actu…

> You've just taken the label conventionally attached to a bad behavior, attached it to a broader category that includes all behavior, and used the power of equivocation to make something that sounds novel without saying anything new.

You mean like every evangelist says AI changes everything every 5 min...When in reality what they mean is neural nets statistical code generators are getting pretty good? because that is almost all AI there is at the moment?

Just to make the current AI sound bigger than it is?

Re: Some thoughts on LLMs and software development

#286

Post by dragonwriter »

bartread wrote: post #278
dragonwriter wrote: post #265

> My former colleague Rebecca Parsons, has been saying for a long time that hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful. This is an example of my least favorite style of feigned insight: redefining a term into meaninglessness just so you can say something that sounds different while not actu…

I’ve never liked that this behaviour is described using the term “hallucination”. If a human being talked confidently about something that they were just making up out of thin air by synthesizing based (consciously or unconsciously) on other information they know you wouldn’t call it “hallucination”: you’d call it “bullshit”. And, honestly, “bullshit” is a much more helpful way of thinking about this behaviour becaus…

> I’ve never liked that this behaviour is described using the term “hallucination”.

I have a standard canned rant about "confabulation" is a much better metaphor, but it wasn't the point I was focussed on here.

> Fundamentally, if you don’t want to work with LLMs because they sometimes “bullshit”, are you planning on no longer working with human beings as well?

I will very much not voluntarily rely on a human for particular tasks if that human has demonstrated a pattern of bullshitting me when given that kind of task, yes, especially if, on top of the opportunity cost inherent in relying on a person for a particular task, I am also required to compensate them—e.g., financially—for their notional attention to the task.

Re: Some thoughts on LLMs and software development

#287

Post by red75prime »

jstrieb wrote: post #214
oo0shiny wrote: post #122

> My former colleague Rebecca Parsons, has been saying for a long time that hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful. What a great way of framing it. I've been trying to explain this to people, but this is a succinct version of what I was stumbling to convey.

I have been explaining this to friends and family by comparing LLMs to actors. They deliver a performance in-character, and are only factual if it happens to make the performance better. https://jstrieb.github.io/posts/llm-thespians/

The analogy goes down the drain when a criterion for good performance is being objectively right. Like with Reinforcement Learning from Verifiable Rewards.

Re: Some thoughts on LLMs and software development

#288

Post by jstummbillig »

whstl wrote: post #253
jeppester wrote: post #107

In my company I feel that we getting totally overrun with code that's 90% good, 10% broken and almost exactly what was needed. We are producing more code, but quality is definitely taking a hit now that no-one is able to keep up. So instead of slowly inching towards the result we are getting 90% there in no time, and then spending lots and lots of time on getting to know the code and fixing and fine-tuning everything…

LLMs are amazing at producing boilerplate, which removes the incentive to get rid of it. Boilerplate sucks to review. You just see a big mass of code and can't fully make sense of it when reviewing. Also, Github sucks for reviewing PRs with too many lines. So junior/mid devs are just churning boilerplate-rich code and don't really learn. The only outcome here is code quality is gonna go down very very fast.

I envy the people working at mystical places where humans were on average writing code of high quality prior LLMs. I'll never know you now.

Re: Some thoughts on LLMs and software development

#289

Post by Kiro »

johnnienaked wrote: post #260
raducu wrote: post #228

Earlier quoted context omitted.

> No one seems to be able to grasp the possibility that AI is a failure. Do you think by the time GPT-9 comes, we'll say "That's it, AI is a failure, we'll just stop using it!" Or do you speak in metaphorical/bigger picture/"butlerian jihad" terms?

I don't see the use-case now, maybe there will be one by GPT-9

Absence of your need isn't evidence of no need.

Re: Some thoughts on LLMs and software development

#290

Post by ngc248 »

jstrieb wrote: post #214
oo0shiny wrote: post #122

> My former colleague Rebecca Parsons, has been saying for a long time that hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful. What a great way of framing it. I've been trying to explain this to people, but this is a succinct version of what I was stumbling to convey.

I have been explaining this to friends and family by comparing LLMs to actors. They deliver a performance in-character, and are only factual if it happens to make the performance better. https://jstrieb.github.io/posts/llm-thespians/

A better analogy is of an overconfident 5 year old kid, who never says that they don't know the answer and always has an "answer" for everything.
Post reply on HN