Some thoughts on LLMs and software development • Hacker News Board

Re: Some thoughts on LLMs and software development

#201

Post by cortesoft » Fri, Aug 29, 2025, 3:59 AM UTC

furyofantares wrote: post #99

nicwolff wrote: post #81

> I’ve often heard, with decent reason, an LLM compared to a junior colleague. No, they're like an extremely experienced and knowledgeable senior colleague – who drinks heavily on the job. Overconfident, forgetful, sloppy, easily distracted. But you can hire so many of them, so cheaply, and they don't get mad when you fire them!

These metaphors all suck. Well, ok, yours is funny. But anyway, LLMs are just very different from any human. They are extremely shallow, even compared to a junior developer. But extremely broad, even compared to the most experienced developer. They type real fuckin fast compared to anyone on earth, but they need to be told what to do much more carefully than anyone on earth.

And they are incessantly cheerful.

I asked Claude to design me a UI, and it made a lovely one... but I wanted a web ui. It very happily through away all its work and made a brand new web UI.

I can't imagine any employee being that quick to just move on after something like that.

Re: Some thoughts on LLMs and software development

#202

Post by makeitdouble » Fri, Aug 29, 2025, 4:03 AM UTC

tptacek wrote: post #154

makeitdouble wrote: post #126

Earlier quoted context omitted.

That was an interesting take, but "probabilistic" is to me different from "random". In particular other field get error tolerances, LLMs give us nothing like that. We're introducing chaos monkeys, not just variability.

Note that he's talking about the same nondeterminism in that post that we're talking about here.

From the linked comment

> Process engineers for example have to account for human error rates. [...] Designing systems to detect these errors (which are highly probabilistic!)

> Likewise even for regular mechanical engineers, there are probabilistic variances in manufacturing tolerances.

I read them as relatively confined, thus probabilistic. When a human pushes the wrong button, an elephant isn't raining from the sky. Same way tolerances are bounded.

When requesting a JSON array from an LLM, it could as well decide this time that JSON is a mythological Greek fighter.

Re: Some thoughts on LLMs and software development

#203

Post by tptacek » Fri, Aug 29, 2025, 4:04 AM UTC

makeitdouble wrote: post #202

tptacek wrote: post #154

Earlier quoted context omitted.

Note that he's talking about the same nondeterminism in that post that we're talking about here.

From the linked comment > Process engineers for example have to account for human error rates. [...] Designing systems to detect these errors (which are highly probabilistic!) > Likewise even for regular mechanical engineers, there are probabilistic variances in manufacturing tolerances. I read them as relatively confined, thus probabilistic. When a human pushes the wrong button, an elephant isn't raining from the sk…

I'm just saying, it's a cite to a thread about the implications of AI-based nondeterminism, just like this one. That's all.

Re: Some thoughts on LLMs and software development

#204

Post by FlyingSnake » Fri, Aug 29, 2025, 4:16 AM UTC

nicwolff wrote: post #81

> I’ve often heard, with decent reason, an LLM compared to a junior colleague. No, they're like an extremely experienced and knowledgeable senior colleague – who drinks heavily on the job. Overconfident, forgetful, sloppy, easily distracted. But you can hire so many of them, so cheaply, and they don't get mad when you fire them!

“You’re absolutely right!”

Re: Some thoughts on LLMs and software development

#205

Post by thrown-0825 » Fri, Aug 29, 2025, 4:20 AM UTC

Dont forget the gas lighting when they make a mistake and try to act like it was you.

Re: Some thoughts on LLMs and software development

#206

Post by DrewADesign » Fri, Aug 29, 2025, 4:22 AM UTC

vkou wrote: post #93

nicwolff wrote: post #81

> I’ve often heard, with decent reason, an LLM compared to a junior colleague. No, they're like an extremely experienced and knowledgeable senior colleague – who drinks heavily on the job. Overconfident, forgetful, sloppy, easily distracted. But you can hire so many of them, so cheaply, and they don't get mad when you fire them!

Experienced and knowledgeable and they also believe in the technical equivalent of flat-Earthism in many, many non-trivial corners. And if you push back on that insanity, they'll smile and nod and agree with you and in the next sentence, go right back to pushing that nonsense.

“Here’s the code:” looks totally plausible but hallucinated libFakeCall.

“libFakeCall doesn’t exist. Use libRealCall instead of libFakeCall.”

“You’re absolutely correct. I apologize for blah blah blah blah. Here’s the updated code with libRealCall instead. :[…]”

“You just replaced the libFakeCall reference with libRealCall but didn’t update the calls themselves. Re-write it and cite the docs. “

“Sorry about the confusion! I’ve found these calls in the libRealCall docs. Here’s the new code and links to the docs.”

“That’s the same code but with links to the libRealCall docs landing page.”

“You’re absolutely correct.” It appears that these calls belong to another library with that functionality:” looks totally plausible but hallucinated libFakeCall.

Re: Some thoughts on LLMs and software development

#207

Post by platevoltage » Fri, Aug 29, 2025, 4:22 AM UTC

TexanFeller wrote: post #133

jack_pp wrote: post #121

Earlier quoted context omitted.

Fwiw I think the ratio of times I needed to go to google for a solution instead of an LLM is like 20:1 for me so your mileage may vary. Depends a lot on the specific niche you're working in. Unrelated to software but recently I wanted to revive an old dumbphone I haven't used since 2014 and apparently I had it password protected and forgot the password and wanted to factory reset it. I found the exact model of the ph…

Google search has been enshittified. Kagi is where you get real search results now.

Unsure why this is downvoted. "Google search has been enshittified" should be a common sentiment here.

Re: Some thoughts on LLMs and software development

#208

Post by DrewADesign » Fri, Aug 29, 2025, 4:27 AM UTC

gck1 wrote: post #148

nicwolff wrote: post #81

> I’ve often heard, with decent reason, an LLM compared to a junior colleague. No, they're like an extremely experienced and knowledgeable senior colleague – who drinks heavily on the job. Overconfident, forgetful, sloppy, easily distracted. But you can hire so many of them, so cheaply, and they don't get mad when you fire them!

> Overconfident, forgetful, sloppy, easily distracted And constantly microdosing, sometimes a bit too much.

And sometimes it seems like a macro-dose huffing xylene-based glue.

Re: Some thoughts on LLMs and software development

#209

Post by vkou » Fri, Aug 29, 2025, 4:36 AM UTC

DrewADesign wrote: post #206

vkou wrote: post #93

Earlier quoted context omitted.

Experienced and knowledgeable and they also believe in the technical equivalent of flat-Earthism in many, many non-trivial corners. And if you push back on that insanity, they'll smile and nod and agree with you and in the next sentence, go right back to pushing that nonsense.

“Here’s the code:” looks totally plausible but hallucinated libFakeCall. “libFakeCall doesn’t exist. Use libRealCall instead of libFakeCall.” “You’re absolutely correct. I apologize for blah blah blah blah. Here’s the updated code with libRealCall instead. :[…]” “You just replaced the libFakeCall reference with libRealCall but didn’t update the calls themselves. Re-write it and cite the docs. “ “Sorry about the confu…

You forgot to include it's best excuse for why the documentation it cites is hallucinated: "The links may be broken/website is down".

For all the hot air I hear about the user having to give the system the proper context to give you good answers - does anyone claim to have a solution for dealing with such a belligerent approach to bullshit?

They are by no means useless, but once they fall into that hole, there's no further value in interrogating them.

Re: Some thoughts on LLMs and software development

#210

Post by hatefulmoron » Fri, Aug 29, 2025, 4:39 AM UTC

ninetyninenine wrote: post #119

BlueTemplar wrote: post #94

Earlier quoted context omitted.

LLMs don't hallucinate : they bullshit (which is not caring about truth).

This isn't a good characterization of it either. I don't think LLMs know the difference. Bullshit implies they are lying. It's possible LLMs are lying but my guess is that they really just can't tell the difference.

They're using the term 'bullshit' as it is understood as a term of art, which doesn't imply lying. It's closer to creating a response without any regard for telling the truth. Bullshitting is often most effective when you happen to be telling the truth, although the bullshitter has no commitment to that.