Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's talk about LLMs

b-list.org

Post reply on HN
Print view View original

31–40 of 201 posts

Re: Let's talk about LLMs

#31

Post by slopinthebag »

I really enjoyed this article, it's well written and does a good job of dismantling the flawed arguments by language model maxis' while presenting a more realistic outlook on where we are now and where we are going.

I think the biggest benefit language models have provided me is in the auxiliary aspects to programming: search, debugging, rubber ducking, planning, refactoring. The actual code generation has been mixed.

I had an LLM try and implement a fairly involved feature the other day, providing it with API spec details, examples from other open source libraries, and plenty of specifications. It's also something readily available in training data as well, but still fairly involved.

On first glance it looked great, and had I not spent the time to investigate deeper I would have missed some glaring deficiencies and omissions that render its implementation worthless. I am now going back and writing it by hand, but with language models providing assistance along the way, and it's going much better.

I think people are being unrealistic by thinking that the usage of language models in their side projects represent something broader. It's almost the perfect situation for language models: small, greenfield code bases, no review, no responsibility, and no users. It goes up on GitHub with a pretty readme, and then off to social media where they post about how developers are "cooked". It's just not a very realistic test.

In the end we will probably see large productivity increases by integrating language models, but they won't be replacing developers but rather augmenting them.

Re: Let's talk about LLMs

#32

Post by slopinthebag »

atleastoptimal wrote: post #16

"LLM's Aren't Going to Fundamentally Change Software Development" Says Increasingly Nervous Man For Seventh Time This Year

I didn't get the sense that the author is nervous. What I tend to see are people who are nervous that going all-in on LLM workflows might not have the payoff they are expecting, and are becoming increasingly fanatical as a result.

Just one more harness bro. Just one more agentic swarm. Please bro, just one more Claude Max subscription. Please bro.

Re: Let's talk about LLMs

#34

Post by oytis »

mfro wrote: post #4

I think you're misunderstanding the paradigm shift completely -- AI does not just generate code N(x) more quickly. It thinks N(x) faster, it researches N(x) faster, it tests N(x) faster. There are hundreds of tasks that you'll find engineers are offloading to AI every day. The major hurdle right now is actually pivoting LLMs from just generating code: integrating those tasks into workflows. This is why tool-use and a…

The article addresses exactly this objection. Most importantly, it quotes that AI coding tools have a detrimental effect on software stability - which is basically raison d'etre for our profession. When it produces more robust software and handles on-call shifts better than humans, I will consider programming done.

Re: Let's talk about LLMs

#35

Post by slopinthebag »

AIorNot wrote: post #12

the problem with this article is that he is right of course, but only right now. There is no reason to believe that future AI platforms won't be able to review code themselves and manage some aspects of themselves with minimal human oversight, yes we likely will always need a few experts I'm reminded of this scene from the Matrix: https://www.youtube.com/watch?v=cD4nhYR-VRA where the older wise man discusses societie…

> There is no reason to believe that future AI platforms won't be able to review code themselves and manage some aspects of themselves with minimal human oversight

Really? That's like someone during an economic boom saying "The economy is the worst it'll ever be. There is no reason to expect things to not continue to improve".

Re: Let's talk about LLMs

#36

Post by slopinthebag »

taormina wrote: post #26
keybored wrote: post #8

I have no stake in Fred Brooks. But No Silver Bullet seemed to be taken as gospel on this board. Sufficiently productivity-enhancing technology? Gimme a break man. Maybe you’ll get a 30% boost. Not a 10X boost. Until recently. dramatic pause And then AI happened.

Great! So all of this 10x boosting is visible in which economic indicator?

Debt.

Re: Let's talk about LLMs

#37

Post by jb1991 »

michaelchisari wrote: post #20
mfro wrote: post #4

I think you're misunderstanding the paradigm shift completely -- AI does not just generate code N(x) more quickly. It thinks N(x) faster, it researches N(x) faster, it tests N(x) faster. There are hundreds of tasks that you'll find engineers are offloading to AI every day. The major hurdle right now is actually pivoting LLMs from just generating code: integrating those tasks into workflows. This is why tool-use and a…

Debugging, sanity checking, testing, etc. are the best uses of LLMs. Much better than writing code. Developers should write their own code and use LLMs to design and verify. Better, faster architecture and planning, pre-cleaned PRs and no skill atrophy or loss of understanding on the part of the developer.

Funny, I have the complete opposite impression after using claude code for a while. I would never trust it to design anything. Never again. But it can code pretty well given a very tight and limited scope.

Re: Let's talk about LLMs

#38

Post by michaelchisari »

jb1991 wrote: post #37
michaelchisari wrote: post #20

Earlier quoted context omitted.

Debugging, sanity checking, testing, etc. are the best uses of LLMs. Much better than writing code. Developers should write their own code and use LLMs to design and verify. Better, faster architecture and planning, pre-cleaned PRs and no skill atrophy or loss of understanding on the part of the developer.

Funny, I have the complete opposite impression after using claude code for a while. I would never trust it to design anything. Never again. But it can code pretty well given a very tight and limited scope.

To clarify, AI should not do the design itself. You develop the design in conversation with AI.

I come in knowing what I need to build and at least one idea or more of how it should be done. I present the problem, constraints, potential solutions, and ask for criticisms and alternatives. I can keep it as broad as possible or I can get more granular like struct layouts, api endpoints, etc. I go back and forth until there's an approach I prefer and then I code that approach.

| it can code pretty well given a very tight and limited scope.

It's wildly better at tight and limited scope than large scale changes but even then I would rather code it myself.

Re: Let's talk about LLMs

#39

Post by CamperBob2 »

frizlab wrote: post #21
AIorNot wrote: post #12

the problem with this article is that he is right of course, but only right now. There is no reason to believe that future AI platforms won't be able to review code themselves and manage some aspects of themselves with minimal human oversight, yes we likely will always need a few experts I'm reminded of this scene from the Matrix: https://www.youtube.com/watch?v=cD4nhYR-VRA where the older wise man discusses societie…

> There is no reason to believe that future AI platforms won't be able to review code themselves and manage some aspects of themselves with minimal human oversight There are, IMHO, fewer reasons to believe they will be able to do that rather than not, though.

LLMs became much better at both reviewing and writing code over the last 12-18 months. Did you?

The current state of the art is irrelevant. Only the first couple of time derivatives matter.

Re: Let's talk about LLMs

#40

Post by pingou »

smartmic wrote: post #19
pingou wrote: post #11

Earlier quoted context omitted.

Not sure why you are downvoted but I agree. Additionally, perhaps LLMs are just like another higher programming language as the author said, and they still need someone to steer them. I'm sure it was very difficult to program in machine code, but if now (or soon) anyone can just write software using a LLM without any sort of learning it changes everything. LLMs can plan and create something usable from simple instruc…

> I'm sure it was very difficult to program in machine code, but if now (or soon) anyone can just write software using a LLM without any sort of learning it changes everything. LLMs can plan and create something usable from simple instructions or ideas, and they will only get better. Did you read the section "Power to the People?" ? In it, the author dismantles your thesis with powerful, highly plausible arguments.

I have indeed missed the arguments that are so powerful that they dismantles my thesis.

Would there even be a debate in the tech community if such unassailable arguments existed? The author is entirely entitled to his opinion, just as I am allowed to disagree with him (not sure why I am also downvoted). The good thing is, if I'm right, we will see it in less than 10 years.

Post reply on HN