Ask HN: Anyone struggling to get value out of coding LLMs?

Post by infecto » Tue, May 27, 2025, 10:21 AM UTC

morsecodist wrote: post #107

prmph wrote: post #93

Earlier quoted context omitted.

Ok, that's the "you're holding it wrong" thing. When I start to stray into even moderately complex work, LLMs become pretty useless quick. Tell me your setup, and I will give a quick sample task that it will fail at. Stop the fanboyism please

Thank you. It is so frustrating that you hear that LLMs are a PhD level intelligence capable of any task you can throw at it and when it can't solve your problem you hear: "well you are using a1.36 and b1.37-high is really the one this time" (despite the fact that you have been hearing these claims since before that model came out) or "you are prompting it wrong, have you tried describing all of your app features and…

Post by infecto » Tue, May 27, 2025, 10:38 AM UTC

soraminazuki wrote: post #64

infecto wrote: post #56

Earlier quoted context omitted.

You’ll usually find three kinds of people talking about LLMs online: 1. The ones writing Medium posts about being 100x more productive. 2. The skeptics, like yourself, who construct made-up examples to prove it’s impossible to get value, framing things in a way where failure is the only plausible outcome. 3. And then there’s the middle group, people quietly getting real, uneven, but worthwhile value. You’d be surpris…

GP didn't "construct made-up examples" anywhere in this thread as of the time of writing this post. Where did you hallucinate that from? Seriously, why is every comment hyping up AI generated code like this. No concrete evidence, just turtles all the way down. But when actual examples do show up, it's a complete clown show [1][2]. [1]: https://news.ycombinator.com/item?id=44050152 [2]: https://news.ycombinator.com/it…

Post by infecto » Tue, May 27, 2025, 11:57 AM UTC

morsecodist wrote: post #254

infecto wrote: post #251

Earlier quoted context omitted.

Totally fair frustration. Unfortunately model/version does matter—it’s not pedantry, it’s debugging. And no, you shouldn’t need a prompt engineering PhD to get value, but some structure and awareness of tool limits go a long way.

It's not that I think model version doesn't matter. I switch between them all the time (often to downgrade as much as upgrade honestly). It's that I think people are misrepresenting the kinds of results you can get from these models and seem to take it as a personal attack and come up with excuses when you talk about limitations that you've encountered. It makes it difficult to engage in conversations about tools and…

Post by infecto » Tue, May 27, 2025, 12:05 PM UTC

esafak wrote: post #207

elzbardico wrote: post #61

There's no silver bullet. It is amazing how in our field we repeatedly forget this simple advice from Fred Brooks. In my experience, LLMs are way more useful for coding and less problem-prone when you use them without exaggerated expectations and understand that it was trained on buggy code, and that of course it is going to generate buggy code. Because almost all code is buggy. Don't delegate design for it, use func…

I agree with what you said except the delegation of design, if by which you mean architecture, because I find that it is good at that too. When it comes to design I ask it to describe it, and iterate at a high level before proceeding to implementation. Just as one would in real life.

Post by quartzeee » Tue, May 27, 2025, 12:31 PM UTC

noosphr wrote: post #235

joshstrange wrote: post #51

Earlier quoted context omitted.

I couldn’t agree more and I’ve said the same thing many times. I have yet to see an LLM-generated app not collapse under it’s own weight after enough iterations/prompts. It gets stuck in loops (removing and adding the same code/concept), it gets hung up on simple errors, etc. For greenfield it’s amazing, no doubt, but unless you are watching it very closely and approving/reviewing the code along the way it will go of…

>I expect things to get better, this will not always be the state of things, but for now “vibe coding” (specifically not reviewing/writing code yourself) is not sustainable. It will not. And I say this as someone whose been building internal LLM tools since 2021. The issue is their context window. If you increase the context window so they can see more code costs skyrocket as n^2 the size of the code base. If you don…