Live data from Hacker News

Ask HN: Anyone struggling to get value out of coding LLMs?

news.ycombinator.com

91–100 of 293 posts

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#91
post #13

There are two kinds of engineers. Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before. And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best. The only thing I don’t understand is why people from the former gro…

> And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best.

I am very much in that category.

I would describe my experience as the old adage "why keep a dog and bark yourself".

And no, I don't buy into people who say you have to engage in "prompt engineering". That's bullshit. Let's face it "prompt engineering" is a synonym for "I have to waste time coming up with hacky workarounds to make this dumb LLM come up with something usable".

In the end, its always quicker and easier to do it myself. And unless you are fresh out of school, if you've got more than 10 minutes of coding experience, you will always do a better job than the LLM.

Every time a new LLM model comes out, I fall into the same old trap "oh well, this one must be better, surely" I say to myself.

And yet its always the same "you can't polish a turd but you can roll it in glitter" experience.

I'm sick and tired of LLMs hallucinating shit.

I'm sick and tired of LLMs inventing functions that have been available in stdlib for years.

I'm sick and tired of LLMs generating useless code with boilerplate subfunctions that just contain a single commented "implement it yourself" line.

On top of that LLMs are simply terrible for the environment, guzzling up all that electricity, producing nothing but hot air and bullshit results in return.

Basically I'm sick and tired of LLMs and all the LLM fetishists that surround them.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#93
post #58
post #45

Earlier quoted context omitted.

It's not about the tools. The complaint here is a general one that shows up no matter what tools you are using

The tools do matter. The original complaint is dribble. The poster does not even know what model they used in cursor which is surprising, it’s the most important part of the process. They also compared two entirely brand new tools that are not adjacent to cursor.

Ok, that's the "you're holding it wrong" thing.

When I start to stray into even moderately complex work, LLMs become pretty useless quick. Tell me your setup, and I will give a quick sample task that it will fail at. Stop the fanboyism please

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#94
Give it context and copy paste the code yourself.

They're good at coming up with new code.

Give it function signature with types and it will give pretty good implementation.

Tell it to edit something, and it will lose track.

The write-lint-fix workflow with LLMs doesn't work for me - LLM is monkey brain edits unrelated parts of code.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#95
post #45

Earlier quoted context omitted.

It's not about the tools. The complaint here is a general one that shows up no matter what tools you are using

And I tell from personal experience: The tools matter (!) for more complex problems. Op said literally "Am I just not using the tools correctly?" and I said for the 2 of 3 tools OP tried are so new and experimental I would put them away (for now).

You're still perpetuating this myth that it is about the tools, when every interaction I've had with coding assistants and LLMs veers into the territory OP describes. To be fair, the work I am doing is pretty complex and novel.

And I use a wide range of model/versions.I mostly use Claude, from 3.5 to the newest 4.0. I also use Gemini and with Copilot.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#96
I tend to spend 15mins to write clear requirements (functional and non-functional specs). And then ChatGTP works its miracle. When I ask it "do write code in X language that does so-and-so", it's proper crapware. But those 15mins of reqs, save me 4-5 hours of writing back and forth, looking at manuals, etc (I'm not a super dev, I'm not even a professional dev.) But I do ask it to write code for me as am I trying to solve (my) small IT problems, and from seeing various marketplaces, nobody has posted code/software to do what I want.

Perhaps one day I'll 'incorporate myself' and start posting my solutions and perhaps make some dough.. but the I benefit far more than the $20 a month I am paying.

The right 'prompt' (with plenty of specs and controls) saves me from the (classic!) swing-on-tree example: https://fersys.cloud/wp-content/uploads/2023/02/4.jpg

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#97
I have had a pretty similar experience to you. I have found some value here:

- I find it is pretty good at making fairly self-contained react components or even pages especially if you are using a popular UI library

- It is pretty reliable at making well-defined pure functions and I find it easier to validate that these are correct

- It can be good for boilerplate in popular frameworks

I sometimes feel like I am losing my mind because people report these super powerful end to end experiences and I have yet to see anything close in my day to day usage despite really trying. I find it completely falls over on a complete feature. I tried using aider and people seem to love it but it was just a disaster for me. I wanted to implement a fairly simple templated email feature in a Next.js app. The kind of thing that would take me about a day. This is one of the most typical development scenarios I can imagine. I described the feature in it's entirety and aider completely failed, not even close. So I started describing sub-features one by one and it seemed to work better. But as I added more and more, existing parts began to break, I explained the issues to aider and it just got worse and worse with every prompt. I tried to fix it manually but the code was a mess.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#98
I've only had great luck with the LLMs(chatgpt 3o) generated Perl code. It was able to synthesize code for a GTK2/3 application fairly consistently, without generating any syntax errors. Most of the code worked as described, and it seemed to make more mistakes misunderstand my descriptions of features rather than when implementing them. My colleagues suggested it was because Perl's popularity had fallen significantly before 2016, and the training data set might've had much less noise.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#99
post #44

Earlier quoted context omitted.

> So I ask ChatGPT and one of the things it said is that historically it meant "measure" This is a great example of why using LLMs to learn is perilous. This is nonsense.

> This is a great example of why using LLMs to learn is perilous. This is nonsense. These type of answers from teachers, co-students, web communities, blogs etc. are – I would assume – why people ask LLMs in the first place.

It is a problem that people who are unwilling to perform some basic research resort to 'learning' the output of LLMs. No one is entitled to answers.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#100
post #13

There are two kinds of engineers. Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before. And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best. The only thing I don’t understand is why people from the former gro…

I've managed to use ChatGPT to write a small app in Kotlin, and later used Cursor to fix some issues around performance. Personally I'm not a finish-it kind of guy unless I'm getting paid for it.

To be fair, I'm not sure the quality of the code is much worse than all the boatloads of crapware in all the app stores out there.

I suspect that the reason "revolutionary products" and "blazing fast iteration" isn't blowing our minds is because the problem was never the code - it was always the people around it, who know better than their customers. The best way to make good software is to test it on real people and iterate - that's going to take a long time no matter how fast the coding part is.

Post reply on HN