Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Anyone struggling to get value out of coding LLMs?

news.ycombinator.com

Post reply on HN
Print view View original

171–180 of 293 posts

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#171

Post by bigtex »

sph wrote: post #118
Philip-J-Fry wrote: post #41

Earlier quoted context omitted.

I find LLMs 100x more productive for greenfield work. If I want to create a React app with X amount of pages, some Redux stores, Auth, etc. then it can smash that out in minutes. I can say "now add X" and it'll do it. Generally with good results. But when it comes to maintaining existing systems, or adding more complicated features, or needing to know business domain details, a LLM is usually not that great for me. T…

> I find LLMs 100x more productive for greenfield work. Greenfield != boilerplate and basic CRUD app. I'm a consultant writing greenfield apps solo, and 90% of my time is spent away from my editor thinking, planning, designing, meeting with stakeholders. I see no benefit in using a low-IQ autocomplete tool to automate a small part of the remaining 10% of the job, the easiest and most enjoyable part in fact. Personall…

You shared my sentiments exactly. What none of these "all junior developers will be out of a job by 2026" proclamations never deal with is the non coding stuff. Sure it can generate a boiler plate app in 1.5 seconds, but can in communicate with a stakeholder about the requirements and ask the right questions to determine the scope and importance of features.? I just imagine my Sales President spending more 90 seconds trying to write the proper prompt before he gives up and calls and talks to a human. There is just no way a C-suite executive is sitting in front of a computer typing in prompts.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#172

Post by ChrisClark »

igor47 wrote: post #149
TimPC wrote: post #141

LLMs give you superpowers if you work on a wide range of tasks and aren't a deep expert on all of them. They aren't that great if you know the area you are working in inside and out and never stray from it. For example, using an LLM to help you write a Dockerfile when you write Dockerfiles once a project and don't have a dedicated expert like a Deployment Engineer in your company is fantastic. Or using an LLM to get…

> For example, using an LLM to help you write a Dockerfile when you write Dockerfiles once a project and don't have a dedicated expert like a Deployment Engineer in your company is fantastic. This made me think of gell-mann amnesia: https://en.m.wikipedia.org/wiki/Gell-Mann_amnesia_effect Basically, when you understand the problem domain, you know the llm is generating crap that, as the OP says, you're unwilling to p…

As long as it's a simple enough problem and it works, I'm happy. The other day I needed to do something in powershell, which I probably use twice a year. But an LLM was just able to spit out what I needed for me.

I find it's the best for that but once things get complicated, you're just arguing with the LLM, when you should have done it yourself.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#173

Post by max_on_hn »

gyomu wrote: post #13

There are two kinds of engineers. Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before. And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best. The only thing I don’t understand is why people from the former gro…

(disclaimer: I have a vested interest in the space as the purveyor of an AI software development agent)

The result you described is coming soon. CheepCode[0] agents already produce working code in a satisfying percentage of cases, and I am at most 3 months away from it producing end-to-end apps and complex changes that are at least human-quality. It would take way less if I got funded to work on it full time.

Given that I'm this close as a solo founder with no employees, you can imagine what's cooking inside large companies.

[0] My product, cloud-based headless coding agents that connect directly to Linear, accept tickets, and submit GitHub PRs

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#174

Post by decGetAc »

Maybe I'm using these models incorrectly but I just don't ask it to do a lot at once and find it extremely useful.

Write tests for x in the style of this file to cover a, b, c.

Help me find a bug here within this pseudo code that covers three classes and a few functions. Here's the behavior I see, here's what I think could be happening.

I rarely give it access to all the code and usually give it small portions of code and ask for small things. I basically treat it as if I was reaching out to another senior developer in a large company or SO. They don't care to learn about all the details that don't matter, and want a good promoted question that's not wasting their time and that they can help with.

Using it this way I absolutely see the benefits and I'd say an arbitrary 1.25x sounds right (and I'm an experienced engineer in my field).

I'll just quietly keep using it this way and ignore the overwhelming hype on both sides (it's not a speed up camp and it's 100x camp. Imo both are wrong but the it's not a speed up camp make me question how they're using it the most

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#176

Post by dowager_dan99 »

irrational wrote: post #170

At my current job, I need to write quite a bit of python. I been programming for enough decades that I can look at Python code and tell what it is doing, but creating it from scratch? No. But copilot “knows” Python and can write the code that I can then read and tweak. Maybe someone who actually learned Python would write the code differently, but so far it works very well.

I did this last week and the Python code it created technically "worked" but felt on par with your typical Excel VBA macro. Not something you'd want to maintain or show anyone else

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#177

Post by [deleted] »

decGetAc wrote: post #174

Maybe I'm using these models incorrectly but I just don't ask it to do a lot at once and find it extremely useful. Write tests for x in the style of this file to cover a, b, c. Help me find a bug here within this pseudo code that covers three classes and a few functions. Here's the behavior I see, here's what I think could be happening. I rarely give it access to all the code and usually give it small portions of cod…

[deleted]

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#178

Post by sawmurai »

Today I tried to use Copilot to go through a code base and export unexported types, interfaces and unions (typescript). It had to also rename them to make sure they are unique in context of the package. Otherwise I would have used search&replace :)

It started out promising, renaming the symbols according to my instructions. Slower than if I had done it myself, but not horrible slow. It skipped over a few renames so I did them manually. I had to tell it to continue every 2 minutes so I could not really do anything else in the meantime.

I figured it’s quicker if I find the files in question (simple ripgrep search) and feed them to copilot. So I don’t have to wait for it to search all files.

Cool, now it started to rename random other things and ignored the naming scheme I taught it before. It took quite some time to manually fix its mess.

Maybe I should have just asked it to write a quick script to do the rename in an automated way instead :)

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#179

Post by AdrianB1 »

I tried using LLMs for 2 types of work:

1. Improve the code that I already have. Waste of time, it never works. This is not because my code is too good, but because it is SQL with complex context and I get more hallucinations than usable code, still the usable code is good for basic tasks and not for more.

2. Areas I rarely use and I don't maintain an expertise on. This is where it is good value, I get 80% of what I need in 20% of the time, I take it and complete the work. But this does not happen too often, so the overall value is not there yet.

In a way is like RPA: it does something, not great, but it saves some time.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#180

Post by zkry »

Use cases like the ones you mentioned having are truly amazing. It's a shame that the AI hype machine has left us thinking of these use cases as practically nothing, leaving us disappointed.

My belief is that true utility will make itself apparent and won't have to be forced. The usages of LLMs that provide immense utility have already spread across most the industry.

Post reply on HN