Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Anyone struggling to get value out of coding LLMs?

news.ycombinator.com

Post reply on HN
Print view View original

121–130 of 293 posts

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#121

Post by deadeye »

Philip-J-Fry wrote: post #41
gyomu wrote: post #13

There are two kinds of engineers. Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before. And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best. The only thing I don’t understand is why people from the former gro…

I find LLMs 100x more productive for greenfield work. If I want to create a React app with X amount of pages, some Redux stores, Auth, etc. then it can smash that out in minutes. I can say "now add X" and it'll do it. Generally with good results. But when it comes to maintaining existing systems, or adding more complicated features, or needing to know business domain details, a LLM is usually not that great for me. T…

Yes! This is exaclty my experience: https://youtube.com/shorts/1_rSrkXovOk?si=AOBFN3l9_2VlzpDx

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#122

Post by kordlessagain »

I'm using Claude Desktop to develop an agentic crawler system. My workflow is VSCode next to Claude Desktop. The tooling for Claude uses a bunch of MCP servers I wrote for Evolve: https://github.com/kordless/gnosis-evolve

Core Development Capabilities:

- File Discovery & Navigation: file_explorer with pattern matching and recursive search

- Intelligent Code Search: search_in_file_fuzzy with similarity thresholds for finding relevant code sections

- Advanced Code Editing: file_diff_writer with fuzzy matching that can handle code changes even after refactoring

- Backups: backup and restores of any files at any state of change.

- System Monitoring: Real-time log analysis and container management

- Hot Deployment: docker_rebuild for instant container updates (Claude can do the rebuild)

The Agentic Workflow:

- Claude searches your codebase to understand current implementation

- Uses fuzzy search to find related code patterns and dependencies

- Makes intelligent edits using fuzzy replacement (handles formatting changes)

- Monitors logs to verify changes work correctly

- Restarts containers as needed for testing

- Iterates based on log feedback

- Error handling requires analyzing logs and adjusting parsing strategies

- Performance tuning benefits from quick deploy-test-analyze cycles

I've not had any issues with Claude being able to handle changes, even doing things like refactoring overly large HTML files with inline CSS and JS. Had it move all that to a more manageable layout and helped out by deleting large blocks when necessary.

The fuzzy matching engine is the heart of the system. It uses several different strategies working in harmony. First, it tries exact matching, which is straightforward. If that fails, it normalizes whitespace by collapsing multiple spaces, removing trailing whitespace, and standardizing line breaks, then attempts to match again. This handles cases where code has been reformatted but remains functionally identical.

When dealing with multi-line code blocks, the system gets particularly clever. It breaks both the search text and the target content into individual lines, then calculates similarity scores for each line pair. If the average similarity across all lines exceeds the threshold, it considers it a match. This allows it to find code blocks even when individual lines have been slightly modified, variable names changed, or indentation adjusted.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#124

Post by melvinroest »

namaria wrote: post #87
melvinroest wrote: post #57

Earlier quoted context omitted.

Is it? I didn't bother to check as it wasn't too relevant. I wasn't stuck anymore. Worst-case it's a memnonic that is untrue. Best case: it is. For actual derivations I check the work. This is what it wrote by the way. I posted an image and asked: Why is this m m for angle mangle? ChatGPT: In geometry we often need to distinguish the angle itself (a geometric object) from the numerical size of that angle (a real-numb…

I thought it was a different kind of nonsense, but it still has a subtle error. Goes to show even more how risky it is to repeat LLM answers as factual.

Could you point out what you mean? It's really hard to follow you. You say it's nonsense but it is not clear why. Then I write down a response that took me quite a while to format correctly (and to look up) and you then say "I thought it was a different kind of nonsense." Then you say it still has a subtle error.

It is really hard to follow you if you don't explain yourself.

I'm not saying it's factual. The reason I showed that answer was simply to verify to see if it was what you thought it was (hence I asked "is it?"). It turns out that it wasn't fully.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#125

Post by nico »

It really depends on the task and your preferences

I’ve had great success using LLMs for things that I haven’t done in a while or never before. They allow me to build without getting too bogged down into the details of syntax

Yes, they require constant attention, they are not fully independent or magical. And if you are building a project for the longer run, LLM-driven coding slows down a lot once the code base grows beyond just a couple of basic files (or when your files start getting to about 500-800+ lines)

I’ve tried several agentic editors and tools, including cursor, they can def be helpful, but I’d rather just manually loop between ChatGPT (o4-high-mini for the most part) and the editor. I get a very quick and tight feedback loop in which I get plenty of control

Git is essential for tracking changes, and tests are gold once you are at a certain size

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#126

Post by wooque »

gyomu wrote: post #13

There are two kinds of engineers. Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before. And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best. The only thing I don’t understand is why people from the former gro…

Because code is small part of making successful business and usually not the bottleneck.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#127

Post by michaelrpeskin »

gyomu wrote: post #13

There are two kinds of engineers. Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before. And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best. The only thing I don’t understand is why people from the former gro…

A little snarky but: In my experience, the folks who are 100x more productive are multiplying 100 times a small number.

I've found great success with LLMs in the research phase of coding. Last week I needed to write some domain-specific linear algebra and because of some other restrictions, I couldn't just pull in LAPACK. So I had to hand code the work (yes, I know you shouldn't hand code this kind of stuff, but it was a small slice and the domain didn't require the fully-optimized LAPACK stuff). I used an LLM to do the research part that I normally would have had to resort to a couple of math texts to fully understand. So in that case it did make me 100x more effective because it found what I needed and summarized it so that I could convert it to code really quickly.

For the fun of it, I did ask the LLM to generate the code for me too, and it made very subtle mistakes that wouldn't have been obvious unless you were already an expert in the field. I could see how a junior engineer would have been impressed by it and probably just check it in and go on.

I'm still a firm believer in understanding every bit of code you check in, so even if LLMs get really good, the "code writing" part of my work probably won't ever get faster. But for figuring out what code to write - I think LLMs will make people much faster. The research and summarize part is amazing.

The real value in the world is synthesis and novel ideas. And maybe I'm a luddite, but I still think that takes human creativity. LLMs will be a critical support structure, but I'm not sold on them actually writing high-value code.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#128

Post by tonyhart7 »

loveparade wrote: post #22
gyomu wrote: post #13

There are two kinds of engineers. Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before. And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best. The only thing I don’t understand is why people from the former gro…

That's because the reality is somewhere in the middle. It obviously isn't 100x or even 10x except for very specific toy tasks, but for me it's probably around ~1.5-2x in everyday work. And I work on mostly long-tail scientific stuff, I imagine it's more if you do something like frontend dev with a lot of boilerplate. I'm absolutely convinced that people who say LLMs are not making them more productive either don't un…

"That kind of productivity increase is absolutely huge."

then company decide to hire less, the market flooded with junior AI powered skill, mid level programmer cant asking rise as easy anymore unless you are senior dealing with really specific stuff that need know how

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#129

Post by Phelinofist »

The company I work for rolled out GitHub Copilot. It is quite underwhelming honestly. We have a lot of homegrown frameworks that it does not know about (I mean, to be fair, it actually cannot know about these). When I asked it to explain some piece of code it just repeated what was stated in the JavaDoc, nearly verbatim. I had a class that had obvious null issues and I asked it to fix it and it introduced quite a lot of unnecessary "// we have to check it is not null "-style comments.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#130

Post by jghn »

gyomu wrote: post #13

There are two kinds of engineers. Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before. And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best. The only thing I don’t understand is why people from the former gro…

> There are two kinds of engineers.

Apparently there are at least three as I fit neither of these molds of yours. They are neither making me 100x more productive, nor am I putting in an extreme amount of coddling.

For context, in the 30ish years I've been writing code professionally, I've *always* needed to look stuff up constantly. I know what I want, and I know what's possible. I just can never remember things like syntax, specifics, that sort of thing. It doesn't help that I often wind up in jobs where I'm switching between languages often.

Before quality search engines like Google, this meant I was always flipping through reference books, man pages, and worse. Then came search engines and that was a step more productive. Then came things like SO, and that was a step more productive. I find LLMs to be another step forward, but not like a quantum leap or anything. Take autocomplete suggestions for instance: Sometimes it's exactly the prompt (pardon the pun) that I need to jog my memory. Other times I know it's not what i want and I ignore it. And the chat interfaces I find better than Googling SO as I can have a back & forth with it.

Post reply on HN