Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Anyone struggling to get value out of coding LLMs?

news.ycombinator.com

Post reply on HN
Print view View original

231–240 of 293 posts

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#231

Post by t917910 »

mbac32768 wrote: post #106
lm28469 wrote: post #18

Earlier quoted context omitted.

That's because they fall under the "engineers" category like my local postman and Michael Schumacher both fall under the "driver" category. Most people don't work on anything technically hard, most problems are business logic issues that aren't solved technically or legacy code workarounds for which you need to put 3-10 domain experts in a room for a few hours to solve.

Right yes, I imagine most people who program are in companies that have nothing to do with high tech, writing SQL or scripts or CRUD apps. LLMs are probably amazing here.

Writing SQL in the context of 20 years old application with 2500 stored procedures just inside one of many databases is not so easy for LLM.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#232

Post by t917910 »

johnecheck wrote: post #194
max_on_hn wrote: post #173

Earlier quoted context omitted.

(disclaimer: I have a vested interest in the space as the purveyor of an AI software development agent) The result you described is coming soon. CheepCode[0] agents already produce working code in a satisfying percentage of cases, and I am at most 3 months away from it producing end-to-end apps and complex changes that are at least human-quality. It would take way less if I got funded to work on it full time. Given t…

"At most 3 months away" This is a strong claim. I'm not aware of any AI model or system that can consistently and correctly make complex changes in existing codebases. What makes you so confident that you can achieve this, alone, in the next 3 months?

The circumference of a circle is easily calculated as piradius2, so calculating the same length of an ellipse would not be much more difficult. Hehe.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#233

Post by t917910 »

namaria wrote: post #50
belZaah wrote: post #25

Earlier quoted context omitted.

There’s no silver bullet. Conceptualization is the hard part, exactly as you say. Mythical man-month is an important text and should be studied more.

And also the seminal papers by Gödel, Turing, and the books on cybernetics (robotics and automation in 1950s) by Ashby and Wiener. The limits of what can be automated have been clear for almost a century now. Yet people insist in selling ways to automate your way out of social, political, economical problems. Come to think about it, people should also read Comways's paper as well.

What is Comways's paper?

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#234

Post by _boffin_ »

maxnevermind wrote: post #203
_boffin_ wrote: post #198

I just had an llm build the my side business’s website and full client portal. About 15k loc. It’s amazing. Better design in terms of UI / UX than I could have fathomed and so much more. There’s a lot of duplicated code that I’ll clean up, but the site functions and will be launched for clients to start using soon. For my day job, it’s also helping me build out the software at a faster pace than before and is an amaz…

Have you read through all 15K loc, What is your feeling about maintainability of it? Also as I understand one the main problem with LLM right now is trying to apply some surgical changes to large enough code base or adding some extra functionality without breaking/altering existing ones, have you faces issues with that? Another issue is security, I've heard some horror stories of non-tech people developing web soluti…

I’m finishing up some hardening features and some additionally functionality right now. It’s “maintainable” par se, but I will be going through it and cleaning it up a lot. It’s a Django application. It likes to write out large single files. Already went through to break things up into similar views and functionality.

And as for security., were they even caring about it? Were they asking good questions? Reading about things? Learning ways to secure things?

If someone blindly follows what it gives you, something interesting will happen at some point. Use it as an enhancer.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#235

Post by noosphr »

joshstrange wrote: post #51
Philip-J-Fry wrote: post #41

Earlier quoted context omitted.

I find LLMs 100x more productive for greenfield work. If I want to create a React app with X amount of pages, some Redux stores, Auth, etc. then it can smash that out in minutes. I can say "now add X" and it'll do it. Generally with good results. But when it comes to maintaining existing systems, or adding more complicated features, or needing to know business domain details, a LLM is usually not that great for me. T…

I couldn’t agree more and I’ve said the same thing many times. I have yet to see an LLM-generated app not collapse under it’s own weight after enough iterations/prompts. It gets stuck in loops (removing and adding the same code/concept), it gets hung up on simple errors, etc. For greenfield it’s amazing, no doubt, but unless you are watching it very closely and approving/reviewing the code along the way it will go of…

>I expect things to get better, this will not always be the state of things, but for now “vibe coding” (specifically not reviewing/writing code yourself) is not sustainable.

It will not.

And I say this as someone whose been building internal LLM tools since 2021.

The issue is their context window. If you increase the context window so they can see more code costs skyrocket as n^2 the size of the code base. If you don't then you have all the issues people have in this thread.

The reason why I have a job right now is that you can get around this by building tooling for intelligent search that limits the overfill of each context window. This is neither easy, fast, or cheap when done at scale. Worse the problems that you have when doing this are at best very weakly related to the problems the major AI labs are focusing on currently - I've interviewed at two of the top five AI labs and none of the people I talked to cared or really understood what a _real_ agentic system that solves coding should look like.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#236

Post by ungreased0675 »

bigtex wrote: post #171
sph wrote: post #118

Earlier quoted context omitted.

> I find LLMs 100x more productive for greenfield work. Greenfield != boilerplate and basic CRUD app. I'm a consultant writing greenfield apps solo, and 90% of my time is spent away from my editor thinking, planning, designing, meeting with stakeholders. I see no benefit in using a low-IQ autocomplete tool to automate a small part of the remaining 10% of the job, the easiest and most enjoyable part in fact. Personall…

You shared my sentiments exactly. What none of these "all junior developers will be out of a job by 2026" proclamations never deal with is the non coding stuff. Sure it can generate a boiler plate app in 1.5 seconds, but can in communicate with a stakeholder about the requirements and ask the right questions to determine the scope and importance of features.? I just imagine my Sales President spending more 90 seconds…

I agree with the larger point you’re making, but an LLM absolutely can ask the right questions to determine scope and features for a basic application. It can even turn those into decent user stories. It’s the edge cases and little details that will be messed up.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#237

Post by paulcole »

bagacrap wrote: post #227
paulcole wrote: post #225

> but trying to get it to generate code that I am willing to maintain and "put my name on" took longer than writing the code would have Why don’t you consider that the AI will be the one maintaining it?

If that were possible we'd have already pointed LLMs at the Linux bug tracker (for example) and fixed all the open issues.

Was all that code written by LLMs?

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#238

Post by mlsu »

loveparade wrote: post #22
gyomu wrote: post #13

There are two kinds of engineers. Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before. And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best. The only thing I don’t understand is why people from the former gro…

That's because the reality is somewhere in the middle. It obviously isn't 100x or even 10x except for very specific toy tasks, but for me it's probably around ~1.5-2x in everyday work. And I work on mostly long-tail scientific stuff, I imagine it's more if you do something like frontend dev with a lot of boilerplate. I'm absolutely convinced that people who say LLMs are not making them more productive either don't un…

If it really is 2x in general, we would have noticed.

LLM are useful to me personally as a learning tool. It’s valuable to have it talk through something or explain why something is designed a certain way. So it’s definitely helped me grow as an engineer. But I have not come close to being 2x as productive, I don’t think my workload would be done by 2 people in the pre-LLM era. Maybe 1.25?

Because for the most part being a computer programmer is not about programming the computer but about… everything else.

Overall, LLM are a net negative for humanity in my opinion. The amount of slop on the internet now is absurd. I can never be sure if I’m reading the writing of a real person or a machine, which makes me feel like I’m slowly becoming brain poisoned by the devil himself on a psychological level. I have to think twice when I read any email longer than 2 paragraphs, or did my coworker start using more em dashes?

And also if it’s so damn easy for a machine to do all of this complex valuable work for a few kilojoules how complex or valuable is it anyway? Maybe it’s all just a fiction, that any of this stuff provides any value to anyone. Is another app really actually necessary? What about a hundred million new apps?! Even if we all became 100x as “productive,” what would that actually mean, in the real world you know the one that’s outside that has smells and textures and objects and people in it talking to one another face to face with their wet vocal chords??

These things are going to drive us all absolutely insane I can guarantee it. The logical end of this is complete madness.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#239

Post by qwerasdf5 »

morsecodist wrote: post #107
prmph wrote: post #93

Earlier quoted context omitted.

Ok, that's the "you're holding it wrong" thing. When I start to stray into even moderately complex work, LLMs become pretty useless quick. Tell me your setup, and I will give a quick sample task that it will fail at. Stop the fanboyism please

Thank you. It is so frustrating that you hear that LLMs are a PhD level intelligence capable of any task you can throw at it and when it can't solve your problem you hear: "well you are using a1.36 and b1.37-high is really the one this time" (despite the fact that you have been hearing these claims since before that model came out) or "you are prompting it wrong, have you tried describing all of your app features and…

Who told you that LLMs are a "PHD level intelligence"?

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#240

Post by [deleted] »

esafak wrote: post #222
nottorp wrote: post #219

Earlier quoted context omitted.

Yes but why? On a large code base it would probably work better if you didn't need to put it all in the context... if you even could...

I am saying the same thing; you want to selectively establish the context, not pass it all.

[deleted]
Post reply on HN