Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Anyone struggling to get value out of coding LLMs?

news.ycombinator.com

Post reply on HN
Print view View original

141–150 of 293 posts

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#141

Post by TimPC »

LLMs give you superpowers if you work on a wide range of tasks and aren't a deep expert on all of them. They aren't that great if you know the area you are working in inside and out and never stray from it.

For example, using an LLM to help you write a Dockerfile when you write Dockerfiles once a project and don't have a dedicated expert like a Deployment Engineer in your company is fantastic.

Or using an LLM to get answers faster than google for syntax errors and other minor issues is nice.

Even using LLM with careful prompting to discuss architecture tradeoffs and get its analysis (but make the final decision yourself) can be helpful.

Generally, you want to be very careful about how you constrain the LLM through prompts to ensure you keep it on a very narrow path so that it doesn't do something stupid (as LLMs are prone to do), you also often have to iterate because LLMs will occasionally do things like hallucinate APIs that don't actually exist. But even with iteration it can often make you faster.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#143

Post by slashtmpslashme »

DonHopkins wrote: post #62

I am struggling to programmatically get syntactically valid JSON out of LLMs, using both the OpenAI and Vertex apis. I am using: "response_format": { "type": "json_object" } And with Vertex: "generationConfig": { "responseMimeType": "application/json" } And even: "response_format": { "type": "json_schema", "json_schema": { ... And with Vertex: "generationConfig": { "responseMimeType": "application/json", "responseSch…

Use structured output / response format with JSON schema feature if your models provide it (openai for sure does, I think it's json_schema instead of json_object and provide a schema).

Otherwise there are libraries like instructor (python) which can help to generate json in markdown code blocks.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#144

Post by conductr »

gyomu wrote: post #13

There are two kinds of engineers. Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before. And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best. The only thing I don’t understand is why people from the former gro…

My guess is it depends on what you work with. I’ve been impressed by my ability to spin up nice looking websites and landing pages. Yet, I’ve tried to build a game and despite it appearing to work, it eventually has gotten confused about its own codebase and I end up with a ball of spaghetti where requirements of yesterday get lost or broken when adding functionality today. It feels more like I’m chasing my tail, vibe coding as they say, it’s just seems endlessly circular.

My game dev is going to be a hybrid approach going forward. I’ll ask it to create a new project with the desired functionality and see how the solution was proposed and then incorporate that into my main code branch.

I find this easier than the Cursor roll back or being overly cautious with suggested edits. I’m just getting to this realization so tbd if it works well. FYI, I’m good at programming generally but game dev is my Achilles. We’ll see how it it goes.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#145

Post by peacebeard »

gyomu wrote: post #13

There are two kinds of engineers. Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before. And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best. The only thing I don’t understand is why people from the former gro…

I don’t fit in either category. My experience is that LLMs are good at writing code that is easy to write. This is not game-changing, but it is useful.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#146

Post by l33tman »

s_ting765 wrote: post #138
sph wrote: post #118

Earlier quoted context omitted.

> I find LLMs 100x more productive for greenfield work. Greenfield != boilerplate and basic CRUD app. I'm a consultant writing greenfield apps solo, and 90% of my time is spent away from my editor thinking, planning, designing, meeting with stakeholders. I see no benefit in using a low-IQ autocomplete tool to automate a small part of the remaining 10% of the job, the easiest and most enjoyable part in fact. Personall…

I completely agree with you that the coders who are "smitten" by LLMs are just inexperienced. I personally find that LLMs get subtle improvements in capability over time and it's usually worthwhile to check in on the progress from time to time. Even if it's just for fun.

I can see why you would chime in to say that in your experience you don't get any value out of it, but to chime in to say that the millions of people who do are "inexperienced" is pretty offensive. In the hands of skilled developers these tools are a complete gamechanger.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#147

Post by harrall »

I find Junie (from JetBrains / IntelliJ) great compared to other offerings. I have 10+ years experience as well.

It writes JUnit tests with mocks, chooses to run them, and fixes the test or sometimes my (actually broken) code.

It’s not helpful for 90% of my work, but like having a hammer, it’s good to have when you know that you have a nail.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#148

Post by daveguy »

wejick wrote: post #78
Philip-J-Fry wrote: post #41

Earlier quoted context omitted.

I find LLMs 100x more productive for greenfield work. If I want to create a React app with X amount of pages, some Redux stores, Auth, etc. then it can smash that out in minutes. I can say "now add X" and it'll do it. Generally with good results. But when it comes to maintaining existing systems, or adding more complicated features, or needing to know business domain details, a LLM is usually not that great for me. T…

Not really my case. Found that Codebase with good code benefit more with LLM, but it's not the prerequisite. I just rewrote 300ish advanced PostgreSQL queries to mysql queries. The process is not magical, but it will take me 1 week rather than 3 days. Now I'm on testing phase, seems promising. The point is if we can find a work to work along with the agent, can be very productive.

> I just rewrote 300ish advanced PostgreSQL queries to mysql queries.

Translation from one set of tokens to another is exactly the primary use case of LLMs. This is exactly what it should be good at. Developing new queries, much less so. Translation from one set of database queries to another was already very well defined and well covered before LLMs came about.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#149

Post by igor47 »

TimPC wrote: post #141

LLMs give you superpowers if you work on a wide range of tasks and aren't a deep expert on all of them. They aren't that great if you know the area you are working in inside and out and never stray from it. For example, using an LLM to help you write a Dockerfile when you write Dockerfiles once a project and don't have a dedicated expert like a Deployment Engineer in your company is fantastic. Or using an LLM to get…

> For example, using an LLM to help you write a Dockerfile when you write Dockerfiles once a project and don't have a dedicated expert like a Deployment Engineer in your company is fantastic.

This made me think of gell-mann amnesia: https://en.m.wikipedia.org/wiki/Gell-Mann_amnesia_effect

Basically, when you understand the problem domain, you know the llm is generating crap that, as the OP says, you're unwilling to put your name on. But when it's in a domain you don't know, you forget about how crappy the LLM is normally and go with the vibe.

Post reply on HN