Live data from Hacker News

Ask HN: Anyone struggling to get value out of coding LLMs?

news.ycombinator.com

261–270 of 293 posts

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#261

Earlier quoted context omitted.

It's not that I think model version doesn't matter. I switch between them all the time (often to downgrade as much as upgrade honestly). It's that I think people are misrepresenting the kinds of results you can get from these models and seem to take it as a personal attack and come up with excuses when you talk about limitations that you've encountered. It makes it difficult to engage in conversations about tools and…

When the original post has no clue what model they are using it throws all credibility out the window. At that point it’s appropriate to point that out to them with suggestions. Nobody here was suggesting that LLMs are PhDs like you are saying. You are the only one bringing that up.

> When the original post has no clue what model ...

Well, that's the point. As long as they are using a recent-ish model it really doesn't matter. Not that there are no differences in performances between models, it's that there is no model today that even comes close not requiring extensive hand-holding to accomplish real-world software engineering of even slightly moderate complexity.

Case in point: I have been frustrated that most markdown viewers don't do automatic indentation of section levels. I thought, this is a perfect test of coding assistants: the problem and solution is straightforward conceptually, and I don't even care about the platform and architecture used to accomplish it.

I've asked all the major models to implement a simple markdown viewer that could do automatic indentation, and they all fall flat. Some even give me code that will not run; of the rest, none has provided code that basically does the thing I've asked for.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#262

Earlier quoted context omitted.

> create a React app with X amount of pages, some Redux stores, Auth, etc. Unless you're a contractor making basic websites for small businesses, how many of these do you need to make? This really a small fraction of the job of most developers, except for entry-level devs. > when it comes to maintaining existing systems, or adding more complicated features, or needing to know business domain details, This is what exp…

Your conclusion is wrong I think. LLMs cannot magically replace entry levels devs. Who’s gonna ask the LLM to create the basic website? The product owner? The accountant? The sales guy? They wouldn’t know how to be precise enough to state what they actually need. An entry level engineer would make use of the LLM to produce the website and push it to production. Hell, only engineers know that the devil is in the detai…

> An LLM is very capable of implementing all of that… if only someone who knows all of that stuff tell them first.

I agree with you, but I don't think it's the entry-level dev who is going to be interfacing with the client to discuss and resolve all the questions you posed, and/or decide on them. That was part of the OP's point -- that much of their time is spent interfacing with the client to very precisely determine the requirements.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#263
post #78

Earlier quoted context omitted.

Not really my case. Found that Codebase with good code benefit more with LLM, but it's not the prerequisite. I just rewrote 300ish advanced PostgreSQL queries to mysql queries. The process is not magical, but it will take me 1 week rather than 3 days. Now I'm on testing phase, seems promising. The point is if we can find a work to work along with the agent, can be very productive.

> I just rewrote 300ish advanced PostgreSQL queries to mysql queries. Translation from one set of tokens to another is exactly the primary use case of LLMs. This is exactly what it should be good at. Developing new queries, much less so. Translation from one set of database queries to another was already very well defined and well covered before LLMs came about.

I mean, using a good ORM with DB adapter options could achieve this in minutes. Sure, LLM has utility here for raw queries, hardly "replace SWEs" type of utility though

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#264
post #13

There are two kinds of engineers. Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before. And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best. The only thing I don’t understand is why people from the former gro…

That's because the reality is somewhere in the middle. It obviously isn't 100x or even 10x except for very specific toy tasks, but for me it's probably around ~1.5-2x in everyday work. And I work on mostly long-tail scientific stuff, I imagine it's more if you do something like frontend dev with a lot of boilerplate. I'm absolutely convinced that people who say LLMs are not making them more productive either don't un…

It's really hard to measure. Sure, maybe 20-30% of the time I get a perfect autocomplete and boom, 2x speedup. Then there is the partially correct or completely incorrect suggestions that tools like Cursor keep spamming, making me either accept by mistake or continuously rage hit the escape key, which both slows things down and breaks the thinking flow.

How do you account for the latter? It's nearly impossible and I have seen no improvement in that realm. As the good suggestions have become better, the ratio of good to bad hasn't really changed for me.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#265
post #183

Earlier quoted context omitted.

I'm learning math through Math Academy, ChatGPT and YouTube (mostly 3Blue1Brown and similar channels). Without that specifc combination, it would've been hell. Now it's just nice. A few years I did it with something similar to Math Academy (from the Universiteit of Amsterdam). ChatGPT wasn't intelligent enough back then so I didn't use it. It felt a lot tougher. ChatGPT answers questions that a teacher would find obs…

> why is the letter m used when talked about an angle? This isn’t the best example, because a simple Google search gives you immediate results for that as well, and arguably more nuanced ones. You’ll learn, for example, that m is also the standard letter for slope, which may or may not be related. Also, if a teacher finds that question “obscure nonsense”, I’d argue that they are a bad teacher.

Hmm, it's not the best example yea. It's just that ChatGPT understands math notation a lot better than Google. If needed, you can always let ChatGPT do a Bing search.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#266

Earlier quoted context omitted.

"No one is entitled to answers" feels very definitive, defeating and tiring. Especially because you don't explain your own thought process. Could you please assume a good faith discussion?

I have. I was criticized for pointing out spurious nonsense in LLM slop by someone who claimed people wouldn't have to resort to it if other people made an effort to explain things better. But I don't believe anyone is entitled to an explanation. I find things out by looking up books and testing things. Any explanation someone deigns to give me is a bonus and doubted until corroborated. I don't know why anyone would…

> I was criticized for pointing out spurious nonsense in LLM slop

I can see that you experience it as such but I think it's more of a spectrum. Often times, LLMs give good answers. Often enough times they don't. One needs to keep that in mind. In my example, given that it was just a symbol, all I needed was knowledge at the level of a memnonic which would, on average, at least somewhat also point directionally to the truth. But that's a bonus. I could make up a memnonic myself, but I like having that bonus.

Given that ChatGPT is directionally towards the truth, but not fully (on average), I'd need to test it or verify it if I want a better level of knowledge than that. If that's the case, then ChatGPT is basically acts as a sort of cache as it's quicker to ask a question to ChatGPT than to research on one's own. One can experience a cache hit or cache miss. Such a thing will happen in the verification stage. Specifically, for math this is quicker, in my experience.

But anyways that's my experience. Your experience is that it's spurious nonsense slop. And I suppose you therefore find it a problem. I don't see the issue as there are different levels of knowledge and different time commitments you need to give to them. A lot of my knowledge is based on trust anyway and sometimes it's broken (e.g. replication crisis in psychology, I felt betrayed having studied the field).

> I don't know why anyone would think they are owed a custom explanation for their specific questions

I'm not sure if anyone said anything like it. Regardless of that, the need still exists. People will still act on that need. I suspect you see that as a problem. I'm neutral on it.

> thinking like that will get you in trouble when you come to depend on what anyone (or anything) is willing to chew up for you.

IMO teaching and learning is a 2 way street. It's the teacher's job to explain it well enough. It's the student's job to do their best to understand it. Math Academy offers exercises and explanations. Sometimes I find their explanations a bit lacking. So I use other sources to augment it.

> Maybe I was terse but I don't think I was rude or illogical.

Reading/writing text is tough, which is why I stated how I felt. It'd probably have been easier in an actual conversation. I didn't mean to imply you were being rude or illogical.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#267

Earlier quoted context omitted.

>I expect things to get better, this will not always be the state of things, but for now “vibe coding” (specifically not reviewing/writing code yourself) is not sustainable. It will not. And I say this as someone whose been building internal LLM tools since 2021. The issue is their context window. If you increase the context window so they can see more code costs skyrocket as n^2 the size of the code base. If you don…

> building tooling for intelligent search that limits the overfill of each context window I'm interested to know what you mean by this, in our system we've been trying to compress the context but this is the first I've seen about filtering it down.

For general text you run some type of vector search against the full-text corpus to see what relevant hits there are and where. Then you feed the first round of results into a ranking/filtering system which does pair wise comparison between each chunk that you've had a good score from the vector search. Contract/expand until you've reach the limit of the context window for your model and run against the original query.

For source code, you are even luckier since there are a lot of deterministic tools which provide solid grounding, e.g., etags, and the languages themselves enforce a hierarchical tree-like structure on the source code, viz. block statements. The above means that ranking and chunking strategies are solved already - which is a huge pain for general text.

The vector search is then just an enrichment layer on top which brings in documentation and other soft grounding text that keeps the LLM from going berserk.

Of course, none of the commercial offerings come even close to letting you do this well. Even the dumb version of search needs to be a self-recursive agent which comes with a good set of vector embeddings and the ability to decide if it's searched enough before it starts answering your questions.

If you're interested drop a line on my profile email.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#268

Earlier quoted context omitted.

> I just rewrote 300ish advanced PostgreSQL queries to mysql queries. Translation from one set of tokens to another is exactly the primary use case of LLMs. This is exactly what it should be good at. Developing new queries, much less so. Translation from one set of database queries to another was already very well defined and well covered before LLMs came about.

I mean, using a good ORM with DB adapter options could achieve this in minutes. Sure, LLM has utility here for raw queries, hardly "replace SWEs" type of utility though

A good reason to have "300ish advanced PostgreSQL queries" is because an "ORM with DB adapter" is inadequate for your requirements.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#270
post #169
post #60

Earlier quoted context omitted.

Go look at the latest YC batch. 30% of them were built with 90% of the code being written by LLMs.

Yeah, let's all pull numbers out of a hat and see who wins. Learn some real skills or prepare to not have a job.

https://techcrunch.com/2025/03/06/a-quarter-of-startups-in-y...

What real skill do you have exactly

Post reply on HN