Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Human coders are still better than LLMs

antirez.com

Post reply on HN
Print view View original

221–230 of 762 posts

Re: Human coders are still better than LLMs

#221

Post by bluefirebrand »

californical wrote: post #209
bluefirebrand wrote: post #146

Earlier quoted context omitted.

> If you remove the context of LLMs from the discussion, it reads like you're arguing that technological progress in general is bad because people would eventually struggle to live without it. I'm arguing that there are always tradeoffs and we often do not fully understand the tradeoffs we are making or the consequences of those tradeoffs 10, 50, 100 years down the road When we moved from more physical jobs to desk j…

I do think you’re missing something, though. In a healthy competitive market (like most of the history of the US, maybe not the last 30-40 years), if all of the farms do that, it drives down the cost of the food. The reduction in labor necessary to produce the food causes competition and brings down the cost to produce the food. That still doesn’t directly benefit the farmhands. But if it happens gradually throughout…

> I think creating an environment that allows many more medium-sized companies and allowing them to compete heavily will ease away any risk of job loss. Same applies to a bunch of fields other than tech. The US companies are far too consolidated

How do we achieve this environment?

It's not through AI, that is still the same problem. The AI companies will be the 6-10 mega companies and anyone relying on AI will still be small fry

Every time in my lifetime that we have had a huge jump in technological progress, all we've seen is that the rich get richer and the poor get poorer and the gap gets bigger

You even call this out explicitly: "most of the history of the US, maybe not the last 30-40 years"

Do we have any realistic reason to assume the trend of the last 30-40 years will change course at this point?

Re: Human coders are still better than LLMs

#222

Post by 12_throw_away »

platevoltage wrote: post #86
_tom_ wrote: post #51

Earlier quoted context omitted.

For me, it's a bit like pair programming. I have someone to discuss ideas with. Someone to review my code and suggest alternative approaches. Some one that uses different feature than I do, so I learn from them.

This is how I use it too. It's great at quickly answering questions. I find it particularly useful if I have to work with a language of framework that I'm not fully experienced in.

> I find it particularly useful if I have to work with a language of framework that I'm not fully experienced in

Yep - my number 1 use case for LLMs is as a template and example generator. It actually seems like a fairly reasonable use for probabilistic text generation!

Re: Human coders are still better than LLMs

#223

Post by garciasn »

xhevahir wrote: post #184
UncleOxidant wrote: post #71

There's some whistling past the graveyard in these comments. "You still need humans for the social element...", "LLMs are bad at debugging", "LLMs lead you astray". And yeah, there's lots of truth in those assertions, but since I started playing with LLMs to generate code a couple of years ago they've made huge strides. I suspect that over the next couple of years the improvements won't be quite as large (Pareto Prin…

This attitude is depressingly common in lots of professional, white-collar industries I'm afraid. I just came from the /r/law subreddit and was amazed at the kneejerk dismissal there of Dario Amodei's recent comments about legal work, and of those commenters who took them seriously. It's probably as much a coping mechanism as it is complacency, but, either way, it bodes very poorly for our future efforts at mitigatin…

This is the response to most new technologies; folks simply don't want to accept the future before the ramifications truly hit. If technology folk cannot see the INCREDIBLE LEAP FORWARD made by LLMs since ChatGPT came on the market, they're not seeing the forest through the trees because their heads are buried in the sand.

LLMs for coding are not even close to imperfect, yet, but the saturation curves are not flattening out; not by a long shot. We are living in a moment and we need to come to terms with it as the work continues to develop; and, we need to adapt and quickly in order to better understand what our place will become as this nascent tech continues its meteoric trajectory toward an entirely new world.

Re: Human coders are still better than LLMs

#224

Post by AstroBen »

am17an wrote: post #67

All the world's smartest minds are racing towards replacing themselves. As programmers, we should take note and see where the wind is blowing. At least don't discard the possibility and rather be prepared for the future. Not to sound like a tin-foil hat but odds of achieving something like this increase by the day. In the long term (post AGI), the only safe white-collar jobs would be those built on data which is not…

Ultimately this needs to be solved politically

Making our work more efficient, or humans redundant should be really exciting. It's not set in stone that we need to leave people middle aged with families and now completely unable to earn enough to provide a good life

Hopefully if it happens, it happens to such a huge amount of people that it forces a change

Re: Human coders are still better than LLMs

#225

Post by palmotea »

jajko wrote: post #216
Bukhmanizer wrote: post #97

Earlier quoted context omitted.

There are a couple people I work with who clearly don’t have a good understanding of software engineering. They aren’t bad to work with and are in fact great at collaborating and documenting their work, but don’t seem to have the ability to really trace through code and logically understand how it works. Before LLMs it was mostly fine because they just didn’t do that kind of work. But now it’s like a very subtle chao…

Sounds almost like you definitely shouldnt use llms nor those juniors for such an important work. Is it just me or are we heading into a period of explosion of software done, but also a massive drop of its quality? Not uniformly, just a bit of chaotic spread

> Is it just me or are we heading into a period of explosion of software done, but also a massive drop of its quality? Not uniformly, just a bit of chaotic spread

I think we are, especially with executives mandating the use LLMs use and expecting it to massively reduce costs and increase output.

For the most part they don't actually seem to care that much about software quality, and tend to push to decrease quality at every opportunity.

Re: Human coders are still better than LLMs

#226

Post by coffeeismydrug »

writeslowly wrote: post #49
decasia wrote: post #10

We aren't expecting LLMs to come up with incredibly creative software designs right now, we are expecting them to execute conventional best practices based on common patterns. So it makes sense to me that it would not excel at the task that it was given here. The whole thing seems like a pretty good example of collaboration between human and LLM tools.

I haven't actually had that much luck with having them output a boring API boilerplate in large Java projects. Like "I need to create a new BarOperation that has to go in a different set of classes and files and API prefixes than all the FooOperations and I don't feel like copy pasting all the yaml and Java classes" but the AI has problems following this. Maybe they work better in small projects. I actually like LLMs…

> Like "I need to create a new BarOperation that has to go in a different set of classes and files and API prefixes than all the FooOperations and I don't feel like copy pasting all the yaml and Java classes" but the AI has problems following this.

To be fair, I also have problems following this.

Re: Human coders are still better than LLMs

#227

Post by dinfinity »

empath75 wrote: post #200
ch4s3 wrote: post #151

Earlier quoted context omitted.

It seems like LLMs made really big strides for a while but don't seem to be getting better recently, and in some ways recent models feel a bit worse. I'm seeing some good results generating test code, and some really bad results when people go to far with LLM use on new feature work. Base on what I've seen it seems like spinning up new projects and very basic features for web apps works really well, but that doesn't…

the LLM's themselves are making marginal gains, but the tools for using LLMs productively are getting so much better.

This. MCP/tool usage in agentic mode is insanely powerful. Let the agent ingest a Gitlab issue, tell it how it can run commands, tests etc. in the local environment and half of the time it can just iterate towards a solution all by itself (but watching and intervening when it starts going the wrong way is still advisable).

Recently I converted all the (Google Docs) documentation of a project to markdown files and added those to the workspace. It now indexes it with RAG and can easily find relevant bits of documentation, especially in agent mode.

It really stresses the importance of getting your documentation and processes in order as well as making sure the tasks at hand are well-specified. It soon might be the main thing that requires human input or action.

Re: Human coders are still better than LLMs

#228

Post by 1718627440 »

JoshCole wrote: post #195
marcosdumay wrote: post #65

Earlier quoted context omitted.

Why state the same arguments everybody has been repeating for ages? LLMs can only give you code that somebody has wrote before. This is inherent. This is useful for a bunch of stuff, but that bunch won't change if OpenAI decides to spend the GDP of Germany training one instead of Costa Rica.

> LLMs can only give you code that somebody has wrote before. This premise is false. It is fundamentally equivalent to the claim that a language model being trained on a dataset: ["ABA", "ABB"] would be unable to generate, given input "B" the string "BAB" or "BAA".

Isn't the claim, that it will never make up "C"?

Re: Human coders are still better than LLMs

#230

Post by JoshCole »

1718627440 wrote: post #228
JoshCole wrote: post #195

Earlier quoted context omitted.

> LLMs can only give you code that somebody has wrote before. This premise is false. It is fundamentally equivalent to the claim that a language model being trained on a dataset: ["ABA", "ABB"] would be unable to generate, given input "B" the string "BAB" or "BAA".

Isn't the claim, that it will never make up "C"?

They don't claim that. They say LLMs only generate text someone has written. Another way you could refute their premise was by showing the existence of AI-created programs for which someone isn't a valid description of the writer (e.g., from evolutionary algorithms) then training a network on that data such that it can output it. It is just as trivial a way to prove that the premise is false.

Your claim here is slightly different.

You're claiming that if a token isn't supported, it can't be output [1]. But we can easily disprove this by adding minimal support for all tokens, making C appear in theory. Such support addition shows up all the time in AI literature [2].

[1]: https://en.wikipedia.org/wiki/Support_(mathematics)

[2]: In some regimes, like game theoretic learning, support is baked into the solving algorithms explicitly during the learning stage. In others, like reinforcement learning, its accomplished by making the policy a function of two objectives, one an exploration objective, another an exploitation objective. That existing cross pollination already occurs between LLMs in the pre-trained unsupervised regime and LLMs in the post-training fine-tuning via forms of reinforcement learning regime should cause someone to hesitate to claim that such support addition is unreasonable if they are versed in ML literature.

Edit:

Got downvoted, so I figure maybe people don't understand. Here is the simple counterexample. Consider an evaluator that gives rewards: F("AAC") = 1, all other inputs = 0. Consider a tokenization that defines "A", "B", "C" as tokens, but a training dataset from which the letter C is excluded but the item "AAA" is present.

After training "AAA" exists in the output space of the language model, but "AAC" does not. Without support, without exploration, if you train the language model against the reinforcement learning reward model of F, you might get no ability to output "C", but with support, the sequence "AAC" can be generated and give a reward. Now actually do this. You get a new language model. Since "AAC" was rewarded, it is now a thing within the space of the LLM outputs. Yet it doesn't appear in the training dataset and there are many reward models F for which no person will ever have had to output the string "AAC" in order for the reward model to give a reward for it.

It follows that "C" can appear even though "C" does not appear in the training data.

Post reply on HN