Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Human coders are still better than LLMs

antirez.com

Post reply on HN
Print view View original

681–690 of 762 posts

Re: Human coders are still better than LLMs

#681

Post by JoshCole »

gitaarik wrote: post #532
JoshCole wrote: post #345

Earlier quoted context omitted.

> it can't "think" of something entirety conceptionally new, because it doesn't really "think". Hierarchical optimization (fast global + slow local) is a precise, implementable notion of "thinking." Whenever I've seen this pattern implemented, humans, without being told to do so by others in some forced way, seem to converge on the use of verb think to describe the operation. I think you need to blacklist the term th…

Yes ok, it can generate new stuff, but it's dependent on human curated reward models to score the output to make it usable. So it still depends on human thinking, it's own "thinking" is not sufficient. And there won't be a point when human curated reward models are not needed anymore. LLM's will make a lot of things easier for humans, because most of the thinking the humans do have been automated into the LLM. But ul…

> dependent on human curated reward models to score the output to make it usable.

This is a false premise, because there already exist systems, currently deployed, which are not dependent on human-curated reward models.

Refutations of your point include existing systems which generate a reward model based on some learned AI scoring function, allowing self-bootstrapping toward higher and higher levels.

A different refutation of your point is the existing simulation contexts, for example, by R1, in which coding compilation is used as a reward signal; here the reward model comes from a simulator, not a human.

> So it still depends on human thinking

Since your premise was false your corollary does not follow from it.

> And there won't be a point when human curated reward models are not needed anymore.

This is just a repetition of your previously false statement, not a new one. You're probably becoming increasingly overconfident by restating falsehoods in different words, potentially giving the impression you've made a more substantive argument than you really have.

Re: Human coders are still better than LLMs

#682

Post by v3xro »

garciasn wrote: post #223
xhevahir wrote: post #184

Earlier quoted context omitted.

This attitude is depressingly common in lots of professional, white-collar industries I'm afraid. I just came from the /r/law subreddit and was amazed at the kneejerk dismissal there of Dario Amodei's recent comments about legal work, and of those commenters who took them seriously. It's probably as much a coping mechanism as it is complacency, but, either way, it bodes very poorly for our future efforts at mitigatin…

This is the response to most new technologies; folks simply don't want to accept the future before the ramifications truly hit. If technology folk cannot see the INCREDIBLE LEAP FORWARD made by LLMs since ChatGPT came on the market, they're not seeing the forest through the trees because their heads are buried in the sand. LLMs for coding are not even close to imperfect, yet, but the saturation curves are not flatten…

Many of us would prefer to see the technological leaps to be evenly distributed (so e.g. even clean drinking water that does not need to be boiled before consumption is not a baseline in 2025). So if you want to adapt to your new and improved position where you are just pushing buttons fine - but some of us are actually interested in how computers work (and are actually really uninterested in most companies' bottom lines). It's just how it is ;)

Re: Human coders are still better than LLMs

#683

Post by klabb3 »

energy123 wrote: post #550
klabb3 wrote: post #434

Earlier quoted context omitted.

Pareto is about diminishing returns. > but each succeeding iteration seems to be more disappointing This is because the scaling hypothesis (more data and more compute = gains) is plateauing, because all text data is used and compute is reaching diminishing returns for some reason I’m not smart enough to say why, but it is. So now we're seeing incremental core model advancements, variations and tuning in pre- and post…

We're seeing diminishing returns in benchmark space , which is partly an artefact of construction, not an absolutely true commentary on how things are progressing.

Well yes but there is no better way to measure without resorting to pure hearsay. How would you make an accurate assessment of something so inherently vague?

Re: Human coders are still better than LLMs

#684

Post by ch4s3 »

oconnor663 wrote: post #554
ch4s3 wrote: post #151

Earlier quoted context omitted.

It seems like LLMs made really big strides for a while but don't seem to be getting better recently, and in some ways recent models feel a bit worse. I'm seeing some good results generating test code, and some really bad results when people go to far with LLM use on new feature work. Base on what I've seen it seems like spinning up new projects and very basic features for web apps works really well, but that doesn't…

> don't seem to be getting better recently o3 came out just one month ago. Have you been using it? Subjectively, the gap between o3 and everything before it feels like the biggest gap I've seen since ChatGPT originally came out.

I haven't used it extensively, but toyed around with it for Elixir code and I wasn't particularly impressed.

Re: Human coders are still better than LLMs

#685

Post by Sohcahtoa82 »

AnimalMuppet wrote: post #41
zdragnar wrote: post #13

Earlier quoted context omitted.

I think that's great if it's for something outside of your primary language. I've used it to good effect in that way myself. However, denying yourself the reflexive memory of having learned those things is a quick way to become wholly dependent upon the tool. You could easily end up with compromised solutions because the tool recommends something you don't understand well enough to know there's a better way to do som…

So here's an analogy. (Yeah, I know, proof by analogy is fraud. But it's going to illustrate the question.) Here's a kid out hoeing rows for corn. He sees someone planting with a tractor, and decides that's the way to go. Someone tells him, "If you get a tractor, you'll never develop the muscles that would make you really great at hoeing." Different analogy: Here's someone trying to learn to paint. They see someone p…

The analogy I would use is that coding via LLM is like learning to drive in a self-driving car that has manual controls as an option that drives overly cautiously (Leaves excessively large following distances, takes corners slower, etc.) while in self-driving mode.

You can let it self-drive, but you'd probably learn nothing, and it will actually take you longer. Put an expert driver behind the wheel, and they'll drive faster and only use automation features for the boring parts.

Re: Human coders are still better than LLMs

#686

Post by wvenable »

addaon wrote: post #674
wvenable wrote: post #465

Earlier quoted context omitted.

I don't know how someone can legitimately say that they're useless. Perfect, no. But useless, also no.

> I don't know how someone can legitimately say that they're useless. Clearly, statistical models trained on this HN thread would output that sequence of tokens with high probability. Are you suggesting that a statement being probable in a text corpus is not a legitimate source of truth? Can you generalize that a little bit?

Who said anything about truth? We're talking about usefulness.

Re: Human coders are still better than LLMs

#687

Post by km144 »

AstroBen wrote: post #205
devjab wrote: post #101

Earlier quoted context omitted.

LLM's do no software engineering at all, and that can be fine. Because you don't actually need software engineering to create successful programs. Some applications will not even need software engineering for their entire life cycles because nobody is really paying attention to efficiency in the ocean of poor cloud management anyway. I actually imagine it's the opposite of what you say here. I think technically incli…

That's the thing too right.. the vast majority of software out there barely needs to scale or be super efficient It does need to be reliable, though. LLMs have proven very bad at that

Maybe automated testing of all forms will just become much more ubiquitous as a safeguard against the worst of AI hallucinations? I feel that would solve a lot of people's worries about LLMs. I'm imagining a world where a software developer is a person who gathers requirements, writes some tests, asks the AI to modify the codebase, ensures the tests still work, makes sure they are a human who understands the change the AI just made, and continues with the next requirement.

Re: Human coders are still better than LLMs

#688

Post by JoshCole »

gitaarik wrote: post #561
gitaarik wrote: post #532

Earlier quoted context omitted.

Yes ok, it can generate new stuff, but it's dependent on human curated reward models to score the output to make it usable. So it still depends on human thinking, it's own "thinking" is not sufficient. And there won't be a point when human curated reward models are not needed anymore. LLM's will make a lot of things easier for humans, because most of the thinking the humans do have been automated into the LLM. But ul…

So to clarify, it could potentially come up with (something close to) C, but if you want it to get to D, E, F etc, it will become less and less accurate for each consequentive step, because it lacks the human curated reward models up to that point. Only if you create new reward models for C, the output for D will improve, and so on.

> Only if you create new reward models for C, the output for D will improve, and so on.

Again, tons of false claims. One is that 'you' have to create the reward model. Another that it has to be human-curated at all. Yet another is that you even need to do that at all: you can instead have the model build a bigger model of itself, train using its existing resources or more of them, then synthesize itself back down. Another way you can get around it is to augment the existing dataset in some way. No other changes except resource usage and yet the resulting model will be better, because more resources went into its construction.

Seriously notice: you keep making false claims again and again and again and again and again. You're not stating true things. You really need to reflect. If almost every sentence you speak on this topic is false, why is it that you think you should be able to persuade me to your views? Why should I believe your views, when you say so many things that are factually inaccurate, rather than my own views?

Re: Human coders are still better than LLMs

#689

Post by JoshCole »

gitaarik wrote: post #680
vidarh wrote: post #647

Earlier quoted context omitted.

> But reward models are always curated by humans. There is no inherent reason why they need to be. > So yeah theoretically you could generate reward models with LLMs, but they won't be any good, unless they are curated by other reward models that are ultimately curated by humans. This reasoning is begging the question: The reasoning is true only if the conclusion is true. It's therefore a logically invalid argument.…

Sorry but I don't follow your logic. Are you claiming that reward models that aren't curated by humans perform as well as ones that are? Then what is a reward model's function according to you?

In reinforcement learning and related fields, a _reward model_ is a function that assigns a scalar value (a reward) to a given state, representing how desirable it is. You're at liberty to have compound states: for an example, a trajectory (often called tau) or a state action pair (typically represented by s and a).

Re: Human coders are still better than LLMs

#690

Post by twodave »

sumeno wrote: post #386
twodave wrote: post #366

Earlier quoted context omitted.

I would be much more in favor of them as a software developer if I felt the qualifications of other software developers were reliable. But they never have been, and today it’s worse than ever. So I think it’s important to weed some of that out before we start talking about making it harder to fire people.

You're doing your boss's job for them by making this an engineer vs engineer thing instead of an engineer vs management thing.

No, today I can tell my manager when someone sucks at their job, and if enough other people are saying the same thing then that engineer will either be given help so they stop sucking or be fired so they don’t subtract from the team. Because this happens fairly often I wouldn’t want to add a lot of friction to that process. Idk, maybe you can protect people from being laid off without also complicating performance-based firings.
Post reply on HN