Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Human coders are still better than LLMs

antirez.com

Post reply on HN
Print view View original

551–560 of 762 posts

Re: Human coders are still better than LLMs

#551

Post by Jolter »

mullingitover wrote: post #508
Jolter wrote: post #132

Earlier quoted context omitted.

Here’s the deal: if you won’t write your replacement, a competitor will do it and outprice your employer. Either way you’re out of a job. May be more prudent to adapt to the new tools and master them rather than be left behind? Do you want to be a jobless weaver, or an engineer building mechanical looms for a higher pay than the weaver got?

> Either way you’re out of a job. Tools and systems which increase productivity famously always put everyone out of a job, which is why after a couple centuries of industrial revolution we're all unemployed.

This is kind of my point; people tend to not be silly enough to stay unemployed and starve. Instead when push comes to shove, sensible folks will adapt to the circumstances.

Re: Human coders are still better than LLMs

#552

Post by materiallie »

Buttons840 wrote: post #379
mattnewton wrote: post #5

This matches my experience. I actually think a fair amount of value from LLM assistants to me is having a reasonably intelligent rubber duck to talk to. Now the duck can occasionally disagree and sometimes even refine. https://en.m.wikipedia.org/wiki/Rubber_duck_debugging I think the big question everyone wants to skip right to and past this conversation is, will this continue to be true 2 years from now? I don’t kno…

LLMs aren't my rubber duck, they're my wrong answer. You know that saying that the best way to get an answer online is to post a wrong answer? That's what LLMs do for me. I ask the LLM to do something simple but tedious, and then it does it spectacularly wrong, then I get pissed off enough that I have the rage-induced energy to do it myself.

This is my experience, too. As a concrete example, I'll need to write a mapper function to convert between a protobuf type and Go type. The types are mirror reflections of each other, and I feed the complete APIs of both in my prompt.

I've yet to find an LLM that can reliability generate mapping code between proto.Foo{ID string} to gomodel.Foo{ID string}.

It still saves me time, because even 50% accuracy is still half that I don't have to write myself.

But it makes me feel like I'm taking crazy pills whenever I read about AI hype. I'm open to the idea that I'm prompting wrong, need a better workflow, etc. But I'm not a luddite, I've "reached up and put in the work" and am always trying to learn new tools.

Re: Human coders are still better than LLMs

#553

Post by solatic »

Human coders are necessary because writing code is a political act of deciding between different trade-offs. antirez's whole post is explaining to Gemini what the trade-offs even were in the first place. No analysis of a codebase in isolation (i.e. without talking to the original coders, and without comments in the code) can distinguish between intentional prioritization of certain trade-offs or whether behavior is unintentional / written by a human in an imperfect way because they didn't know any better / buggy.

LLMs will never be able to figure out for themselves what your project's politics are and what trade-offs are supposed to be made. The penultimate model will still require a user to explain the trade-offs in a prompt.

Re: Human coders are still better than LLMs

#554

Post by oconnor663 »

ch4s3 wrote: post #151
UncleOxidant wrote: post #71

There's some whistling past the graveyard in these comments. "You still need humans for the social element...", "LLMs are bad at debugging", "LLMs lead you astray". And yeah, there's lots of truth in those assertions, but since I started playing with LLMs to generate code a couple of years ago they've made huge strides. I suspect that over the next couple of years the improvements won't be quite as large (Pareto Prin…

It seems like LLMs made really big strides for a while but don't seem to be getting better recently, and in some ways recent models feel a bit worse. I'm seeing some good results generating test code, and some really bad results when people go to far with LLM use on new feature work. Base on what I've seen it seems like spinning up new projects and very basic features for web apps works really well, but that doesn't…

> don't seem to be getting better recently

o3 came out just one month ago. Have you been using it? Subjectively, the gap between o3 and everything before it feels like the biggest gap I've seen since ChatGPT originally came out.

Re: Human coders are still better than LLMs

#555

Post by oconnor663 »

parliament32 wrote: post #281
UncleOxidant wrote: post #71

There's some whistling past the graveyard in these comments. "You still need humans for the social element...", "LLMs are bad at debugging", "LLMs lead you astray". And yeah, there's lots of truth in those assertions, but since I started playing with LLMs to generate code a couple of years ago they've made huge strides. I suspect that over the next couple of years the improvements won't be quite as large (Pareto Prin…

I'd like to agree with you and remain optimistic, but so much tech has promised the moon and stagnated into oblivion that I just don't have any optimism left to give. I don't know if you're old enough, but remember when speech-to-text was the next big thing? DragonSpeak was released in 1997, everyone was losing their minds about dictating letters/documents in MS Word, and we were promised that THIS would be the key i…

> 27 years later, talking to the latest Siri, it makes just as many mistakes as it did back then

Have you tried talking to ChatGPT voice mode? It's mind blowing. You just have a conversation with it. In any language. About anything. The other day I wanted to know about the difference between cast iron and wrought iron, and it turned into a 10 or 15 minute conversation. That's maybe a good example of an "easy" topic for LLMs (lots of textbooks for it to memorize), but the world is full of easy topics that I know nothing about!

Re: Human coders are still better than LLMs

#556

Post by materiallie »

drodgers wrote: post #474
wijwp wrote: post #333

Earlier quoted context omitted.

> Not to sound like a tin-foil hat but odds of achieving something like this increase by the day. Where do you get this? The limitations of LLMs are becoming more clear by the day. Improvements are slowing down. Major improvements come from integrations, not major model improvements. AGI likely can't be achieved with LLMs. That wasn't as clear a couple years ago.

I don't know how someone could be following the technical progress in detail and hold this view. The progress is astonishing, and the benchmarks are becoming saturated so fast that it's hard to keep track. Are there plenty of gaps left between here and most definitions of AGI? Absolutely. Nevertheless, how can you be sure that those gaps will remain given how many faculties these models have already been able to exce…

I think benchmark targeting is going to be a serious problem going forward. The recent Nate Silver podcast on poker performance is interesting. Basically, the LLM models still suck at playing poker.

Poker tests intelligence. So what gives? One interesting thing is that for whatever reason, poker performance isn't used a benchmark in the LLM showdown between big tech companies.

The models have definitely improved in the past few years. I'm skeptical that there's been a "break-through", and I'm growing more skeptical of the exponential growth theory. It looks to me like the big tech companies are just throwing huge compute and engineering budgets at the existing transformer tech, to improve benchmarks one by one.

I'm sure if Google allocated 10 engineers a dozen million dollars to improve Gemini's poker performance, it would increase. The idea before AGI and the exponential growth hypothesis is that you don't have to do that because the AI gets smarter in a general sense all on it's own.

Re: Human coders are still better than LLMs

#557

Post by energy123 »

solatic wrote: post #553

Human coders are necessary because writing code is a political act of deciding between different trade-offs. antirez's whole post is explaining to Gemini what the trade-offs even were in the first place. No analysis of a codebase in isolation (i.e. without talking to the original coders, and without comments in the code) can distinguish between intentional prioritization of certain trade-offs or whether behavior is u…

> LLMs will never be able to figure out for themselves what your project's politics are and what trade-offs are supposed to be made.

I wouldn't declare that unsolvable. The intentions of a project and how they fit into user needs can be largely inferred from the code and associated docs/README, combined with good world knowledge. If you're shown a codebase of a GPU kernel for ML, then as a human you instantly know the kinds of constraints and objectives that go into any decisions. I see no reason why an LLM couldn't also infer the same kind of meta-knowledge. Of course, this papers over the hard part of training the LLMs to actually do that properly, but I don't see why it's inherently impossible.

Re: Human coders are still better than LLMs

#558

Post by travisgriggs »

mattnewton wrote: post #5

This matches my experience. I actually think a fair amount of value from LLM assistants to me is having a reasonably intelligent rubber duck to talk to. Now the duck can occasionally disagree and sometimes even refine. https://en.m.wikipedia.org/wiki/Rubber_duck_debugging I think the big question everyone wants to skip right to and past this conversation is, will this continue to be true 2 years from now? I don’t kno…

LLMs are a passel of eager to please know it all interns that you can command at will without any moral compunctions.

They drive you nuts trying to communicate with them what you actually want them to do. They have a vast array of facts at immediate recall. They’ll err in their need to produce and please. They do the dumbest things sometimes. And surprise you at other times. You’ll throw vast amounts of their work away or have to fix it. They’re (relatively) cheap. So as an army of monkeys, if you keep herding them, you can get some code that actually tells a story. Mostly.

Re: Human coders are still better than LLMs

#559

Post by shrewduser »

lillecarl wrote: post #397
daveguy wrote: post #353

Earlier quoted context omitted.

> the raw code quality that LLMs generate is usually top notch, even if the functionality is completely off. I'm not even sure what this is supposed to mean. It doesn't make syntax errors? Code that doesn't have the correct functionality is obviously not "top notch".

No syntax errors, good error handling and such. Just because it implemented the wrong function doesn't mean the function is bad.

i wish i could do that in an interview.
Post reply on HN