Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Human coders are still better than LLMs

antirez.com

Post reply on HN
Print view View original

581–590 of 762 posts

Re: Human coders are still better than LLMs

#581

Post by 0points »

insane_dreamer wrote: post #527

Coders may want to look at translators for an idea of what might happen. Translation software has been around for a couple of decades. It was pretty shitty. But about 10 years ago it started to get to the point where it could translate relatively accurately. However, it couldn't produce text that sounded like it was written by a human. A good translator (and there are plenty of bad ones) could easy outperform a machi…

> When LLMs came out, and they now produce text that sounded like it was written by a native speaker (in major languages).

While the general syntax of the language seem to be somewhat correct now, the LLM's still don't know anything about those languages and keep mis-translating words due to its inherit insane design around english. A whole lot of concepts don't even exist in english so these translation oracles just can never do it successfully.

If i i read a few minutes of LLM translated text, there's always a couple of such errors.

I notice younger people don't see these errors because of their worse language skills, and the LLM:s enforce their incorrect understanding.

I don't think this problem will go away as long as we keep pushing this inferior tech, but instead the languages will devolve to "fix" it.

Languages will morph into a 1-to-1 mapping of english and all the cultural nuances will get lost to time.

Re: Human coders are still better than LLMs

#582

Post by pwdisswordfishz »

seattle_spring wrote: post #402
Buttons840 wrote: post #379

Earlier quoted context omitted.

LLMs aren't my rubber duck, they're my wrong answer. You know that saying that the best way to get an answer online is to post a wrong answer? That's what LLMs do for me. I ask the LLM to do something simple but tedious, and then it does it spectacularly wrong, then I get pissed off enough that I have the rage-induced energy to do it myself.

This has been my experience as well. The biggest problem is that the answers look plausible, and only after implementation and experimentation do you find them to be wrong. If this happened every once in a while then it wouldn't be a big deal, but I'd guess that more than half of the answers and tutorials I've received through ChatGPT have ended up being plain wrong. God help us if companies start relying on LLMs for…

> If this happened every once in a while then it wouldn't be a big deal, but I'd guess that more than half of the answers and tutorials I've received through ChatGPT have ended up being plain wrong.

It would actually have been more pernicious that way, since it would lull people into a false sense of security.

Re: Human coders are still better than LLMs

#583

Post by lazyasciiart »

codr7 wrote: post #492
wvenable wrote: post #461

Earlier quoted context omitted.

I almost never bother using Google anymore. When I search for something, I'm usually looking for an answer to question. Now I can just ask the question and get the answer without all the other stuff. I will often ask the LLM to give me web pages to look at it when I want to do further reading. As LLMs get better, I can't see myself going back to Google as it is or even as it was.

You get an answer. If that's the answer, or even the best answer, is impossible to tell without doing the research you're trying to avoid.

By that logic, it's barely worth reading a newspaper or a book. You don't know if they're giving you accurate information without doing all the research you're trying to avoid.

Re: Human coders are still better than LLMs

#585

Post by otabdeveloper4 »

AdieuToLogic wrote: post #457
bsder wrote: post #393

Earlier quoted context omitted.

LLMs are a decent search engine a la Google circa 2005. It's been 20 years since that, so I think people have simply forgotten that a search engine can actually be useful as opposed to ad infested SEO sewage sludge. The problem is that the conversational interface, for some reason, seems to turn off the natural skepticism that people have when they use a search engine.

> LLMs are a decent search engine a la Google circa 2005. Statistical text (token) generation made from an unknown (to the user) training data set is not the same as a keyword/faceted search of arbitrary content acquired from web crawlers. > The problem is that the conversational interface, for some reason, seems to turn off the natural skepticism that people have when they use a search engine. For me, my skepticism…

> Statistical text (token) generation made from an unknown (to the user) training data set is not the same as a keyword/faceted search of arbitrary content acquired from web crawlers.

Well, it's roughly the same under the hood, mathematically.

Re: Human coders are still better than LLMs

#586

Post by carlmr »

Buttons840 wrote: post #388
Buttons840 wrote: post #379

Earlier quoted context omitted.

LLMs aren't my rubber duck, they're my wrong answer. You know that saying that the best way to get an answer online is to post a wrong answer? That's what LLMs do for me. I ask the LLM to do something simple but tedious, and then it does it spectacularly wrong, then I get pissed off enough that I have the rage-induced energy to do it myself.

I'm probably suffering undiagnosed ADHD, and will get stuck and spend minutes picking a function name and then writing a docstring. LLMs do help with this even if they get the code wrong, because I usually won't bother to fix their variables names or docstring unless needed. LLMs can reliably solve the problem of a blank-page.

So much this, the blank page problem is almost gone. Even if it's riddled with errors.

Re: Human coders are still better than LLMs

#587

Post by magicalhippo »

pixl97 wrote: post #494
AdieuToLogic wrote: post #457

Earlier quoted context omitted.

> LLMs are a decent search engine a la Google circa 2005. Statistical text (token) generation made from an unknown (to the user) training data set is not the same as a keyword/faceted search of arbitrary content acquired from web crawlers. > The problem is that the conversational interface, for some reason, seems to turn off the natural skepticism that people have when they use a search engine. For me, my skepticism…

Search engines can be really good still if you have a good idea what you're looking for in the domain you're searching. Search engines can suck when you don't know exactly what you're looking for and the phrases you're using have invited spammers to fill up the first 10 pages.

They also suck if you want to find something that's almost exactly like a very common thing, but different in some key aspect.

For example, I wanted to find some texts on solving a partial differential equation numerically using 6th-order or higher finite differences, as I wanted to know how to handle boundry conditions (interior is simple enough).

Searching only turned up the usual low-order methods that I already knew.

Asking some LLMs I got some decent answer and could proceed.

Back in the day you could force the search engines to restrict their search scope, but they all seem so eager to return results at all cost these days, making them useless in niche topics.

Re: Human coders are still better than LLMs

#588

Post by akoboldfrying »

materiallie wrote: post #552
Buttons840 wrote: post #379

Earlier quoted context omitted.

LLMs aren't my rubber duck, they're my wrong answer. You know that saying that the best way to get an answer online is to post a wrong answer? That's what LLMs do for me. I ask the LLM to do something simple but tedious, and then it does it spectacularly wrong, then I get pissed off enough that I have the rage-induced energy to do it myself.

This is my experience, too. As a concrete example, I'll need to write a mapper function to convert between a protobuf type and Go type. The types are mirror reflections of each other, and I feed the complete APIs of both in my prompt. I've yet to find an LLM that can reliability generate mapping code between proto.Foo{ID string} to gomodel.Foo{ID string}. It still saves me time, because even 50% accuracy is still hal…

This honestly seems like something that could be better handled with pre-LLM technology, like a 15-line Perl script that reads one on stdin, applies some crufty regexes, and writes the other to stdout. Are there complexities I'm not seeing?

Re: Human coders are still better than LLMs

#589

Post by foobar83 »

sixQuarks wrote: post #380
foobar83 wrote: post #183

Earlier quoted context omitted.

Nobody knows what the future holds, including you.

Yes, but we can see current progress and extrapolate into the future. I give it 2/3 years before AI can code as well as a senior level coder

Recent benchmarks show that improvements in the latest models are beginning to slow down, what makes you so sure there's another breakthrough coming?

Re: Human coders are still better than LLMs

#590

Post by geros »

xhevahir wrote: post #184
UncleOxidant wrote: post #71

There's some whistling past the graveyard in these comments. "You still need humans for the social element...", "LLMs are bad at debugging", "LLMs lead you astray". And yeah, there's lots of truth in those assertions, but since I started playing with LLMs to generate code a couple of years ago they've made huge strides. I suspect that over the next couple of years the improvements won't be quite as large (Pareto Prin…

This attitude is depressingly common in lots of professional, white-collar industries I'm afraid. I just came from the /r/law subreddit and was amazed at the kneejerk dismissal there of Dario Amodei's recent comments about legal work, and of those commenters who took them seriously. It's probably as much a coping mechanism as it is complacency, but, either way, it bodes very poorly for our future efforts at mitigatin…

"It is difficult to get a man to understand something when his salary depends upon his not understanding it." - Upton Sinclair
Post reply on HN