Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Human coders are still better than LLMs

antirez.com

Post reply on HN
Print view View original

431–440 of 762 posts

Re: Human coders are still better than LLMs

#431

Post by all2 »

suddenlybananas wrote: post #296
all2 wrote: post #263

Earlier quoted context omitted.

My typical approach is prompt, be disgusted by the output, tinker a little on my own, prompt again -- but more specific, be disgusted again by the output, tinker a littler more, etc. Eventually I land on a solution to my problem that isn't disgusting and isn't AI slop. Having a sounding board, even a bad one, forces me to order my thinking and understand the problem space more deeply.

Why not just write the code at that point instead of cajoling an AI to do it.

I don't cajole the model to do it. I rarely use what the model generates. I typically do my own thing after making an assessment of what the model writes. I orient myself in the problem space with the model, then use my knowledge to write a more concise solution.

Re: Human coders are still better than LLMs

#432

Post by andrekandre »

bsder wrote: post #393
Buttons840 wrote: post #379

Earlier quoted context omitted.

LLMs aren't my rubber duck, they're my wrong answer. You know that saying that the best way to get an answer online is to post a wrong answer? That's what LLMs do for me. I ask the LLM to do something simple but tedious, and then it does it spectacularly wrong, then I get pissed off enough that I have the rage-induced energy to do it myself.

LLMs are a decent search engine a la Google circa 2005. It's been 20 years since that, so I think people have simply forgotten that a search engine can actually be useful as opposed to ad infested SEO sewage sludge. The problem is that the conversational interface, for some reason, seems to turn off the natural skepticism that people have when they use a search engine.

  > the conversational interface, for some reason, seems to turn off the natural skepticism that people have
n=1 but after having chatgpt "lie" to me more than once i am very skeptical of it and always double check it, whereas something like tv or yt videos i still find myself being click-baited or grifted (iow less skeptical) much more easily still... any large studies about this would be very interesting...

Re: Human coders are still better than LLMs

#433

Post by ChrisMarshallNY »

Really good coders (like him) are better.

Mediocre ones … maybe not so much.

When I worked for a Japanese optical company, we had a Japanese engineer, who was a whiz. I remember him coming over from Japan, and fixing some really hairy communication bus issues. He actually quit the company, a bit after that, at a very young age, and was hired back as a contractor; which was unheard of, in those days.

He was still working for them, as a remote contractor, at least 25 years later. He was always on the “tiger teams.”

He did awesome assembly. I remember when the PowerPC came out, and “Assembly Considered Harmful,” was the conventional wisdom, because of pipelining, out-of-order instructions, and precaching, and all that.

His assembly consistently blew the doors off anything the compiler did. Like, by orders of magnitude.

Re: Human coders are still better than LLMs

#434

Post by klabb3 »

retetr wrote: post #424
UncleOxidant wrote: post #71

There's some whistling past the graveyard in these comments. "You still need humans for the social element...", "LLMs are bad at debugging", "LLMs lead you astray". And yeah, there's lots of truth in those assertions, but since I started playing with LLMs to generate code a couple of years ago they've made huge strides. I suspect that over the next couple of years the improvements won't be quite as large (Pareto Prin…

Unrelated, but is this a case of the Pareto Principle? (Admittedly the first time I'm hearing of it) Wherein 80% of the effect is caused by 20% of the input. Or is this more a case of diminishing returns? Where the initial results were incredible, but each succeeding iteration seems to be more disappointing?

Pareto is about diminishing returns.

> but each succeeding iteration seems to be more disappointing

This is because the scaling hypothesis (more data and more compute = gains) is plateauing, because all text data is used and compute is reaching diminishing returns for some reason I’m not smart enough to say why, but it is.

So now we're seeing incremental core model advancements, variations and tuning in pre- and post training stages and a ton of applications (agents).

This is good imo. But obviously it’s not good for delusional valuations based exponential growth.

Re: Human coders are still better than LLMs

#436

Post by ben-schaaf »

AYBABTME wrote: post #400
ben-schaaf wrote: post #375

Earlier quoted context omitted.

Friendly reminder that people like you were saying the exact same thing about metaverse, VR, web3, crypto, etc.

The value of these was always a far fetch, and requires a critical mass adopting it before becoming potentially useful. But LLMs value is much more immediate and doesn't require any change in the rest of the world. If you use it and are amplified by it, you are... simply better off.

Frankly I disagree that LLMs value is immediate. What I do see is a whole lot of damage it's causing, just like the hype cycles before it. It's fine for us to disagree on this, but to say I'm burying my head in the sand not wanting to accept "the future" is exactly the same hype-driven bullshit the crypto crowd was pushing.

Re: Human coders are still better than LLMs

#437

Post by myvoiceismypass »

therealpygon wrote: post #399
Buttons840 wrote: post #379

Earlier quoted context omitted.

LLMs aren't my rubber duck, they're my wrong answer. You know that saying that the best way to get an answer online is to post a wrong answer? That's what LLMs do for me. I ask the LLM to do something simple but tedious, and then it does it spectacularly wrong, then I get pissed off enough that I have the rage-induced energy to do it myself.

LLMs follow instructions. Garbage in = garbage out generally. When attention is managed and a problem is well defined and necessary materials are available to it, they can perform rather well. On the other hand, I find a lot of the loosely-goosey vibe coding approach to be useless and gives a lot of false impressions about how useful LLMs can be, both too positive and too negative.

They should maybe have a verifiable specification for said instructions. Kinda like a programming language maybe!

Re: Human coders are still better than LLMs

#438

Post by taneq »

marcosdumay wrote: post #56
mattnewton wrote: post #5

This matches my experience. I actually think a fair amount of value from LLM assistants to me is having a reasonably intelligent rubber duck to talk to. Now the duck can occasionally disagree and sometimes even refine. https://en.m.wikipedia.org/wiki/Rubber_duck_debugging I think the big question everyone wants to skip right to and past this conversation is, will this continue to be true 2 years from now? I don’t kno…

It's a damning assertive duck, completely out of proportion to its competence. I've seen enough people led astray by talking to it.

Treat it as that enthusiastic co-worker who’s always citing blog posts and has a lot of surface knowledge about style and design patterns and whatnot, but isn’t that great on really understanding algorithms.

They can be productive to talk to but they can’t actually do your job.

Re: Human coders are still better than LLMs

#439

Post by myvoiceismypass »

andrekandre wrote: post #432
bsder wrote: post #393

Earlier quoted context omitted.

LLMs are a decent search engine a la Google circa 2005. It's been 20 years since that, so I think people have simply forgotten that a search engine can actually be useful as opposed to ad infested SEO sewage sludge. The problem is that the conversational interface, for some reason, seems to turn off the natural skepticism that people have when they use a search engine.

> the conversational interface, for some reason, seems to turn off the natural skepticism that people have n=1 but after having chatgpt "lie" to me more than once i am very skeptical of it and always double check it, whereas something like tv or yt videos i still find myself being click-baited or grifted (iow less skeptical) much more easily still... any large studies about this would be very interesting...

I get irrationally frustrated when ChatGPT hallucinates npm packages / libraries that simply do not exist.

This happens… weekly for me.

Re: Human coders are still better than LLMs

#440

Post by klabb3 »

michaelsalim wrote: post #269
marcosdumay wrote: post #44

Earlier quoted context omitted.

Hum... I imagine LLMs are better than every developer on getting CSS keywords right like the GP pointed. And I expect every LLM to be slightly worse than most classical autocompletes.

This is like saying that LLMs are better at knowing the name of that one obscure API. It's not wrong, but it's also not the hard part about CSS

Wait until they hear how good dictionaries are at spelling.
Post reply on HN