Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Human coders are still better than LLMs

antirez.com

Post reply on HN
Print view View original

261–270 of 762 posts

Re: Human coders are still better than LLMs

#261

Post by emseetech »

marcosdumay wrote: post #56
mattnewton wrote: post #5

This matches my experience. I actually think a fair amount of value from LLM assistants to me is having a reasonably intelligent rubber duck to talk to. Now the duck can occasionally disagree and sometimes even refine. https://en.m.wikipedia.org/wiki/Rubber_duck_debugging I think the big question everyone wants to skip right to and past this conversation is, will this continue to be true 2 years from now? I don’t kno…

It's a damning assertive duck, completely out of proportion to its competence. I've seen enough people led astray by talking to it.

I use it as a rubber duck but you're right. Treat it like a brilliant idiot and never a source of truth.

I use it for what I'm familiar with but rusty on or to brainstorm options where I'm already considering at least one option.

But a question on immunobiology? Waste of time. I have a single undergraduate biology class under my belt, I struggled for a good grade then immediately forgot it all. Asking it something I'm incapable of calling bullshit on is a terrible idea.

But rubber ducking with AI is still better than let it do your work for you.

Re: Human coders are still better than LLMs

#263

Post by all2 »

marcosdumay wrote: post #56
mattnewton wrote: post #5

This matches my experience. I actually think a fair amount of value from LLM assistants to me is having a reasonably intelligent rubber duck to talk to. Now the duck can occasionally disagree and sometimes even refine. https://en.m.wikipedia.org/wiki/Rubber_duck_debugging I think the big question everyone wants to skip right to and past this conversation is, will this continue to be true 2 years from now? I don’t kno…

It's a damning assertive duck, completely out of proportion to its competence. I've seen enough people led astray by talking to it.

My typical approach is prompt, be disgusted by the output, tinker a little on my own, prompt again -- but more specific, be disgusted again by the output, tinker a littler more, etc.

Eventually I land on a solution to my problem that isn't disgusting and isn't AI slop.

Having a sounding board, even a bad one, forces me to order my thinking and understand the problem space more deeply.

Re: Human coders are still better than LLMs

#264

Post by Waterluvian »

mattnewton wrote: post #5

This matches my experience. I actually think a fair amount of value from LLM assistants to me is having a reasonably intelligent rubber duck to talk to. Now the duck can occasionally disagree and sometimes even refine. https://en.m.wikipedia.org/wiki/Rubber_duck_debugging I think the big question everyone wants to skip right to and past this conversation is, will this continue to be true 2 years from now? I don’t kno…

Same. Just today I used it to explore how a REST api should behave in a specific edge case. It gave lots of confident opinions on options. These were full of contradictions and references to earlier paragraphs that didn’t exist (like an option 3 that never manifested). But just by reading it, I rubber ducked the solution, which wasn’t any of what it was suggesting.

Re: Human coders are still better than LLMs

#265

Post by lyu07282 »

AstroBen wrote: post #224
am17an wrote: post #67

All the world's smartest minds are racing towards replacing themselves. As programmers, we should take note and see where the wind is blowing. At least don't discard the possibility and rather be prepared for the future. Not to sound like a tin-foil hat but odds of achieving something like this increase by the day. In the long term (post AGI), the only safe white-collar jobs would be those built on data which is not…

Ultimately this needs to be solved politically Making our work more efficient, or humans redundant should be really exciting. It's not set in stone that we need to leave people middle aged with families and now completely unable to earn enough to provide a good life Hopefully if it happens, it happens to such a huge amount of people that it forces a change

But that already happened to lots of industries and lots of people, we never cared before about them, now it's us so we care, but nothing is different about us. Just learn to code!

Re: Human coders are still better than LLMs

#266

Post by emseetech »

soerxpso wrote: post #246
ch4s3 wrote: post #151

Earlier quoted context omitted.

It seems like LLMs made really big strides for a while but don't seem to be getting better recently, and in some ways recent models feel a bit worse. I'm seeing some good results generating test code, and some really bad results when people go to far with LLM use on new feature work. Base on what I've seen it seems like spinning up new projects and very basic features for web apps works really well, but that doesn't…

> hallucinate whole APIs for D3 on multiple occasions, which should be really well represented in the training sets With many existing systems, you can pull documentation into context pretty quickly to prevent the hallucination of APIs. In the near future it's obvious how that could be done automatically. I put my engine on the ground, ran it and it didn't even go anywhere; Ford will never beat horses.

It's true that manually constraining an LLM with contextual data increases their performance on that data (and reduces performance elsewhere), but that conflicts with the promise of AI as an everything machine. We were promised an everything machine but if we have to not only provide it the proper context, but already know what constitutes the proper context, then it is not in any way an everything machine.

Which means it's back to being a very useful tool, but not the earth-shattering disruptor we hoped (or worried) it would be.

Re: Human coders are still better than LLMs

#267

Post by p1necone »

mattnewton wrote: post #5

This matches my experience. I actually think a fair amount of value from LLM assistants to me is having a reasonably intelligent rubber duck to talk to. Now the duck can occasionally disagree and sometimes even refine. https://en.m.wikipedia.org/wiki/Rubber_duck_debugging I think the big question everyone wants to skip right to and past this conversation is, will this continue to be true 2 years from now? I don’t kno…

> the duck can occasionally disagree

This has not been my experience. LLMs have definitely been helpful, but generally they either give you the right answer or invent something plausible sounding but incorrect.

If I tell it what I'm doing I always get breathless praise, never "that doesn't sound right, try this instead."

Re: Human coders are still better than LLMs

#268

Post by the_af »

bbarn wrote: post #207
traceroute66 wrote: post #163

Earlier quoted context omitted.

> When talking with reasonable people When talking with reasonable people, they will tell you if they don't understand what you're saying. When talking with reasonable people, they will tell you if they don't know the answer or if they are unsure about their answer. LLMs do none of that. They will very happily, and very confidently, spout complete bullshit at you. It is essentially a lotto draw as to whether the answ…

They will also not be offended or harbor ill will when you completely reject their "pull request" and rephrase the requirements.

They will also keep going in circles when you rephrase the requirements, unless with every prompt you keep adding to it and mentioning everything they've already suggested that got rejected. While humans occasionally also do this (hey, short memories), LLMs are infuriatingly more prone to it.

A typical interaction with an LLM:

"Hey, how do I do X in Y?"

"That's a great question! A good way to do X in Y is Z!"

"No, Z doesn't work in Y. I get this error: 'Unsupported operation Z'."

"I apologize for making this mistake. You're right to point out Z doesn't work in Y. Let's use W instead!"

"Unfortunately, I cannot use W for company policy reasons. Any other option?"

"Understood: you cannot use W due to company policy. Why not try to do Z?"

"I just told you Z isn't available in Y."

"In that case, I suggest you do W."

"Like I told you, W is unacceptable due to company policy. Neither W nor Z work."

...

"Let's do this. First, use Z [...]"

Re: Human coders are still better than LLMs

#269

Post by michaelsalim »

marcosdumay wrote: post #44
kccqzy wrote: post #21

Earlier quoted context omitted.

> and most devs I’d imagine What an awful imagination. Yes there are people who don't like CSS but are forced to use it by their job so they don't learn it properly, and that's why they think CSS is rote memorization. But overall I agree with you that if a company is too cheap to hire a person who is actually skilled at CSS, it is still better to hoist that CSS job onto LLMs than an unwilling human. Because that unwi…

Hum... I imagine LLMs are better than every developer on getting CSS keywords right like the GP pointed. And I expect every LLM to be slightly worse than most classical autocompletes.

This is like saying that LLMs are better at knowing the name of that one obscure API. It's not wrong, but it's also not the hard part about CSS

Re: Human coders are still better than LLMs

#270

Post by dwaltrip »

jdiff wrote: post #91
cortesoft wrote: post #58

Earlier quoted context omitted.

Currently, I find AI to be a really good autocomplete

The crazy thing is that people think that a model designed to predict sequences of tokens from a stem, no matter how advanced the model, to be much more than just "really good autocomplete." It is impressive and very unintuitive just how far that can get you, but it's not reductive to use that label. That's what it is on a fundamental level, and aligning your usage with that will allow it to be more effective.

It’s reductive and misleading because autocomplete, as it’s commonly known, existed for many years before generative AI, and is very different and quite dumber than LLMs.
Post reply on HN