Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Human coders are still better than LLMs

antirez.com

Post reply on HN
Print view View original

391–400 of 762 posts

Re: Human coders are still better than LLMs

#391

Post by ilaksh »

yua_mikami wrote: post #42

The thing everyone forgets when talking about LLMs replacing coders is that there is much more to software engineering than writing code, in fact that's probably one of the smaller aspects of the job. One major aspect of software engineering is social, requirements analysis and figuring out what the customer actually wants, they often don't know. If a human engineer struggles to figure out what a customer wants and a…

It actually comes down to feedback loops which means iterating on software being used or attempting to be used by the customer.

Chat UIs are an excellent customer feedback loop. Agents develop new iterations very quickly.

LLMs can absolutely handle abstractions and different kinds of component systems and overall architecture design.

They can also handle requirements analysis. But it comes back to iteration for the bottom line which means fast turnaround time for changes.

The robustness and IQ of the models continue to be improved. All of software engineering is well underway of being automated.

Probably five years max where un-augmented humans are still generally relevant for most work. You are going to need deep integration of AI into your own cognition somehow in order to avoid just being a bottleneck.

Re: Human coders are still better than LLMs

#392

Post by mock-possum »

mattnewton wrote: post #5

This matches my experience. I actually think a fair amount of value from LLM assistants to me is having a reasonably intelligent rubber duck to talk to. Now the duck can occasionally disagree and sometimes even refine. https://en.m.wikipedia.org/wiki/Rubber_duck_debugging I think the big question everyone wants to skip right to and past this conversation is, will this continue to be true 2 years from now? I don’t kno…

Yeah in my experience as long as you don’t stray too far off the beaten path, LLMs are great at just parroting conventional wisdom for how to implement things - but the second you get to something more complicated - or especially tricky bug fixing that requires expensive debuggery - forget about it, they do more harm than good. Breaking down complex tasks into bite sized pieces you can reasonably expect the robot to perform is part of the art of the LLM.

Re: Human coders are still better than LLMs

#393

Post by bsder »

Buttons840 wrote: post #379
mattnewton wrote: post #5

This matches my experience. I actually think a fair amount of value from LLM assistants to me is having a reasonably intelligent rubber duck to talk to. Now the duck can occasionally disagree and sometimes even refine. https://en.m.wikipedia.org/wiki/Rubber_duck_debugging I think the big question everyone wants to skip right to and past this conversation is, will this continue to be true 2 years from now? I don’t kno…

LLMs aren't my rubber duck, they're my wrong answer. You know that saying that the best way to get an answer online is to post a wrong answer? That's what LLMs do for me. I ask the LLM to do something simple but tedious, and then it does it spectacularly wrong, then I get pissed off enough that I have the rage-induced energy to do it myself.

LLMs are a decent search engine a la Google circa 2005.

It's been 20 years since that, so I think people have simply forgotten that a search engine can actually be useful as opposed to ad infested SEO sewage sludge.

The problem is that the conversational interface, for some reason, seems to turn off the natural skepticism that people have when they use a search engine.

Re: Human coders are still better than LLMs

#394

Post by mock-possum »

traceroute66 wrote: post #167
_tom_ wrote: post #51

Earlier quoted context omitted.

For me, it's a bit like pair programming. I have someone to discuss ideas with. Someone to review my code and suggest alternative approaches. Some one that uses different feature than I do, so I learn from them.

I guess if you enjoy programming with someone you can never really trust, then yeah, sure, its "a bit like" pairs programming.

Trust, but verify ;]

Re: Human coders are still better than LLMs

#395

Post by devmor »

I have been evaluating LLMs for coding use in and out of a professional context. I’m forbidden to discuss the specifics regarding the clients/employers I’ve used them with due to NDAs, but my experience has been mostly the same as my private use - that they are marginally useful for less than one half of simple problem scenarios, and I have yet to find one that has been useful for any complex problem scenarios.

Neither of these issues is particularly damning on its own, as improvements to the technology could change this. However, the reason I have chosen to avoid them is unlikely to change; that they actively and rapidly reduce my own willingness for critical thinking. It’s not something I noticed immediately, but once Microsoft’s study showing the same conclusions came out, I evaluated some LLM programming tools again and found that I generally had a more difficult time thinking through problems during a session in which I attempted to rely on said tools.

Re: Human coders are still better than LLMs

#396

Post by const_cast »

hatefulmoron wrote: post #218
bluefirebrand wrote: post #146

Earlier quoted context omitted.

> If you remove the context of LLMs from the discussion, it reads like you're arguing that technological progress in general is bad because people would eventually struggle to live without it. I'm arguing that there are always tradeoffs and we often do not fully understand the tradeoffs we are making or the consequences of those tradeoffs 10, 50, 100 years down the road When we moved from more physical jobs to desk j…

> When we moved from more physical jobs to desk jobs many of us became sedentary and overweight. Now we are in an "obesity crisis". There's multiple factors to that, it's not just being in desk jobs, but being sedentary is a big factor. Sure, although I think our lives are generally better than they were a few hundred years ago. Besides, if you care about your health you can always take steps yourself. > The only one…

> If technology makes us redundant, then we find new jobs. Same as it ever was.

Except, of course, it's not the same as it ever was because you do actually run out of jobs. And it's significantly sooner than you think, because people have limits.

I can't be Einstein, you can't be Einstein. If that becomes the standard, you and I will both starve.

We've been pushing people up and up the chain of complexity, and we can do that because we got all the low hanging fruit. It's easy to get someone to read, then to write, then to do basic math, then to do programming. It gets a bit harder though with every step, no? Not everyone who reads has the capability of doing basic math, and not everyone who can do basic math has the capability of being a programmers.

So at each step, we lose a little bit of people. Those people don't go anywhere, we just toss them aside as a society and force them into a life of poverty. You and I are detached from that, because we've been lucky to not be those people. I know some of those people, and that's just life for them.

My parents got high paying jobs straight out of highschool. Now, highschool grads are destined to flip burgers. We've pushed people up - but not everyone can graduate college. Then, we have to think about what happens when we continue to push people up.

Eventually, you and I will not be able to keep up. You're smart, I'm smart, but not that smart. We will become the burger flippers or whatever futuristic equivalent. Uh... robot flippers.

Re: Human coders are still better than LLMs

#397

Post by lillecarl »

daveguy wrote: post #353
sigmoid10 wrote: post #158

Earlier quoted context omitted.

It's mostly a question of experience. I've been writing software long enough that when I give chat models some code and a problem, I can immediately tell if they understood it or if they got hooked on something unrelated. But junior devs will have a hell of a hard time, because the raw code quality that LLMs generate is usually top notch, even if the functionality is completely off.

> the raw code quality that LLMs generate is usually top notch, even if the functionality is completely off. I'm not even sure what this is supposed to mean. It doesn't make syntax errors? Code that doesn't have the correct functionality is obviously not "top notch".

No syntax errors, good error handling and such. Just because it implemented the wrong function doesn't mean the function is bad.

Re: Human coders are still better than LLMs

#398

Post by JeremyNT »

suddenlybananas wrote: post #304
JeremyNT wrote: post #289

Earlier quoted context omitted.

I'm in this camp... today. But there's going to be an inflection point - soon - as things continue to improve. The industry is going to change rapidly. Now is the time to either get ready for that - by being ahead of the curve, at least by being familiar with the tooling - or switch careers and cede your job to somebody who will play ball. I don't like any of this, but I see it as inevitable.

You just said they worked one comment ago and now you agree that they don't?

I'm in the same camp in that I don't use them because they don't work well enough for my own tastes. But I know what I'm doing, and I'm picky.

Clearly they do work in a general sense. People who don't want to code are making things that work this way right now!

This isn't yet replacing me, but I'm certain it will relatively soon be the standard for how software is developed.

Re: Human coders are still better than LLMs

#399

Post by therealpygon »

Buttons840 wrote: post #379
mattnewton wrote: post #5

This matches my experience. I actually think a fair amount of value from LLM assistants to me is having a reasonably intelligent rubber duck to talk to. Now the duck can occasionally disagree and sometimes even refine. https://en.m.wikipedia.org/wiki/Rubber_duck_debugging I think the big question everyone wants to skip right to and past this conversation is, will this continue to be true 2 years from now? I don’t kno…

LLMs aren't my rubber duck, they're my wrong answer. You know that saying that the best way to get an answer online is to post a wrong answer? That's what LLMs do for me. I ask the LLM to do something simple but tedious, and then it does it spectacularly wrong, then I get pissed off enough that I have the rage-induced energy to do it myself.

LLMs follow instructions. Garbage in = garbage out generally. When attention is managed and a problem is well defined and necessary materials are available to it, they can perform rather well. On the other hand, I find a lot of the loosely-goosey vibe coding approach to be useless and gives a lot of false impressions about how useful LLMs can be, both too positive and too negative.

Re: Human coders are still better than LLMs

#400

Post by AYBABTME »

ben-schaaf wrote: post #375
garciasn wrote: post #223

Earlier quoted context omitted.

This is the response to most new technologies; folks simply don't want to accept the future before the ramifications truly hit. If technology folk cannot see the INCREDIBLE LEAP FORWARD made by LLMs since ChatGPT came on the market, they're not seeing the forest through the trees because their heads are buried in the sand. LLMs for coding are not even close to imperfect, yet, but the saturation curves are not flatten…

Friendly reminder that people like you were saying the exact same thing about metaverse, VR, web3, crypto, etc.

The value of these was always a far fetch, and requires a critical mass adopting it before becoming potentially useful. But LLMs value is much more immediate and doesn't require any change in the rest of the world. If you use it and are amplified by it, you are... simply better off.
Post reply on HN