Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Human coders are still better than LLMs

antirez.com

Post reply on HN
Print view View original

601–610 of 762 posts

Re: Human coders are still better than LLMs

#601

Post by procaryote »

zonethundery wrote: post #88

No doubt the headline's claim is true, but Claude just wrote a working MCP serving up the last 10 years of my employer's work product. For $13 in api credits. While technically capable of building it on my own, development is not my day job and there are enough dumb parts of the problem my p(success) hand-writing it would have been abysmal. With rose-tinted glasses on, maybe LLM's exponentially expand the amount of s…

If your own code would have been abysmal, how can you tell if the claude generated code is any good?

Re: Human coders are still better than LLMs

#602

Post by mhast »

myvoiceismypass wrote: post #439
andrekandre wrote: post #432

Earlier quoted context omitted.

> the conversational interface, for some reason, seems to turn off the natural skepticism that people have n=1 but after having chatgpt "lie" to me more than once i am very skeptical of it and always double check it, whereas something like tv or yt videos i still find myself being click-baited or grifted (iow less skeptical) much more easily still... any large studies about this would be very interesting...

I get irrationally frustrated when ChatGPT hallucinates npm packages / libraries that simply do not exist. This happens… weekly for me.

My experience with this is that it is vital to have a system where the system can iterate on its own.

Ideally by having a test or endpoint you can call to actually run the code you want to build.

Then you ask the system to implement the function and run the test. If it hallucinates anything it will find that and fix it.

IME OpenAI is below Claude and Gemini for code.

Re: Human coders are still better than LLMs

#603

Post by motorest »

There is also another side to the mass adoption of LLMs in software engineering jobs: they can quite objectively worsen the output of human coders.

There is a class of developers who are blindly dumping the output of LLMs into PRs without paying any attention to what they are doing, let alone review the changes. This is contributing to introducing accidental complexity in the form of bolting on convoluted solutions to simple problems and even introducing types in the domain model that make absolutely no sense to anyone who has a passing understanding of the problem domain. Of course they introduce regressions no one would ever do if they wrote things by hand and tested what they wrote.

I know this, because I work with them. It's awful.

These vibecoders force the rest of us to waste eve more time reviewing their PRs. They are huge PRs that touch half the project for even the smallest change, they build and pass automated tests, but they enshitify everything. In fact, the same LLMs used by these vibecoders start to struggle how to handle the project after these PRs are sneaked in.

It's tiring and frustrating.

I apologize for venting. It's just that in this past week I lost count of the number of times I had these vibecoders justifying shit changes going into their PRs as "but Copilot did this change", as if that makes them any good. I mean, a PR to refactor the interface of a service also sneaks in changes to the connection string, and they just push the change?

Re: Human coders are still better than LLMs

#604

Post by devjab »

AstroBen wrote: post #205
devjab wrote: post #101

Earlier quoted context omitted.

LLM's do no software engineering at all, and that can be fine. Because you don't actually need software engineering to create successful programs. Some applications will not even need software engineering for their entire life cycles because nobody is really paying attention to efficiency in the ocean of poor cloud management anyway. I actually imagine it's the opposite of what you say here. I think technically incli…

That's the thing too right.. the vast majority of software out there barely needs to scale or be super efficient It does need to be reliable, though. LLMs have proven very bad at that

> the vast majority of software out there barely needs to scale or be super efficient

That was the way I saw it for a while. In recent months I've begun to wonder if I need to reevaluate that, because it's become clear to me that scaling doesn't actually start from zero. By zero I mean that I was naive enough to think that all programs, even the most googled programmed one by a completely new junior would at least have, some, efficiency... but some of these LLM services I get to work on today are so bad they didn't start at zero but at some negative number. It would have been less of an issue if our non-developer-developers didn't use Python (or at least used Python with ruff/pyrefly/whateveryoulike, but some of the things they write can't even scale to do minimal BI reporting.

Re: Human coders are still better than LLMs

#605

Post by gumbojuice »

I like to use llm to produce code for known problems that I don't have memorized.

I memorize really little and tend to spend time on reinventing algorithms or looking them up in documentation. Verifying is easy except the fee cases where the llm produces something really weird. But then fallback to docs or reinventing.

Re: Human coders are still better than LLMs

#606

Post by austin-cheney »

zxexz wrote: post #597
austin-cheney wrote: post #595

In many cases developers are a low expectation commodity. In those cases I strongly believe humans are entirely replaceable by AI and I am saying that as somebody with an exceptionally low opinion of LLMs. Honestly though, when that replacement comes there is no sympathy to be had. Many developers have brought this upon themselves. For roughly the 25 year period from 1995 to 2020 businesses have been trying to turn d…

I find myself wondering about your story, and would love it if you would elaborate more. I have gotten some use out of LLMs, and have been quite involved in training a few compute intensive (albeit domain-specific) ones. Maybe it's the way you talk about 'developers'. Nothing I have seen has felt like the sky falling on an industry; to me at most it's been the sky falling on a segment of silicon valley.

It’s all about perspectives. In many cases the perspectives between how a developer identifies their level of participation versus what they actually do as a work activity differ substantially. For example many developers may refer to themselves as engineers when they have done nothing remotely close to measurements, research, or policy creation in compliance attainment.

With that out of the way let’s look only at what many developers actually do. If a given developer only uses a framework to put text on screen or respond to a user interaction then they can be replaced. LLMs can already do this better than people. That becomes substantially more true after accounting for secondary concerns: security, accessibility, performance, regression, and more.

If a developer is doing something more complex that accounts for systems analysis or human behavior then LLMs are completely insufficient.

Re: Human coders are still better than LLMs

#608

Post by sigmoid10 »

daveguy wrote: post #353
sigmoid10 wrote: post #158

Earlier quoted context omitted.

It's mostly a question of experience. I've been writing software long enough that when I give chat models some code and a problem, I can immediately tell if they understood it or if they got hooked on something unrelated. But junior devs will have a hell of a hard time, because the raw code quality that LLMs generate is usually top notch, even if the functionality is completely off.

> the raw code quality that LLMs generate is usually top notch, even if the functionality is completely off. I'm not even sure what this is supposed to mean. It doesn't make syntax errors? Code that doesn't have the correct functionality is obviously not "top notch".

High quality code is not just correct syntax. In fact if the syntax is wrong, it wouldn't be low quality, it simply wouldn't work. Even interns could spot that by running it. But in professional software development environments, you have many additional code requirements like readability, maintainability, overall stability or general good practice patterns. I've seen good engineers deliver high quality code that was still wrong because of some design oversight or misunderstanding - the exact same thing you see from current LLMs. Often you don't even know what is wrong with an approach until you see it cause a problem. But you should still deliver high quality code in the meantime if you want to be good at your job.

Re: Human coders are still better than LLMs

#609

Post by kaycey2022 »

It doesn't matter. The hiring of cost center people like engineers depends on the capital cycle. Hiring peaked when money and finance was the cheapest. Now it's not anymore. In the absence of easy capital, hiring will plummet.

Another factor is the capture of market sectors by Big Co. When buyers can only approach some for their products/services, the Big Co can drastically reduce quality and enshittify without hurting the bottom line much. This was the big revelation when Elon gutted Twitter.

And so we are in for interesting times. On the plus side, it is easier than ever to create software and distribute it. Hiring doesn't matter if I can get some product sense and make some shit worth buying.

Re: Human coders are still better than LLMs

#610

Post by drodgers »

materiallie wrote: post #556
drodgers wrote: post #474

Earlier quoted context omitted.

I don't know how someone could be following the technical progress in detail and hold this view. The progress is astonishing, and the benchmarks are becoming saturated so fast that it's hard to keep track. Are there plenty of gaps left between here and most definitions of AGI? Absolutely. Nevertheless, how can you be sure that those gaps will remain given how many faculties these models have already been able to exce…

I think benchmark targeting is going to be a serious problem going forward. The recent Nate Silver podcast on poker performance is interesting. Basically, the LLM models still suck at playing poker. Poker tests intelligence. So what gives? One interesting thing is that for whatever reason, poker performance isn't used a benchmark in the LLM showdown between big tech companies. The models have definitely improved in t…

I think that's generally fair, but this point goes too far:

> improve benchmarks one by one

If you're right about that in the strong sense — that each task needs to be optimised in total isolation — then it would be a longer, slower road to a really powerful humanlike system.

What I think is really happening though that each specific task (eg. coding) is having large spillover effects on other areas (eg. helping them to be better at extended verbal reasoning even when not writing any code). The AI labs can't do everything at once, so they're focusing where:

- It's easy to generate more data and measure results (coding, maths etc.) - There's a relative lack of good data in the existing training corpus (eg. good agentic reasoning logic - the kinds of internal monologs that humans rarely write down) - Areas where it would be immediately useful for the models to get better in a targeted way (eg. agentic tool-use; developing great hypothesis generation instincts in scientific fields like algorithm design, drug discovery and ML research)

By the time those tasks are optimised, I suspect the spill over effects will be substantial and the models will generally be much more capable.

Beyond that, the labs are all pretty open about the fact that they want to use the resulting AI talents for coding, reasoning and research skills to accelerate their own research. If that works (definitely not obvious yet) then finding ways to train a much broader array of skills could be much faster because that process itself would be increasingly automated.

Post reply on HN