Human coders are still better than LLMs • Hacker News Board

Re: Human coders are still better than LLMs

#731

Post by vidarh » Sat, May 31, 2025, 9:15 AM UTC

skydhash wrote: post #671

vidarh wrote: post #666

Earlier quoted context omitted.

> If you give it the rules to generate something, why can't it generate it? It can, but that does not mean that what is generate is not new, unless the rules in question constrains the set to the point where onely one outcome is possible. If I tell you that a novel has a minimum of 40,000 words, it does not mean that no novel is, well, novel (not sorry), just because I've given you rules to stay within. Any novel wil…

I was referring as new as some orthogonal dimension in the same space. If we're referring to your definition, any slight changes in the parameters results in something new. I was arguing more about if the model knows about axes x and y, then it's output is constrained to a plane unless you add z. But more often than not it's output will be a cylinder (extruded from a circle in the x,y plane) instead of a sphere. The…

You completely fail to understand Gödel's incompleteness theorems and the halting problem if you think they are evidence of something humans can do that machines can not. It makes the discussion rather pointless if you lack that fundamental understanding of the subject.

Re: Human coders are still better than LLMs

#732

Post by vidarh » Sat, May 31, 2025, 9:44 AM UTC

skydhash wrote: post #679

vidarh wrote: post #668

Earlier quoted context omitted.

> As long as you create the grammar, the language exists. Yes, but it didn't exist during training . Nothing in the training data would provide pre-existing content for the model to produce from, so the output would necessarily be new. > But if you take the time to explain each concept.... Based on the argument you presented, nothing a human does is new, because it is all based on our pre-exististing learned rules of…

See my definition at : https://news.ycombinator.com/item?id=44137201 As mentioned there, I was arguing that without being prompted, there's no way that it can add something that is not a combination of the training data. And that combination does not act on the same terms that you would expect someone learning the same material would do. In Linear regression, you can reduce a big amount of data to a small amount of f…

That comment doesn't provide anything resembling a coherent definition.

The rest of what you wrote here is either also true for humans or not true for machines irrespective of your definitions unless you can demonstrate that humans can exceed the Turing computable.

You can not.

Re: Human coders are still better than LLMs

#733

Post by GreenWatermelon » Sat, May 31, 2025, 10:57 AM UTC

spamizbad wrote: post #254

xhevahir wrote: post #184

Earlier quoted context omitted.

This attitude is depressingly common in lots of professional, white-collar industries I'm afraid. I just came from the /r/law subreddit and was amazed at the kneejerk dismissal there of Dario Amodei's recent comments about legal work, and of those commenters who took them seriously. It's probably as much a coping mechanism as it is complacency, but, either way, it bodes very poorly for our future efforts at mitigatin…

I think it's pretty reasonable to take a CEO's - any CEO in any industry - statements with a grain of salt. They are under tremendous pressure to paint the most rosy picture possible of their future. They actually need you to "believe" just as much as their team needs to deliver.

Just a grain? I say take it with a gargantuan Train loaded with salt on all cars. An entire salt mine's worth. Markets, and CEOs, are downright insane, and they are the only ones who stand to profit from this situation, and have everything to gain.

Re: Human coders are still better than LLMs

#734

Post by cushychicken » Sat, May 31, 2025, 11:00 AM UTC

zahlman wrote: post #718

cushychicken wrote: post #711

Earlier quoted context omitted.

I have a mountain of professional experience that indicates many software engineers are not very good at it either. Why would I add a subpar translation layer into the process of achieving my goals? There’s no inherent value in that.

> Why would I add a subpar translation layer into the process of achieving my goals? Because you don't have a choice. Your thoughts are not code.

I'd still take ChatGPT as that translation layer over all but the best SWEs I've worked with.

Re: Human coders are still better than LLMs

#735

Post by AYBABTME » Sat, May 31, 2025, 11:16 AM UTC

ben-schaaf wrote: post #436

AYBABTME wrote: post #400

Earlier quoted context omitted.

The value of these was always a far fetch, and requires a critical mass adopting it before becoming potentially useful. But LLMs value is much more immediate and doesn't require any change in the rest of the world. If you use it and are amplified by it, you are... simply better off.

Frankly I disagree that LLMs value is immediate. What I do see is a whole lot of damage it's causing, just like the hype cycles before it. It's fine for us to disagree on this, but to say I'm burying my head in the sand not wanting to accept "the future" is exactly the same hype-driven bullshit the crypto crowd was pushing.

That's why it's what I define as immediate value. It's undeniably incredibly amplifying to me, whether you or others agree or not. No network effect required. It doesn't matter whether I convince anyone else of the value, I can capture it all on my own. Unlike ponzi-schemes like web3 or VR experiences that require an entire shift in everyday life and an ecology of supporting software.

I don't need to convince anyone that LLMs are enabling me to do a lot more. This is what makes this hype different. It has bones. Once you've found a way to leverage it, it's undeniably helpful regardless of your prior disposition. Everyone else can say they're not useful and it rings hollow because it obviously is to me. And thus probably useful to everyone else too.

Re: Human coders are still better than LLMs

#736

Post by msgodel » Sat, May 31, 2025, 11:40 AM UTC

vidarh wrote: post #642

msgodel wrote: post #622

Earlier quoted context omitted.

Language models with a loop absolutely aren't Turing complete. Assuming the model can even follow your instructions the output is probabilistic so in the limit you can guarantee failure. In reality though there are lots of instructions LLMs fail to follow. You don't notice it as much when you're using them normally but if you want to talk about computation you'll run into trivial failures all the time. The last time…

> Language models with a loop absolutely aren't Turing complete. They absolutely are. It's trivial to test and verify that you can tell one to act as a suitably small Turing machine and give it instructions to use to manipulate the conversation as "the tape". Anything else would be absolutely astounding given how simple it is to implement a minimal 2-state 3-symbol Turing machine. > Assuming the model can even follow…

Seriously, get any instruct tuned language model and try to do one iteration with grammar production rules. It's coin flip at best if they get it right.

Re: Human coders are still better than LLMs

#737

Post by perrygeo » Sat, May 31, 2025, 12:08 PM UTC

I'm wondering if this statement might be definitionally self-evident. In other words, the entire reason we write software is that it has value to ourselves and other humans - so we have to be involved in its specification. Computers do things faster, more accurately, and in some cases more creatively than human could. But in the end, what a computer produces is still for the benefit of humans and subject to all the human constraints. Aggregate human behavior determines if software is a success or not.

If software is about meeting human demands, humans will always write its requirements, by definition. If we build another machine like LLMs, well the design of those LLMs is subject to human demands. There is no point at which we can demand perfection but not be involved in its definition.

Re: Human coders are still better than LLMs

#738

Post by energy123 » Sat, May 31, 2025, 12:51 PM UTC

klabb3 wrote: post #683

energy123 wrote: post #550

Earlier quoted context omitted.

We're seeing diminishing returns in benchmark space , which is partly an artefact of construction, not an absolutely true commentary on how things are progressing.

Well yes but there is no better way to measure without resorting to pure hearsay. How would you make an accurate assessment of something so inherently vague?

Alter the benchmark space that we care about, for example focus only on ARC-AGI-2 and then suddenly the gains are no longer diminishing but are accelerating.

Re: Human coders are still better than LLMs

#739

Post by vidarh » Sat, May 31, 2025, 1:06 PM UTC

msgodel wrote: post #736

vidarh wrote: post #642

Earlier quoted context omitted.

> Language models with a loop absolutely aren't Turing complete. They absolutely are. It's trivial to test and verify that you can tell one to act as a suitably small Turing machine and give it instructions to use to manipulate the conversation as "the tape". Anything else would be absolutely astounding given how simple it is to implement a minimal 2-state 3-symbol Turing machine. > Assuming the model can even follow…

Seriously, get any instruct tuned language model and try to do one iteration with grammar production rules. It's coin flip at best if they get it right.

I have tried that many times and had good results.

Re: Human coders are still better than LLMs

#740

Post by JoshCole » Sat, May 31, 2025, 1:40 PM UTC

gitaarik wrote: post #726

JoshCole wrote: post #714

Earlier quoted context omitted.

I don't find it reasonable that you didn't understand my corrections, because current AI already do. So I'm exiting the conversation. https://chatgpt.com/share/683a3c88-62a8-8008-92ef-df16ce2e8a...

Ok, this is interesting indeed and I'll investigate more into it. But I think my points still stand. Let me elaborate. An LLM only learns through input text. It doesn't have a first-person 3D experience of the world. So it can't execute physical experiments, or even understand them. It can understand the texts about it, but it can't visualize it, because it doesn't have a visual experience. And ultimately our physica…

You're doing this thing again where you say tons of things that aren't true.

> An LLM only learns through input text.

This is false. There already exist LLM which understand more than just text. Relevant search term: multi-modality.

> It doesn't have a first-person 3D experience of the world.

Again false. It is trivial to create such an experience with multi-modality. Just set up an input device which streams that.

> So it can't execute physical experiments, or even understand them.

Here you get confused again. It doesn't follow, based on perceptual modality, that someone can't do or understand experiments. Hellen Keller can be both blind, but also do an experiment.

Beyond just being confused, you also make another false claim. Current LLMs already have the capacity to run experiments and do so. Search terms: tool usage, ReAct loop, AI agents.

> It can understand the texts about it, but it can't visualize it, because it doesn't have a visual experience.

Again, false!

Multi-modal LLMs currently possess the ability to generate images.

> And ultimately our physical world is governed by physical processes. So at the fundamentals of physical reality, the LLMs lack understanding. And therefore will stay dependent on humans educating and correcting it.

Again false. The same sort of reasoning would claim that Hellen Keller couldn't read a book, but braille exists. The ability to acquire information outside an umwelt is a capability that intelligence enables.