Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Human coders are still better than LLMs

antirez.com

Post reply on HN
Print view View original

721–730 of 762 posts

Re: Human coders are still better than LLMs

#721

Post by lazyasciiart »

lores wrote: post #635
lazyasciiart wrote: post #583

Earlier quoted context omitted.

By that logic, it's barely worth reading a newspaper or a book. You don't know if they're giving you accurate information without doing all the research you're trying to avoid.

Recognised newspapers will curate by hiring smart, knowledgeable reporters and funding them to get reliable information. Recognised books will be written by a reliably informed author, and reviewed by other reliably informed people. There are no recognised LLMs, and their method of working precludes reliability.

Malcolm Gladwell, Jonah Lehrer, Daniel Kahneman, Matthew Walker, Stephen Glass? The New York Times, featuring Judith Miller on the existence of WMD, or their award winning podcast "Caliphate"? (Award returned when it became known the whole thing was made up, in case you haven't heard of that one).

Re: Human coders are still better than LLMs

#722

Post by therealpygon »

GiorgioG wrote: post #403
therealpygon wrote: post #399

Earlier quoted context omitted.

LLMs follow instructions. Garbage in = garbage out generally. When attention is managed and a problem is well defined and necessary materials are available to it, they can perform rather well. On the other hand, I find a lot of the loosely-goosey vibe coding approach to be useless and gives a lot of false impressions about how useful LLMs can be, both too positive and too negative.

So what you’re saying is you need to be very specific and detailed when writing your specifications for the LLM to spit out the code you want. Sounds like I can just skip the middle man and code it myself.

No, but the better specifications you provide to your “development team”, the more likely you are to get what you expected… like always.

Re: Human coders are still better than LLMs

#723

Post by Corey_ »

wvenable wrote: post #461
AdieuToLogic wrote: post #457

Earlier quoted context omitted.

> LLMs are a decent search engine a la Google circa 2005. Statistical text (token) generation made from an unknown (to the user) training data set is not the same as a keyword/faceted search of arbitrary content acquired from web crawlers. > The problem is that the conversational interface, for some reason, seems to turn off the natural skepticism that people have when they use a search engine. For me, my skepticism…

I almost never bother using Google anymore. When I search for something, I'm usually looking for an answer to question. Now I can just ask the question and get the answer without all the other stuff. I will often ask the LLM to give me web pages to look at it when I want to do further reading. As LLMs get better, I can't see myself going back to Google as it is or even as it was.

[dead]

Re: Human coders are still better than LLMs

#724

Post by sanderjd »

codr7 wrote: post #720
sanderjd wrote: post #538

Earlier quoted context omitted.

The point is that I don't dread it anymore, because now there are tools that make it a lot easier the one or two times a year I have some reason to use it.

Just wait until they're not there anymore, that's when you realize what you sacrificed.

Does this apply to other services I use? Should I avoid using Google because when it's not there anymore I'll realize what I've sacrificed?

Re: Human coders are still better than LLMs

#725

Post by willmarch »

nssnsjsjsjs wrote: post #413
willmarch wrote: post #118

Earlier quoted context omitted.

This is objectively not true.

So LLMs have sweated to debug a production issue, got to the bottom of it, realised it is worth having more unit tests so values that and then produces a solution that has more unit tests. So when you ask the LLM to write code it is opinionated and always creates a test to go with it?

I don't know how your comment relates to our comments above. I took the original comment to mean that the poster believes LLMs cannot come up with novel solutions they have never seen before. It has been proven again and again that they absolutely can solve original problems and come up with solutions that were not in their training data.

Re: Human coders are still better than LLMs

#726

Post by gitaarik »

JoshCole wrote: post #714
gitaarik wrote: post #710

Earlier quoted context omitted.

Ok, so you claim that LLMs can get smarter without human validation. So why do they hallucinate at all? And why are all reward models currently curated by humans? Or are you claiming they aren't?

I don't find it reasonable that you didn't understand my corrections, because current AI already do. So I'm exiting the conversation. https://chatgpt.com/share/683a3c88-62a8-8008-92ef-df16ce2e8a...

Ok, this is interesting indeed and I'll investigate more into it. But I think my points still stand. Let me elaborate.

An LLM only learns through input text. It doesn't have a first-person 3D experience of the world. So it can't execute physical experiments, or even understand them. It can understand the texts about it, but it can't visualize it, because it doesn't have a visual experience.

And ultimately our physical world is governed by physical processes. So at the fundamentals of physical reality, the LLMs lack understanding. And therefore will stay dependent on humans educating and correcting it.

You might get pretty impressively far with all kinds of techniques, but you can't cross this barrier with just LLMs. If you want to, you have to give it senses like humans to give it an experience of the world, and make it understand these experiences. And sure they're already working on that, but that is a lot harder to create than a comprehensive machine learning algorithm.

Re: Human coders are still better than LLMs

#727

Post by notyouraibot »

So funny story, I tried using o3 for a relatively complex task yesterday, installing XCode iOS Simulator on an external SSD, it was my first time owning and using a macOS so I was truly lost, I followed everything it told me and by the end of the hour.. things got so bad that my machine couldn't even run normal basic node projects. I had to a proper fresh boot to get things working again. So yeah lesson learned.

Re: Human coders are still better than LLMs

#728

Post by fhd2 »

fl7305 wrote: post #703
fhd2 wrote: post #564

Earlier quoted context omitted.

I would consider evolution a form of intelligence, even though I wouldn't consider nature a being. There's a branch of AI research I was briefly working in 15 years ago, based on that premise: Genetic algorithms/programming. So I'd argue humans were (and are continuously being) designed, in a way.

(non-sarcastically from me this time) Sure, I would agree with that wording. In the same way, neural networks which are trained to do a task could be said to be "designed" to do something. In my view, there's a big difference in what the training data is for a neural network, and what the neural network is "designed" for. We can train a network using word completion examples, with the intent of designing it for intel…

Yup. To counter my own points a bit:

I could also argue that the word "design" has a connotation strictly opposing emergent behaviour like evolution, as in the intelligent design "theory". So not the best word to use perhaps.

And in your example, just because we made a system that exhibits emergent behaviour to some degree, we can't assume it can "design" intelligence the way evolution did, on a much, much shorter timeline, no less.

Re: Human coders are still better than LLMs

#729

Post by vidarh »

gitaarik wrote: post #680
vidarh wrote: post #647

Earlier quoted context omitted.

> But reward models are always curated by humans. There is no inherent reason why they need to be. > So yeah theoretically you could generate reward models with LLMs, but they won't be any good, unless they are curated by other reward models that are ultimately curated by humans. This reasoning is begging the question: The reasoning is true only if the conclusion is true. It's therefore a logically invalid argument.…

Sorry but I don't follow your logic. Are you claiming that reward models that aren't curated by humans perform as well as ones that are? Then what is a reward model's function according to you?

I'm claiming exactly what I wrote: That there is no inherent reason why a human curated one needs to be better.

Re: Human coders are still better than LLMs

#730

Post by vidarh »

zahlman wrote: post #708
vidarh wrote: post #642

Earlier quoted context omitted.

> Language models with a loop absolutely aren't Turing complete. They absolutely are. It's trivial to test and verify that you can tell one to act as a suitably small Turing machine and give it instructions to use to manipulate the conversation as "the tape". Anything else would be absolutely astounding given how simple it is to implement a minimal 2-state 3-symbol Turing machine. > Assuming the model can even follow…

If you'd care to actually implement what you describe, I'm sure the resulting blog post would make a popular submission here.

It's not very interesting - it's basically showing it can run one step of a very trivial state machine , and then add a loop to let it keep running with the conversation acting as the tape io.

It's pretty hard to make any kind of complex system that can't be coerced into being Turing complete once you add iteration.

Post reply on HN