Yes, we are still winning the game, however don't be happy for what is possible today, think what is possible in a decade from now.
In that regard I am less optimistic.
Human coders are still better than LLMs
611–620 of 762 posts
Re: Human coders are still better than LLMs
#612Earlier quoted context omitted.
All. Of. Them. It's quite literally what they do because they are optimistic text generators. Not correct or accurate text generators.
This really grinds my gears. The technology is inherently faulty, but the relentless optimism of its future subtly hiding that by making it the user's mistake instead. Oh you got a wrong answer? Did you try the new OpenAI v999? Did you prompt it correctly? Its definitely not the model, because it worked for me once last night..
> it worked for me once last night..
This !
Yeah, it probably "worked for me" because they spent a gazillion hours engaging in what the LLM fanbois call "prompt engineering", but you and I would call "engaging in endless iterative hacky work-arounds until you find a prompt that works".
Unless its something extremely simple, the chances of an LLM giving you a workable answer on the first attempt is microscopic.
Re: Human coders are still better than LLMs
#613Yes, we are still winning the game, however don't be happy for what is possible today, think what is possible in a decade from now. In that regard I am less optimistic.
I think we will hit a proverbial wall at some point just like with self-driving cars.
Re: Human coders are still better than LLMs
#614Earlier quoted context omitted.
LLMs aren't my rubber duck, they're my wrong answer. You know that saying that the best way to get an answer online is to post a wrong answer? That's what LLMs do for me. I ask the LLM to do something simple but tedious, and then it does it spectacularly wrong, then I get pissed off enough that I have the rage-induced energy to do it myself.
LLMs are a decent search engine a la Google circa 2005. It's been 20 years since that, so I think people have simply forgotten that a search engine can actually be useful as opposed to ad infested SEO sewage sludge. The problem is that the conversational interface, for some reason, seems to turn off the natural skepticism that people have when they use a search engine.
Except a search engine isn't voice controlled, and able to write code for me.
Recently I did some tests with coding agents, and being able to translate a full application from AT&T Assembly into Intel Assembly compatible with NASM, in about half an hour of talking with agent, and having the end result actually working with minor tweeks isn't something a "decent search engine a la Google circa 2005." would ever been able to achieve.
In the past I would have given such a task to a junior dev or intern, to keep them busy somehow, with a bit more tool maturity I have no reason to do it in the future.
And this is the point many developers haven't yet grasped about their future in the job market.
Re: Human coders are still better than LLMs
#615Yes, we are still winning the game, however don't be happy for what is possible today, think what is possible in a decade from now. In that regard I am less optimistic.
I think we will hit a proverbial wall at some point just like with self-driving cars.
May be, yet it is working good enough for Waymo, and not so much for those losing their clients to them.
Or for the supermarkets now able to have about half the employees they used to have as cashiers.
Many times the wall is already disruption enough.
Re: Human coders are still better than LLMs
#616Earlier quoted context omitted.
The pain of that 90% work is how you get libraries and framework. Imagine having many different implementation of sorting algorithms inside your codebase.
OK now we have to spend time figuring out the framework. It's why people say just write plain Javascript, for example.
Is it? Or is it because the framework are not suitable for the project?
Re: Human coders are still better than LLMs
#617Earlier quoted context omitted.
It's trivial to demonstrate that it takes only a tiny LLM + a loop to a have a Turing complete system. The extension of that is that it is utterly crazy to think that the fact it is "a model designed to predict sequences of tokens" puts much of a limitation on what an LLM can achieve - any Turing complete system can by definition simulate any other. To the extent LLMs are limited, they are limited by training and com…
The fact they're Turing complete isn't really getting at the heart of the problem. Python is Turing complete and calling python "intelligent" would be a category error.
It is getting to the heart of the problem when the claim made is that "no matter how advanced the model" they can't be 'much more than just "really good autocomplete."'.
Given that they are Turing complete when you put a loop around them, that claim is objectively false.
Re: Human coders are still better than LLMs
#618Earlier quoted context omitted.
> LLMs can only give you code that somebody has wrote before. This is inherent. This is trivial to prove to be false. Invent a programming language that does not exist. Describe its semantics to an LLM. Ask it to write a program to solve a problem in that language. It will not always work, but it will work often enough to demonstrate that they are very much capable of writing code that has never been written before.…
This doesn't surprise me, i find LLM's are really good at interpolating and translating. so if i made up a language and gave it the rules and asked it to translate i wouldn't expect it to be bad at it.
It shouldn't surprise anyone, but it is clear evidence against the claim I replied to, and clearly a lot of people still hold on to this irrational assumption that they can't produce anything new.
Re: Human coders are still better than LLMs
#619Earlier quoted context omitted.
I’m self taught and don’t code that much but I feel like I benefit a ton from LLMs giving me specific answers to questions that would take me a lot of time to figure out with documentation and stack overflow. Or even generating snippets that I can evaluate whether or not will work. But I actually can’t imagine how you can teach someone to code if they have access to an LLM from day one. It’s too easy to take the easy…
I think you can learn a lot from debugging, and all the code I've put into prod from LLM has needed debugging (rather more than it should from the LOC count).
I agree and that’s definitely part of my current learning process. But I think someone dependent on a LLM from day one might struggle to debug their LLM generated code. Probably just feed it back to the LLM and their mileage is definitely going to vary with that approach.
Re: Human coders are still better than LLMs
#620Earlier quoted context omitted.
They will also not be offended or harbor ill will when you completely reject their "pull request" and rephrase the requirements.
They will also keep going in circles when you rephrase the requirements, unless with every prompt you keep adding to it and mentioning everything they've already suggested that got rejected. While humans occasionally also do this (hey, short memories), LLMs are infuriatingly more prone to it. A typical interaction with an LLM: "Hey, how do I do X in Y?" "That's a great question! A good way to do X in Y is Z!" "No, Z…
It's my experience that once you are in this territory, the LLM is not going to be helpful and you should abandon the effort to get what you want out of it. I can smell blood now when it's wrong; it'll just keep being wrong, cheerfully, confidently.