So your sample size is 1 task and 1 LLM?
I would recommend trying o3, opus 4 (API) with web search enabled.
Human coders are still better than LLMs
511–520 of 762 posts
Re: Human coders are still better than LLMs
#512Earlier quoted context omitted.
LLMs follow instructions. Garbage in = garbage out generally. When attention is managed and a problem is well defined and necessary materials are available to it, they can perform rather well. On the other hand, I find a lot of the loosely-goosey vibe coding approach to be useless and gives a lot of false impressions about how useful LLMs can be, both too positive and too negative.
So what you’re saying is you need to be very specific and detailed when writing your specifications for the LLM to spit out the code you want. Sounds like I can just skip the middle man and code it myself.
The advantage of a llm in that case is that you can skip a lot of syntax: make a LOT of typos in your spec, even pseudo code, will result in a working program. Not so with code. Also small logjcal mistakes, messing up left/right, x/y etc are auto fixed, maybe to your frustration if they were not mistakes, but often they are and you won't notice as they are indeed just repaired for you.
Re: Human coders are still better than LLMs
#513Earlier quoted context omitted.
This attitude is depressingly common in lots of professional, white-collar industries I'm afraid. I just came from the /r/law subreddit and was amazed at the kneejerk dismissal there of Dario Amodei's recent comments about legal work, and of those commenters who took them seriously. It's probably as much a coping mechanism as it is complacency, but, either way, it bodes very poorly for our future efforts at mitigatin…
This is the response to most new technologies; folks simply don't want to accept the future before the ramifications truly hit. If technology folk cannot see the INCREDIBLE LEAP FORWARD made by LLMs since ChatGPT came on the market, they're not seeing the forest through the trees because their heads are buried in the sand. LLMs for coding are not even close to imperfect, yet, but the saturation curves are not flatten…
Ah yes, please enjoy living in your moment and anticipating your entirely new world. I also hear all cars will be driving themselves soon and Jesus is coming back any day now.
Re: Human coders are still better than LLMs
#514Earlier quoted context omitted.
Multi modal LLMs to the rescue. Throw a screenshot or mockup in there and tell the LLM "there, like this". Gemini can do the same with videos.
Still terrible result. Multi modal = actually understands the image
Also tends to write CSS where if you actually have opinions about what good CSS is, is clearly an abomination. But most engineers don’t really care about that.
Re: Human coders are still better than LLMs
#515Earlier quoted context omitted.
LLMs aren't my rubber duck, they're my wrong answer. You know that saying that the best way to get an answer online is to post a wrong answer? That's what LLMs do for me. I ask the LLM to do something simple but tedious, and then it does it spectacularly wrong, then I get pissed off enough that I have the rage-induced energy to do it myself.
Out of curiosity can you give me an example prompt(s) you’ve used and been disappointed I see these comments all the time and they don’t reflect my experience so I’m curious what your experience has been
There are so many examples where all current top models just will loop forever even if you instruct them literally the code. We know many of them, but for instance in a tailwind react project with some degree of complexity (nested components), if you ask for something to scroll in it's space, it will never figure out min-h-0 even if you tell it. It will just loop forever rewriting the code adding and removing things, to the point of it just putting comments like 'This will add overflow' and writing js to force scroll, and it will never work even if you literally tell it what to do. Don't know why, all big and small models have this, and I found Gemini is currently the only model that sometimes randomly has the right idea but then still cannot resolve it. For this we went back to not using tailwind and back to global vanilla css, which I never thought I would say, is rather nice.
Re: Human coders are still better than LLMs
#516Earlier quoted context omitted.
This is the response to most new technologies; folks simply don't want to accept the future before the ramifications truly hit. If technology folk cannot see the INCREDIBLE LEAP FORWARD made by LLMs since ChatGPT came on the market, they're not seeing the forest through the trees because their heads are buried in the sand. LLMs for coding are not even close to imperfect, yet, but the saturation curves are not flatten…
Ah yes, please enjoy living in your moment and anticipating your entirely new world. I also hear all cars will be driving themselves soon and Jesus is coming back any day now.
I found it mildly amusing to contrast the puerile dismissiveness with your sole submission to this site: UK org's Red List of Endangered & Extinct crafts.
Re: Human coders are still better than LLMs
#517Earlier quoted context omitted.
Which LLMs and which versions?
All. Of. Them. It's quite literally what they do because they are optimistic text generators. Not correct or accurate text generators.
Most optimistic text generators do not consider repeating the stuff that was already rejected a desireable path forward. It might be the only path forward they’re aware of though.
Re: Human coders are still better than LLMs
#518Earlier quoted context omitted.
Ironically, I find it strong at things I don't know very well (CSS), but terrible at things I know well (SQL). This is probably really just a way of saying, it's better at simple tasks rather than complex ones. I can eventually get Copilot to write SQL that's complex and accurate, but I don't find it faster or more effective than writing it myself.
Actually, you've reinforced their point. It's only bad at things the user is actually good at because the user actually knows enough in that domain to find the flaws and issues. It appears to be good in domains the user is bad at because the user doesn't know any better. In reality, the LLM is just bad at all domains; it's simply whether a user has the skill to discern it. Of course, I don't believe it's as black and…
It’s like the Gell-Mann Amnesia effect but for LLMs instead of journalism.
Re: Human coders are still better than LLMs
#519Earlier quoted context omitted.
LLMs are a decent search engine a la Google circa 2005. It's been 20 years since that, so I think people have simply forgotten that a search engine can actually be useful as opposed to ad infested SEO sewage sludge. The problem is that the conversational interface, for some reason, seems to turn off the natural skepticism that people have when they use a search engine.
> LLMs are a decent search engine a la Google circa 2005. Statistical text (token) generation made from an unknown (to the user) training data set is not the same as a keyword/faceted search of arbitrary content acquired from web crawlers. > The problem is that the conversational interface, for some reason, seems to turn off the natural skepticism that people have when they use a search engine. For me, my skepticism…
All of the current models have access to Google and will do a search (or multiple searches), filter and analyze the results, then present a summary of results with links.
Re: Human coders are still better than LLMs
#520Earlier quoted context omitted.
Code is meant to power your business Bad code leads to bad business This makes me think of hosting departement; You know, which people who are using vmware, physical firewalls, dpi proxies and whatnot; On the other edge, you have public cloud providers, which are using qemu, netfilter, dumb networking devices and stuff Who got eaten by whom, nobody could have guessed ..
> Bad code leads to bad business Bad business leads to bad business. Bad code might be bad, or might be sufficient. It's situational. And by looking at what exists today, majority of code is pretty bad already - and not all businesses with bad code lead to bad businesses. In fact, some bad code are very profitable for some businesses (ask any SAP integrator).
The vast majority of code that makes money is pretty shitty.