Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The insecure evangelism of LLM maximalists

lewiscampbell.tech

Post reply on HN
Print view View original

141–150 of 295 posts

Re: The insecure evangelism of LLM maximalists

#141

Post by kokanee »

rudedogg wrote: post #3

Hearing people on tech twitter say that LLMs always produce better code than they do by hand was pretty enlightening for me. LLMs can produce better code for languages and domains I’m not proficient in, at a much faster rate, but damn it’s rare I look at LLM output and don’t spot something I’d do measurably better. These things are average text generation machines. Yes you can improve the output quality by writing a…

> These things are average text generation machines.

Funny... seems like about half of devs think AI writes good code, and half think it doesn't. When you consider that it is designed to replicate average output, that makes a lot of sense.

So, as insulting as OP's idea is, it would make sense that below-average devs are getting gains by using AI, and above-average devs aren't. In theory, this situation should raise the average output quality, but only if the training corpus isn't poisoned with AI output.

I have an anecdote that doesn't mean much on its own, but supports OP's thesis: there are two former coworkers in my linkedin feed who are heavy AI evangelists, and have drifted over the years from software engineering into senior business development roles at AI startups. Both of them are unquestionably in the top 5 worst coders I have ever worked with in 15 years, one of them having been fired for code quality and testing practices. Their coding ability, transition to less technical roles, and extremely vocal support for the power of vibe coding definitely would align with OP's uncharitable character evaluation.

Re: The insecure evangelism of LLM maximalists

#142

Post by zahlman »

CapsAdmin wrote: post #79
rudedogg wrote: post #3

Hearing people on tech twitter say that LLMs always produce better code than they do by hand was pretty enlightening for me. LLMs can produce better code for languages and domains I’m not proficient in, at a much faster rate, but damn it’s rare I look at LLM output and don’t spot something I’d do measurably better. These things are average text generation machines. Yes you can improve the output quality by writing a…

I've been playing with vibe coding a lot lately and I think in most cases, the current SOTA LLM's don't produce code that I'd be satisfied with. I kind of feel like LLM's are really really good at hacking on a messy and fragile structure, because they can "keep track many things in their head" BUT An LLM can write a PNG decoder that works in whatever language I choose in one or a few shots. I can do that too, but it…

I kinda like Theo's take on it (https://www.youtube.com/watch?v=Z9UxjmNF7b0): there's a sliding scale of how much slop should reasonably be considered acceptable and engineers are well advised to think about it more seriously. I'm less sold on the potential benefits (since some of the examples he's given are things that I would also find easy by hand), but I agree with the general principle that having the option to do things in a super-sloppy way, combined with spending time developing intuition around having that access (and what could be accomplished that way), can produce positive feedback loops.

In short: when you produce the PNG decoder, and are satisfied with it, it's because you don't have a good reason to care about the code quality.

> Maybe objectively it does, but can we convince them that it does?

I strongly doubt it, and that's why articles like TFA project quite a bit of concern for the future. If non-engineers end up accepting results from a low-quality, not-quite-correct system, that's on them. If those results compromise credentials, corrupt databases etc., not so much.

Re: The insecure evangelism of LLM maximalists

#143

Post by zahlman »

raddan wrote: post #110
CapsAdmin wrote: post #79

Earlier quoted context omitted.

I've been playing with vibe coding a lot lately and I think in most cases, the current SOTA LLM's don't produce code that I'd be satisfied with. I kind of feel like LLM's are really really good at hacking on a messy and fragile structure, because they can "keep track many things in their head" BUT An LLM can write a PNG decoder that works in whatever language I choose in one or a few shots. I can do that too, but it…

I tried vibe coding a BMP decoder not too long ago with the rationale being “what’s simpler than BMP?” What I got was an absolute mess that did not work at all. Perhaps this was because, in retrospect, BMP is not actually all that simple, a fact that I discovered when I did write a BMP decoder by hand. But I spent equal time vibe coding and real coding. At the end of the real coding session, I understood BMP, which I…

Just out of curiousity (as someone fairly familiar with the BMP spec, and also PNG incidentally): what did you find to be the trickiest/most complex aspects?

Re: The insecure evangelism of LLM maximalists

#144

Post by vitaflo »

bashtoni wrote: post #27

The tech industry seems to attract people that feel personally attacked when someone else makes different choices that they do. "Why are you using Go? Rust is best! You should be using that!" "Don't use AWS CDK, use Terraform! Don't you know anything?"

You need to be fairly smart to be in tech. People who grew up smart and were told they were tend to view it as part of their self worth. If someone disagrees with this person later on, their self with has been attacked so of course they are going to lash out.

The worst thing you can say to a dev is they are wrong. Most will do everything in their power to prove otherwise, even on the dumbest of topics.

Re: The insecure evangelism of LLM maximalists

#145

Post by zahlman »

jordwest wrote: post #44
jimbo1167 wrote: post #25

Earlier quoted context omitted.

How are you judging that you'd write "better" code? More maintainable? More efficient? Does it produce bugs in the underlying code it's generating? Genuinely curious where you see the current gaps.

For me the biggest gaps in LLM code are: - it adds superfluous logic that is assumed but isn’t necessary - as a result the code is more complex, verbose, harder to follow - it doesn’t quite match the domain because it makes a bunch of assumptions that aren’t true in this particular domain They’re things that can often be missed in a first pass look at the code but end up adding a lot of accidental complexity that bit…

I've seen a lot of examples where it fails to take advantage of previous work and rewrites functionality from scratch.

Re: The insecure evangelism of LLM maximalists

#146

Post by ambicapter »

Arch-TK wrote: post #57
wild_egg wrote: post #41

Earlier quoted context omitted.

After a certain experience level though, I think most of us get to the point of knowing what that difference in quality actually matters. Some seniors love to bikeshed PRs all day because they can do it better but generally that activity has zero actual value. Sometimes it matters, often it doesn't. Stop with the "I could do this better by hand" and ask "is it worth the extra 4 hours to do this by hand, or is this ac…

"actually good enough to meet the goals?" There's "okay for now" and then there's "this is so crap that if we set our bar this low we'll be knee deep in tech debt in a month". A lot of LLM output in the specific areas _I_ work in is firmly in that latter category and many times just doesn't work.

I mean, there's also, "this looks fine but if I actually had written this code I would've naturally spent more time on it which would have led me to anticipate the future of this code just a little bit more and I will only feel that awkwardness when I come back to this code in two weeks, and then we'll do it all over again". It's a spectrum.

Re: The insecure evangelism of LLM maximalists

#147

Post by falloutx »

sarchertech wrote: post #105

I just want some externally verifiable numbers. If AI is a 10x improvement, we should be seeing new operating systems. If it’s 5x we should see new game engines. If it’s 2x we should see massive amounts of new features for popular open source projects. If it’s less than that, then it’s more like adding syntax highlighting or moving from Java to Ruby on Rails. Both of those were nice, but people weren’t breathlessly s…

I dont think that will happen, Not at least now. I dont even think AI would be suitable for creating an OS or a game engine. What seems to be increasing is no of saas products asking for subscriptions and most of the productivity, if any, is being used to add AI features to the apps.

Re: The insecure evangelism of LLM maximalists

#148

Post by visarga »

throwawayffffas wrote: post #115
wild_egg wrote: post #41

Earlier quoted context omitted.

After a certain experience level though, I think most of us get to the point of knowing what that difference in quality actually matters. Some seniors love to bikeshed PRs all day because they can do it better but generally that activity has zero actual value. Sometimes it matters, often it doesn't. Stop with the "I could do this better by hand" and ask "is it worth the extra 4 hours to do this by hand, or is this ac…

LLM generated code is technical debt. If you are still working on the codebase the next day it will bite you. It might be as simple as an inconvenient interface, a bunch of duplicated functions that could just be imported, but eventually you are going to have to pay it.

Untested undocumented LLM code is technical debt, but if you do specs and tests it's actually the opposite, you can go beyond technical debt and regenerate your code as you like. You just need testing to be so good it guarantees the behavior you care about, and that is easier in our age of AI coding agents.

Re: The insecure evangelism of LLM maximalists

#149

Post by roadside_picnic »

lekevicius wrote: post #73
perardi wrote: post #19

5 anti-AI posts on the home page of Hacker News…yeah, plenty of insecure evangelism amongst the skeptics, too.

Is there enough of new blood on HN? For me it was the best place, my favorite website, when I was entering startup scene. Loved it. I don't think a lot of young founders I know ever go here...

> young founders

While this place has always been attractive to people building startups, back in the day (my original account is from 2009) "Hacker" News was much more about Hackers. Most people posting here had read "On Lisp", respected Paul Graham as a programmer and were enthusiastic about programming and solving problems above all else.

I'm honestly curious how many people that visit HN today even know what a "y combinator" is, and I have a pretty reasonable guess as to how many have implemented it for fun (though probably the applicative order version).

Re: The insecure evangelism of LLM maximalists

#150

Post by ambicapter »

throwawayffffas wrote: post #107
rudedogg wrote: post #3

Hearing people on tech twitter say that LLMs always produce better code than they do by hand was pretty enlightening for me. LLMs can produce better code for languages and domains I’m not proficient in, at a much faster rate, but damn it’s rare I look at LLM output and don’t spot something I’d do measurably better. These things are average text generation machines. Yes you can improve the output quality by writing a…

> Hearing people on tech twitter say that LLMs always produce better code than they do by hand was pretty enlightening for me. That's hilarious LLM code is always very bad. It's only merit is it occasionally works. > LLMs can produce better code for languages and domains I’m not proficient in. I am sure that's not true.

It seems true by construction. If you're not proficient in a language than the bar for "better than you" is necessarily lower.
Post reply on HN