Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The problem with "vibe coding"

dylanbeattie.net

Post reply on HN
Print view View original

141–150 of 154 posts

Re: The problem with "vibe coding"

#141

Post by skydhash »

Zambyte wrote: post #140
skydhash wrote: post #138

Earlier quoted context omitted.

> Even if the book is 100% right, there will be many passages you don't understand, and may even misinterpret. Which is why no one reads only one book on a particular subject, especially if one is new to the domain. Often you bring in an alternate, but correct perspective from another book. The risks of hallucination with LLMs' responses outweighs the advantage of multiple perspectives (even if a broken clock is righ…

No one reads only one book on a subject? People act on mere headlines all the time. Reading one whole book would be a huge win over the status quo > The risks of hallucination with LLMs' responses outweighs the advantage of multiple perspectives You can ask a model a question multiple times. You can ask different models the same question. You can ask the same question different ways. You can do this automatically wit…

> You can ask a model a question multiple times. You can ask different models the same question. You can ask the same question different ways. You can do this automatically with agentic frameworks or reasoning models.

LLMs are for generating text, not producing knowledge. The actual knowledge come from the signal/noise ratio of the training data, not the training method.

It's like recording a violin in a noisy environment, encoding it with a low-bitrate lossy encoder and trying to get the PCM of only the violin back.

We've tried to use a better environment for recording specific notes from the violin (better training data), adjust the parameter of the lossy encoder (tuning the model), mix other violin samples to the respone (RAG, MCP), and redefining what a violin should sound like (LLMs marketing).

But the simpler solution is to record it correctly and play it later (AKA going to the source).

Re: The problem with "vibe coding"

#142

Post by int_19h »

dragonwriter wrote: post #92
int_19h wrote: post #86

Earlier quoted context omitted.

Right, but that's exactly what static typing does in most cases - it provides model with more explicit context for what it's doing. For the same reason, they tend to handle XML better than JSON - sure, both are trees, but XML is has redundancy, and that redundancy helps keep the model on the rails, so to speak. I actually wonder if the perfect LLM language would be something a lot more like COBOL - not in a sense of…

> For query languages, at least, they seem to like SQL the most of all the things I've tried, and I strongly suspect that it's the same underlying cause. The reason they like SQL more than other query languages is primarily that their training data has orders of magnitude more of it than any other query language, and that advantage is so huge that any other possible advantage would probably have comparatively negligi…

I'm not so sure about that. This was in comparison to e.g. walking object graphs in Python, C#, and JavaScript using the usual APIs (i.e. where querying one one-to-many relationship looks like Foo.Bars rather than joins, and using map/fold/filter or equivalents). I would expect there to be a lot more of that kinda stuff in the training set than SQL.

Re: The problem with "vibe coding"

#143

Post by int_19h »

cco wrote: post #129
int_19h wrote: post #76

Earlier quoted context omitted.

> perfectly in the spirit of the hacker ethos I would strongly disagree on this count. Hacker ethos, at least as traditionally understood, emphasizes grokking and elegance.

I couldn't think of any two concepts less associated with "hacking". I think you may have an understanding of that word that is not very common.

It sounds like you're thinking about a "hack", as in quick and dirty solution.

But that does not relate to "hacking", and most certainly does not relate to "hacker ethos". And while "hacking" is sort of ambiguous, I don't recall ever seeing "hacker ethos" defined by quick and dirty solutions.

Re: The problem with "vibe coding"

#144

Post by Zambyte »

skydhash wrote: post #141
Zambyte wrote: post #140

Earlier quoted context omitted.

No one reads only one book on a subject? People act on mere headlines all the time. Reading one whole book would be a huge win over the status quo > The risks of hallucination with LLMs' responses outweighs the advantage of multiple perspectives You can ask a model a question multiple times. You can ask different models the same question. You can ask the same question different ways. You can do this automatically wit…

> You can ask a model a question multiple times. You can ask different models the same question. You can ask the same question different ways. You can do this automatically with agentic frameworks or reasoning models. LLMs are for generating text, not producing knowledge. The actual knowledge come from the signal/noise ratio of the training data, not the training method. It's like recording a violin in a noisy enviro…

I would guess it's been around 1-2 years since you last tried seriously using a language model for any practical use cases, right? It makes sense to not use them if that is your experience, but it's hard to imagine anyone saying this who has actually been following along for the last few years to now. SOTA from 2 years ago now comfortably runs on a single GPU. A high end GPU can easily out perform SOTA from then both in (general!) knowledge reproduction, and capabilities (including saying things like "I don't know" when you ask beyond the encoded knowledge).

Re: The problem with "vibe coding"

#145

Post by nyarlathotep_ »

thaanpaa wrote: post #99
az09mugen wrote: post #73

Earlier quoted context omitted.

As you said, I think also the key differences between skilled and unskilled devs in the context of AI tooling are : 1°) Know how to diagnose and fix critical problems by themselves, because there will be bugs in production for which AI won't be any help 2°) Write maintainable code, AI doesn't care at all of maintainability of code while devs should (must imho) consider pasting AI code as a merge/pull request. For 2°)…

I think it's already happening. I've been using Claude a lot since 3.5. For a while, I barely bothered to check what it was doing because the code it generated tended to be fairly straightforward and usually worked on the first try. But in the last few weeks, I've started reading everything line by line again, because now it's not only often incorrect but also written in a goofy, convoluted way. AI-generated code bei…

There's a lot of subtle, pernicious stuff that "Sneaks In" too--not even stylistic either.

Things like a loss of context about the nature of some installed dependency or something.

You really have to know what you're doing and read carefully to avoid glossing over things that'll catch up to you later.

Re: The problem with "vibe coding"

#146

Post by nyarlathotep_ »

az09mugen wrote: post #73
timr wrote: post #28

Earlier quoted context omitted.

Yeah, the most important part of that video is where he says: "it works well enough" . I think there's lots of room for this kind of bespoke software that's fundamentally unmaintainable -- and not something anyone could reasonably afford to make with professionals -- but good enough for purpose. This seems like a great, life-changing example, but it's still subject to the same thing the article is talking about. I've…

As you said, I think also the key differences between skilled and unskilled devs in the context of AI tooling are : 1°) Know how to diagnose and fix critical problems by themselves, because there will be bugs in production for which AI won't be any help 2°) Write maintainable code, AI doesn't care at all of maintainability of code while devs should (must imho) consider pasting AI code as a merge/pull request. For 2°)…

My experience too, in many cases, is that I already know (even if in a vague sense) how to do the thing I'm trying to coax out of the model, but a quick prompt "split this string on $CHAR" or whatever is much faster than me fiddling with forgotten bash syntax.

This approach is different than blind trust in the output from a place of ignorance.

Re: The problem with "vibe coding"

#147

Post by Vanit »

I agree with the article, but that's not how the vibe coders see themselves. From their perspective they can't see the gap between programming and product, and in my experience are pretty hostile to feedback from real software engineers.

Re: The problem with "vibe coding"

#148

Post by __MatrixMan__ »

lcnPylGDnU4H9OF wrote: post #137
__MatrixMan__ wrote: post #60

This is about the problem with vibe coding your way to a product. But if the vibes are strong enough, maybe we don't need products at all? They do tend to complicate things, with all of their moats and such. I never wanted a product that did the thing, I really just wanted to do the thing. "Works on my machine" might be good enough if you're unlikely to want to repeat yourself on a different machine.

> I never wanted a product that did the thing, I really just wanted to do the thing. Would you vibe code your daily driver car's ECU or a high-frequency trading application to use for your 401k? If you were to do these things (more power to you), I rather suspect you'd still do a whole lot of research and critical thinking beforehand, which sort of obviates the "vibes".

It is hard to imagine trusting an AI that much. But plenty of things happening today would've been hard to imagine by the people who came before. So yeah, let's make that assumption.

At that point, I don't think we'd be using the AI to code something and then switch to a mode where we're now using the recently-coded thing.

We'd just ask the the car to go, and it would happen. Then we'd suggest a strategy for making trades, and that would happen. If the AI decided to write some code and fork a subprocess along the way... that would be an detail that we'd be unaware of.

Re: The problem with "vibe coding"

#149

Post by vaylian »

Zambyte wrote: post #110
vaylian wrote: post #90

Earlier quoted context omitted.

> ask for an explanation That is risky. The AI might just hallucinate an explanation.

So can any human-based sources. A healthy learning experience with AI takes discipline, just as it does with humans.

That is nonsense. A competent human will provide you with an explanation that is based on logic. LLMs just put together words that are statistically most likely to be used together given the current input. That is why you can get hallucinations with LLMs but not with humans.

Re: The problem with "vibe coding"

#150

Post by Zambyte »

vaylian wrote: post #149
Zambyte wrote: post #110

Earlier quoted context omitted.

So can any human-based sources. A healthy learning experience with AI takes discipline, just as it does with humans.

That is nonsense. A competent human will provide you with an explanation that is based on logic. LLMs just put together words that are statistically most likely to be used together given the current input. That is why you can get hallucinations with LLMs but not with humans.

This is incredibly meta
Post reply on HN