Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The problem with "vibe coding"

dylanbeattie.net

Post reply on HN
Print view View original

81–90 of 154 posts

Re: The problem with "vibe coding"

#81

Post by alganet »

caspper69 wrote: post #31
alganet wrote: post #14

Sometimes I want to cook a home meal. If I wanted to open a large world class fast food chain, I wouldn't be cooking home meals then. That would be silly. Copilot can help me cook the equivalent of a McDonalds special in terms of software. It's good, I think McDonalds is delicious. But it cannot help me cook a home meal software. It will insist that my home made fries must go in a little red box, and my fish sandwich…

I am in the middle of my third AI assisted project. I disagree ~90%. If you prompt an LLM like an architect and feed it code rather than expecting it to write your code, both ChatGPT 4o and Claude 3.7 Sonnet do a great job. Do they mess up? Regularly. But the key is to guide the LLM and not let the LLM guide you, otherwise you'll end up in purgatory. It takes some time to get used to what types of prompts work. Remem…

> Be specific, be firm, tell it what to review first, what's important and what is not [...] Detail exactly how you want something to fit together [...]

You mean like in a fast food chain?

I know how to use it. All of that was already implied in my original comment. Sometimes though, I want to cook without a rigid mindset.

Re: The problem with "vibe coding"

#82

Post by cess11 »

asadotzler wrote: post #38

The real takeaway here is that programs you whip up for your own use, by any means, are not the same things as maintainable products you can deliver to customers. Vibe coding, or VB tricks and hacks 25 years ago, or whatever, sure, do it if that works for you, but that's not a product you can maintain for a customer base. It's a program, not a product.

I think "maintainable" is carrying a lot of weight in there.

A lot of profitable or otherwise successful software has been built by people who reasonably couldn't be called software engineers or computer scientists or whatever academic title. With Excel, Access, VB, Delphi, Wordpress. I'm sure there's an astrologer somewhere that made OK money from a hack in Delphi or VB for divining the stars on a computer.

It shouldn't be called "vibe" coding, it seems more like glue coding, which some people have been doing and sometimes made careers out of for a very long time. Wordpress was for a long time the big thing in this area, it allowed (probably) millions of people to call themselves web designers and web developers without actually becoming competent in software design.

Many corporations selling software have developers that aren't much better than that. People that might do 'SELECT * FROM table WHERE foreign_id = ?' to check whether there exists any such row, or create full copies of data to add versioning to some product instead of keeping it as metadata, or generate and store hundreds or thousands of gigabytes of dumb data representing business processes (like bookings and such) for the coming century instead of inferring it from rules on the fly. The corporations where I've seen this are generally considered fine and good software companies by customers and competitors alike.

A new RAD, a new Wordpress, a new MICROS~1 Access or a new LotusNotes isn't particularly revolutionary. I suspect the 'hype' is more about getting people in IT to accept the technology as such and not revolt against the broader applications against other, non-technical, people; for control, disciplining, harassment, surveillance, war or whatever nefarious, tyrannical purpose.

Re: The problem with "vibe coding"

#84

Post by int_19h »

motorest wrote: post #61
alganet wrote: post #14

Sometimes I want to cook a home meal. If I wanted to open a large world class fast food chain, I wouldn't be cooking home meals then. That would be silly. Copilot can help me cook the equivalent of a McDonalds special in terms of software. It's good, I think McDonalds is delicious. But it cannot help me cook a home meal software. It will insist that my home made fries must go in a little red box, and my fish sandwich…

> Copilot can help me cook the equivalent of a McDonalds special in terms of software. It's good, I think McDonalds is delicious. No. The likes of Copilot help you cook the meal you'd like, how you'd like it. In some cases it forgets to crack open eggs, in other cases it cooks a meal far better than whatever you'd be able to pull together in your home kitchen. And does it in seconds. The "does it in seconds" is the k…

Cook a meal, yes. "How you'd like it", no. You have to use the languages that have a large enough presence in its training set that it knows them well, and ditto for frameworks and libraries. Once you step out of that zone, quality rapidly plummets.

"Does it in seconds" is kinda true if you ignore the "supervised by you" part. That part - reading through the generated diffs to catch (sometimes very subtle) errors - can take a while.

And sure, you'd have the same problem with the intern. But the intern learns from you. The LLM will make the same mistakes again in the same circumstances. Yes, I know they do have memory now, but it my experience that RAG-based stuff is not particularly reliable. I suspect this might improve once hardware is fast enough that you can use a (smallish) helper LLM for RAG with reasonable perf, but we aren't quite there yet.

Re: The problem with "vibe coding"

#85

Post by pjs_ »

Right now there are thousands of apps getting built with AI that are going to work surprisingly well, will get embedded into all sorts of fundamental processes — and will start to perform unacceptably badly in two or three years. God knows what we do with this information but it seems clear we are setting up for a tsunami of horrible performance regressions

Re: The problem with "vibe coding"

#86

Post by int_19h »

motorest wrote: post #62
alganet wrote: post #56

Earlier quoted context omitted.

It's kind of true story actually. I found one of my license texts for a hobby project[0] in a Microsoft product[1]. I have no idea how they use it. Copilot is very poor at understanding how the DSL arrangement of the project works or how to assemble validator trees. Before Copilot many other "enterprise" stuff like JetBrains IDEs had problems auto-completing it. But it was always good enough for me, for my own things…

> No, copilot is not endlessly variable. It works best with typed languages and an enterprise mindset. I don't think your assumption is true. The key factor is the corpus used to train it, and the context you fed it. I already experienced Copilot fumbling references to methods of a simple Java class, whereas it pulled off thinks like ARM templates flawlessly. You need to understand that internally LLMs do probability…

Right, but that's exactly what static typing does in most cases - it provides model with more explicit context for what it's doing.

For the same reason, they tend to handle XML better than JSON - sure, both are trees, but XML is has redundancy, and that redundancy helps keep the model on the rails, so to speak.

I actually wonder if the perfect LLM language would be something a lot more like COBOL - not in a sense of being similarly high level, but rather verbosity. And perhaps also being closer to natural English, which is, after all, still a lot of its training set, especially for reasoning stuff. For query languages they seem to like SQL the most of all the things I've tried, and I strongly suspect that it's the same underlying cause.

Of course, a new language designed like that would have the fundamental problem that there's no existing corpus in such a language. Then again, if it is also designed such that one can reliably convert e.g. from Java to that language, then perhaps we can still pull that off.

Re: The problem with "vibe coding"

#87

Post by int_19h »

__MatrixMan__ wrote: post #59
mvdtnz wrote: post #16

Earlier quoted context omitted.

Yeah this is how I interpret it. Type text into the chatbot, copy and paste the outputted code. If it fails to compile, paste the error message and get the resulting code. Similar if it fails at runtime. Supply the LLM with additional code files where necessary but don't really look at the code, just copy and paste in, copy and paste out.

The vibes are much better if you just tell it to run a certain command, then it can see the error message and you don't have to paste back and forth so much. You can go make a cup of tea and it'll just iterate on changing files and running that command until it does the desired thing.

This is what AI-oriented IDEs like Cursor streamline - the LM gets immediate feedback from things like code completion, linter, build errors, test failures etc, and can immediately iterate without you having to do anything about it (sometimes overly aggressively even tbh).

Re: The problem with "vibe coding"

#88

Post by blablabla123 »

pjs_ wrote: post #85

Right now there are thousands of apps getting built with AI that are going to work surprisingly well, will get embedded into all sorts of fundamental processes — and will start to perform unacceptably badly in two or three years. God knows what we do with this information but it seems clear we are setting up for a tsunami of horrible performance regressions

Actually even in the years before AI code generation started, I noticed an increase in code bases that seem very well architected and engineered. But once you start using the code for a week you notice it's full of bugs and the consistent architecture is just make believe. Surely this problem is getting way worse with generated code that is optimized to read like it's very well engineered

Re: The problem with "vibe coding"

#89

Post by int_19h »

0xCE0 wrote: post #47

The real problem with "vibe coding" (or any coding, or with any product what so ever) is that the end user probably is not a programmer, and therefore cannot have professional judgement on the quality of the implementation. The only thing typical end user care is the perceived behavior, not the implementation details. As it is easier and cheaper to do anything, the result is low-quality products. This of course serve…

It depends. Market has demonstrated time and again that premium quality is a rather niche product. Is the niche large enough for all the existing "real professionals" to fit into it?

Re: The problem with "vibe coding"

#90

Post by vaylian »

kmoser wrote: post #12
tjpnz wrote: post #3

Earlier quoted context omitted.

How does the inexperienced developer become experienced under said circumstances?

You can examine the code at any time, ask for an explanation, and even make changes yourself. In other words, you get out of it exactly what you put into it. If you choose to let the AI do all the coding and never dig into the actual code, you'll learn more about how to communicate with an LLM than you will about how to write code.

> ask for an explanation

That is risky. The AI might just hallucinate an explanation.

Post reply on HN