Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vibe Debugging: Enterprises' Up and Coming Nightmare

marketsaintefficient.substack.com

Post reply on HN
Print view View original

61–69 of 69 posts

Re: Vibe Debugging: Enterprises' Up and Coming Nightmare

#61

Post by Jach »

agentultra wrote: post #3

Wasn’t the point of vibe coding to write throwaway code you will never maintain or care about? Why do people think it means we can write enterprise applications without understanding the code/specifications?

The dirty secret of a huge amount of "production code", i.e. code that is in production right now servicing enterprise needs, is that it's written with the care and understanding of typical throwaway code. It's not a new problem. Ball of mud architectures are everywhere, prototypes becoming permanent are common, understanding is low (especially if the original author has left or switched teams), but teams muddle through with the super powers of being able to ship ongoing patch fixes, code review, automated testing, and sometimes even writing code with professional care and forming little oases of quality.

I'm mildly optimistic that vibe coding won't make the problems that much worse and can actually lift the standards of quality in some cases. In my own personal careless / throwaway bash scripts, I've been using AI for them more and more, but I also notice that it puts more care into them than I otherwise would have with respect to things like error checking, friendlier help and other options, progress echos, and comments. AI tools still have a ways to go on larger projects though, and hallucinations seem particularly bad when it comes to foreign library bindings...

Re: Vibe Debugging: Enterprises' Up and Coming Nightmare

#62

Post by tincholio »

juancn wrote: post #47
itsdrewmiller wrote: post #9

> I suspect best practices for "vibe coding" will end up like test-driven development: a proven method for writing better software that many engineers still choose to skip. I’d like to see the proof for TDD; last I heard it slowed development with only minor reliability improvements.

I think it works only for people with a certain thought pattern, the ones that like to think everything up-front before doing anything. Most people prefer to play around and make several crappy attempts and combine them until the whole is somewhat solved, then go over and polish it a little, and maybe then add tests and fix the behavior in place. For this last group, TDD it's jarring, unnatural and requires a lot of…

> the ones that like to think everything up-front before doing anything.

I don't think that's the case. If they were really thinking up-front, they'd be doing proper req analysis and design work, rather than interactively growing a ball of mud that "does the minimal thing to pass a test". To me, it seems like TDD is sold as this "foolproof" design / dev approach, which is anything but.

Re: Vibe Debugging: Enterprises' Up and Coming Nightmare

#63

Post by ramoz »

astrobe_ wrote: post #37

"Isn't it just temporary, and AI will get better?" There are 2 instances of the word "understand" in the first paragraph, 3 if you count the beginning of the second. In my book, "understanding" is a synonym for "intelligence" - the roots of the word are "read between the lines", where something else that just knowledge is, the ability to use and manipulate knowledge [1]. But the thing is, despite this tech being clas…

The underlying model architectures have issues that perpetuate as models get better. We're still using transformers and RL.

  * Optimized for task completion, with limited attention resources for global alignment (RL/RLAIF reward loops/hacking)
  * These systems run outside of chat now.  file systems, CLIs, DBs, browsers → real-world side‑effects that you cannot train for. Hallucination becomes a problem of contradiction in the real world, and alignment is something an agent will struggle with as it's optimized to complete tasks. Which is why you see things like databases being dropped today. 
  * These are baked-in problems, not even considering the the adversarial nuances of things like prompt injection.
As AI advances, so do these issues.

Maybe it's cliche from an AI safety perspective. But I can never get over https://en.wikipedia.org/wiki/Instrumental_convergence as we see micro instances of it in our day-to-day with today's agents. Again, an issue that has existed from the dawn of these types of models. https://www.youtube.com/watch?v=s5qqjyGiBdc&t=1853s

Re: Vibe Debugging: Enterprises' Up and Coming Nightmare

#64

Post by draw_down »

justonceokay wrote: post #52
itsdrewmiller wrote: post #9

> I suspect best practices for "vibe coding" will end up like test-driven development: a proven method for writing better software that many engineers still choose to skip. I’d like to see the proof for TDD; last I heard it slowed development with only minor reliability improvements.

I always love to compare Peter Norvig’s sudoku solver implementation [1] versus Ron Jeffries TDD approach [2]. Norvig starts with the theory building and creates a constraint solver in about 50 lines of code. Jeffries starts with TDD, assumes an implementation, has to change that implementation, therefore has to change the tests, and after a series of five blog posts kind of fizzles out on it. To me it just highlight…

[dead]

Re: Vibe Debugging: Enterprises' Up and Coming Nightmare

#66

Post by _heitoo »

juancn wrote: post #47
itsdrewmiller wrote: post #9

> I suspect best practices for "vibe coding" will end up like test-driven development: a proven method for writing better software that many engineers still choose to skip. I’d like to see the proof for TDD; last I heard it slowed development with only minor reliability improvements.

I think it works only for people with a certain thought pattern, the ones that like to think everything up-front before doing anything. Most people prefer to play around and make several crappy attempts and combine them until the whole is somewhat solved, then go over and polish it a little, and maybe then add tests and fix the behavior in place. For this last group, TDD it's jarring, unnatural and requires a lot of…

TDD does not require an upfront understanding of the solution. It’s primarily about verifying what your application (or module) does, not how it does it. In other words, tests should be written from the outside in.

The common struggle with TDD arises when people reduce it to writing glorified spell checkers. This usually goes hand in hand with the belief that unit tests must always check classes in isolation using excessive mocks—an approach that misses the real purpose of TDD.

It has less to do with thought patterns and more with simply misunderstanding the approach entirely because of clinging to the wrong dogmas they've heard somewhere.

Re: Vibe Debugging: Enterprises' Up and Coming Nightmare

#67

Post by MoreQARespect »

AstroBen wrote: post #45
dogleash wrote: post #31

Earlier quoted context omitted.

Your comment doesn’t distinguish between having a robust automated test suite and doing TDD. I’ll take your comment as testing is good and constraining your workflow to TDD is worthless.

TDD is really commonly misunderstood to be a testing strategy that helps reliability- it's not. It's supposed to guide your software design

It's pretty bad at this. It's much better used as a testing methodology than a design methodology.

It can provide high level guardrails confirming implementation correctness that are as indifferent to software design as possible (giving freedom to refactor).

Re: Vibe Debugging: Enterprises' Up and Coming Nightmare

#68

Post by chickenzzzzu »

0x696C6961 wrote: post #60
chickenzzzzu wrote: post #21

Earlier quoted context omitted.

I wrote a full fledged 3D exporter in 6 days without a single test case. Used by thousands of people every day. How did I test and debug? Run my code and printf.

Try working on a project with lots of contributors.

The whole point is that it shouldn't happen. You should consume the thing I produce.

Re: Vibe Debugging: Enterprises' Up and Coming Nightmare

#69

Post by bicx »

fallpeak wrote: post #48
bicx wrote: post #8

I have a basic rule: understand and agree with every line of code in a PR I’m responsible for generating (manually or via agent). This simple act prevents 99% of AI-related bullshit code.

Sure. But, and I will die on this hill, it's not vibe coding if you're looking at and understanding the code.

Oh I agree. So many people lump all AI coding under the same "vibe code" definition, so I am constantly trying to differentiate.
Post reply on HN