Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vibe coding creates a bus factor of zero

mindflash.org

Post reply on HN
Print view View original

31–40 of 142 posts

Re: Vibe coding creates a bus factor of zero

#31

Post by brandall10 »

p1necone wrote: post #16

If you're using llms to shit out large swathes of unreviewed code you're doing it wrong and your project is indeed doomed to become unmaintainable the minute it goes down a wrong path architecturally, or you get a bug with complex causes or whatever. Where llms excel is in situations like: * I have that I want to apply to - bam, half a days work done in 2 minutes. * I want to set up test data and the bones of unit te…

There's also a middle ground, where you have the AI generate PR reviews and then review them manually. So that 2 minutes of code you spat out (really more like 5-10 using CC) takes another hour or three to review, and maybe 5 to 10 more commits before it's merged in.

I've done this successfully on multiple projects in the 10-20k LOC, ~100 file area - fully LLM generated w/ tons of my personal feedback - and it works just fine. 4/5 features I implement it gets pretty close to nailing from the spec I provide and the work is largely refactoring. But the times it doesn't get it right, it is a slog that could eat the better part of a day. On the whole though it probably is a 3-5x speedup.

I'm a little less confident about doing this on projects that are much bigger... then breaking things up into modules begins to look more attractive.

Re: Vibe coding creates a bus factor of zero

#32

Post by layer8 »

p1necone wrote: post #16

If you're using llms to shit out large swathes of unreviewed code you're doing it wrong and your project is indeed doomed to become unmaintainable the minute it goes down a wrong path architecturally, or you get a bug with complex causes or whatever. Where llms excel is in situations like: * I have that I want to apply to - bam, half a days work done in 2 minutes. * I want to set up test data and the bones of unit te…

> If you're using llms to shit out large swathes of unreviewed code you're doing it wrong

> bam, x days work done in 2 minutes

This is a bit of a misrepresentation, since those two minutes don’t account for the reviewing time needed (nor prorperly, which vastly exceeds that time. Otherwise you end up in the situation of “doing it wrong” described in your first paragraph.

Re: Vibe coding creates a bus factor of zero

#34

Post by spankalee »

There is a really important point here, and it's critical to be aware of it, but we're really just at the beginning of these tools and workflows and these issues can be solved, IMO, possibly better than with humans.

I've been trying to use LLMs to help code more to mixed success, honestly, but it's clear that they're very good at some things, and pretty bad at others. One of the things they good at obviously is producing lots of text, two important others are that they can be very persistent and thorough.

Producing a lot of code can be a liability, but an LLM won't get annoyed at you if you ask it for thorough comments and updates to docs, READMEs, and ADRs. It'll "happily" document what it just did and "why" - to the degree of accuracy that they're able, of course.

So it's conceivable to me at least, that with the right guidance and structure an LLM-generated codebase might be easier to come into cold years later, for both humans and future LLMs, because it could have excellent documentation.

Re: Vibe coding creates a bus factor of zero

#35

Post by zzzeek »

just not even ten years ago the discussion here was all about "software engineering" trying to be more legitimized as a formal engineering practice, if there should be licensing, if there should be certifications, lots of threads about formal methods to prove algorithms work, and look where we are now. Arguing if humans should even care look at the code we are producing and shipping. crazy shit man

Re: Vibe coding creates a bus factor of zero

#36

Post by jsjsksksksksk »

spankalee wrote: post #34

There is a really important point here, and it's critical to be aware of it, but we're really just at the beginning of these tools and workflows and these issues can be solved, IMO, possibly better than with humans. I've been trying to use LLMs to help code more to mixed success, honestly, but it's clear that they're very good at some things, and pretty bad at others. One of the things they good at obviously is produ…

There is a really important point here, and it's critical to be aware of it, but we're really at the end of these tools and workflows and these issues can't be solved.

Re: Vibe coding creates a bus factor of zero

#37

Post by merlincorey »

p1necone wrote: post #16

If you're using llms to shit out large swathes of unreviewed code you're doing it wrong and your project is indeed doomed to become unmaintainable the minute it goes down a wrong path architecturally, or you get a bug with complex causes or whatever. Where llms excel is in situations like: * I have that I want to apply to - bam, half a days work done in 2 minutes. * I want to set up test data and the bones of unit te…

> * I want to apply some repetitive change across a large codebase that's just too complicated for a clever regex, bam work you literally would have never bothered to do before done in 2 minutes.

You would naively think that, as did I, but I've tested it against several big name models and they are all eventually "lazy", sometimes make unrelated changes, and worse as the context fills up.

On a small toy example they will do it flawlessly, but as you scale up to more and more code that requires repetitive changes the errors compound.

Agentic loops help the situation, but now you aren't getting it done in 2 minutes because you have to review to find out it wasn't done and then tell it to do it again N times until it's done.

Having the LLM write a program to make the changes is much more reliable.

Re: Vibe coding creates a bus factor of zero

#38

Post by windows2020 »

ants_everywhere wrote: post #33

LLMs make it vastly easier to work on unfamiliar code bases

A team unfamiliar with a code base demoed asking questions to an LLM about it. The answers genuinely excited some. But anyone who had spent a short time in the code base knew the answers were wrong. Oh well.

Re: Vibe coding creates a bus factor of zero

#39

Post by trod1234 »

s1mplicissimus wrote: post #29

I agree with the premise and the conclusion, but over almost 20 years of writing, adapting and delivering software I've more than once been in exactly the same situation. Noone to ask, the only person even vaguely familiar with software development left half a year ago. Half of the processes have changed since the software was written, and the people who owned them have left, too. So while I agree that LLMs will acce…

The author though neglects what a bus factor of 0 means in real terms and how it gets there, aside from the description of the definition upfront where all knowledge lost.

A company acceptable with a bus factor of zero is a company that is not willing to pay the economic advantage to the expertise required to do the work.

The economic demand, of humanity competing with AI is zero because AI does things its good at with an order of magnitude difference in cost, and the deception and lies surrounding the marketing coupled with communications channel jamming lead to predictable outcomes. What happens when that demand and thus economic benefit go to zero? Any investment in the knowledge in the first place has no return. No one goes into it, no one learns, and that's quite dangerous in economies based in money-printing.

So while there may not be a problem right now, there will no doubt be problems in the next proverbial quarter. Career development pipelines are years in the making. They are sequential pipelines. Zero into a sequential pipeline is zero out with the time-lag between the two (which is years).

~2 years without economic incentive is when you lose your best and brightest. From there is a slow march to a 10 year deadline after which catastrophic losses occur.

I had a chance to have a interesting discussion with a local community college Dean. Apparently they have had to lower the number of Computer Science related program sections because of lack of demand. In intro courses, for 18 sections there were 13 people who had declared for the major, most students when asked citing AI concerns and lack of career development pipeline.

What happens when you have no expertise that you can pay at any price to fix the processes involved because you put them all out of business using what amounts to a semi-thinking non-sentient slave.

Without supply, there can be no demand. Where supply becomes infinite because of external parties, there can be no demand. There can be need, but demand is not need.

So this all started in 2022. Best and brightest are re-skilling. There's a glut of lower competency talent too. Bad things happen when you mess with the foundations of economics, and they happen on a time lag, where you can't react fast enough after-the-fact.

What will it take? At some point there will be a crisis where they will have to treat it as triage on the battlefield. The people in charge didn't want to hear about this when it could have made a difference.

Re: Vibe coding creates a bus factor of zero

#40

Post by siliconc0w »

The problem is that our brains really don't like expending calories on information we don't repeatedly use so the further you get from something, the less you understand it or remember it.

So even if you aren't even vibe coding and are trying to review every change, your skills are atrophying. We see this all the time as engineers enter management, they become super competent at the new skills the role requires but quickly become pretty useless at solving technical problems.

It's similar to why it's so hard to go from level2 to level5 in driving automation. We're not really designed to be partially involved in a process - we quickly loose attention, become lazy, and blindly trust the machine. Which is maybe fine if the machine is 100% reliable but we know that isn't the case.

Post reply on HN