Live data from Hacker News

The great software quality collapse or, how we normalized catastrophe

techtrenches.substack.com

91–100 of 260 posts

Re: The great software quality collapse or, how we normalized catastrophe

#91
> The Replit incident in July 2025 crystallized the danger:

> Jason Lemkin explicitly instructed the AI: "NO CHANGES without permission"

> The AI encountered what looked like empty database queries

> It "panicked" (its own words) and executed destructive commands

> Deleted the entire SaaStr production database (1,206 executives, 1,196 companies)

> Fabricated 4,000 fake user profiles to cover up the deletion

> Lied that recovery was "impossible" (it wasn't)

> The AI later admitted: "This was a catastrophic failure on my part. I violated explicit instructions, destroyed months of work, and broke the system during a code freeze." Source: The Register

This is a bit of a half truth, at least how it's represented. This wasn't Saastr's core database, it was a database of contacts which SaaStr had uploaded for the specific purpose of developing this product.

Lemkin has himself said that the product was an experiment to see how far he could get with a vibe coding tool alone (ie not using a separate database SaaS like Supabase, Firebase) which would have made such an incident much harder to do.

The error was / is completely recoverable through Replit's existing tools, even if Replit's AI initially said it wasn't.

It's one of those where you know the details of one specific thing in the article that they call out, which makes it difficult to completely trust the rest.

Re: The great software quality collapse or, how we normalized catastrophe

#93

I don't want to sound too dismissive, but all these arguments have been brought up time and again. The move from assembler to high level languages. The introduction of OOP. Component architecture / COM / CORBA / etc. The development of the web browser. The introduction of Java. 2018 isn't "the start of the decline", it's just another data point on a line that leads from, y'know, Elite 8-bit on a single tape in a few…

I think another part of this, is that Tech is perhaps the only industry that hasn't quite gotten over itself yet. Writing code is artistic the same way plumbing is artistic. Writing code is artistic the same way home wiring is artistic. Writing code is artistic the same way HVAC is artistic. Which is to say, yes, there is satisfaction to be had, but companies don't care as long as it gets the job done without too man…

I don't see working for most of my employers as "artistic."

I do see it as more of a craft than a typical trade. There are just too many ways to do things to compare it to e.g. an electrician. Our industry does not have (for better or for worse) a "code" like the building trades or even any mandated way to do things, and any attempts to impose (cough cough Ada, etc.) that have been met with outright defiance and contempt in fact.

When I'm working on my own projects -- it's a mix of both. It's a more creative endeavour.

Re: The great software quality collapse or, how we normalized catastrophe

#94

I never read about Replit earlier this year, but I am now glad that I did. This article summarizes it in a way that is outrageously hilarious: The Replit incident in July 2025 crystallized the danger: 1. Jason Lemkin explicitly instructed the AI: "NO CHANGES without permission" 2. The AI encountered what looked like empty database queries 3. It "panicked" (its own words) and executed destructive commands 4. Deleted t…

using phrasing like admitted anthropomorphizes it way too much.

It boggles my mind that folks continue to act that AIs are reliable narrators of their internal state despite all evidence to the contrary.

The best I can figure is that too many people’s salaries depend on the matrix multiplier made of sand somehow manifesting a soul any day now.

Re: The great software quality collapse or, how we normalized catastrophe

#95
The problem is one of incentives: In the software industry, we frequently prioritize building new features over everything else. Because software is (generally) unregulated, customers have little recourse when that new feature malfunctions.

The solution, which is probably unappealing to most people here, is regulation. Other industries have to work within regulations: I once bought a car with a problem that the manufacturer could not fix, so they had to buy it back. My appliances have to confirm to energy efficiency regulations. The regulations fix the incentives.

Perhaps a combination of regulations that block new features while there are outstanding bugs, require refunds if those bugs aren't fixed (or payouts for add-supported software,) and energy efficiency will go a long way.

(And, before you laugh at me, remember that if we can't self-regulate, regulations will be imposed on us.)

Re: The great software quality collapse or, how we normalized catastrophe

#96
post #68
post #28

Unrelated but my current AI text flag is the use of "It's not X. It's Y." It's become so repetitive recently. Examples from this post alone: 1. "This isn't about AI. The quality crisis started years before ChatGPT existed." 2. "The degradation isn't gradual—it's exponential." 3. "These aren't feature requirements. They're memory leaks that nobody bothered to fix." 4. "This wasn't sophisticated. This was Computer Scie…

I've been seeing this pattern of text crop up in many places. On LinkedIn, much of my feed is filled with posts of short sentences that follow this exact pattern. What's more, even the replies are obviously AI-generated.

I agree. LinkedIn is completely dominated by AI slop replies to AI slop posts these days.

It's almost worse than Twitter.

Re: The great software quality collapse or, how we normalized catastrophe

#97
post #73
post #62

Earlier quoted context omitted.

If you haven't noticed a dramatic decline in average software quality, you're not paying attention or willfully ignoring it. The article is right. This is partly related to the explosion of new developers entering the industry, coupled with the classic "move fast and break things" mentality, and further exacerbated by the current "AI" wave. Junior developers don't have a clear path at becoming senior developers anymo…

Is this measureable? Like code readability scores on the GitHub corpus over time?

Maybe. Personally I've observed an increase of major system and security failures in the past 5 years, especially failures that impact very large tech companies. You could measure these public failures and see if frequency or impact has increased.

Re: The great software quality collapse or, how we normalized catastrophe

#98
I've thought a lot about why software doesn't converge on higher quality and I don't know if I've ever come to a satisfying conclusion.

The simple statement is that "Quality isn't what sells" and I think there's some truth to that, but I also think quality can sell and theoretically if all other factors are equal higher quality will sell better.

What I've had a tough time pinning down precisely is I think there's almost an ecosystem balancing act occurring between quality, effort expended, and skill required. Particularly in American culture I think one of our economic 'super powers' is that we sort of have an innate sense for how much something can be half-assed while still being mostly effective. We're great at finding the solution that gets 80% of the job done for 20% of the effort while requiring less skill.

I think you can see that through all sorts of disciplines, and the result is a world of kind of mid quality things that is actually quite efficient and resilient.

Where that goes wrong is when the complexity and problems start to compound in a way that makes any effort substantially more wasteful. It reminds me of how my game jam experiences go: for the first 60% I'm writing code at a tremendous velocity because I'm just putting down whatever works but by the end the spaghetti nightmare of the code reduces my velocity dramatically. Even when that occurs though the ecosystem nature of complexity means that your competitors are likely mired in similar complexity, and the effort to get out may be large enough and require enough talent that you don't you can change.

To improve quality I think it's not enough to just say "People should care about quality more" I think you have to fundamentally target aspects of the ecosystem, like changing the course of a river by moving a few rocks. A good example I think has adjusted the behavior of the ecosystem is Rust: it makes certain things much easier than before and slowly the complex bug-mired world of software is improving just a little bit because of it.

Re: The great software quality collapse or, how we normalized catastrophe

#99
post #52

Just false nostalgia memory. 20 years ago things werent any better. Software didn't consume gigabytes of ram because there was no gigabytes of ram to consume.

20 years ago, you could consistently pick up a phone, get a dial tone, and then call a human to solve a problem. Sure, plenty of stuff didn't work. The issue is we're not bothering to make anything that does. It's a clear cultural shift and all of this "nothing ever worked so why try" talk here is not what I remember. We're in a stochastic era of scale where individual experiences do not matter. AI turning computers…

I think you've got your time ranges wrong. Almost exactly 20 years ago I worked for a company that did dashboards for analyzing and managing call centre traffic. The most important metric wasn't "how happy customers are" or "how many calls served" it was: "how many calls did we avoid having a human agent get involved for?" The metric of success was the degree to which they could avoid using expensive human labour and get you through the automated call tree.

Companies offered such (expensive) services because they had no choice. They made every effort to divert and divest from such activities. Google and companies like them made filthy profits because they figured out the secret sauce to scaling a business without the involvement of humans, but people were trying it for literally decades with mixed results (usually enraged customers).

Stupid red tape, paperwork, and call centre frustrations were the order of the day 20-30 years ago.

Re: The great software quality collapse or, how we normalized catastrophe

#100

Earlier quoted context omitted.

I think another part of this, is that Tech is perhaps the only industry that hasn't quite gotten over itself yet. Writing code is artistic the same way plumbing is artistic. Writing code is artistic the same way home wiring is artistic. Writing code is artistic the same way HVAC is artistic. Which is to say, yes, there is satisfaction to be had, but companies don't care as long as it gets the job done without too man…

I don't see working for most of my employers as "artistic." I do see it as more of a craft than a typical trade. There are just too many ways to do things to compare it to e.g. an electrician. Our industry does not have (for better or for worse) a "code" like the building trades or even any mandated way to do things, and any attempts to impose (cough cough Ada, etc.) that have been met with outright defiance and cont…

I don't see this as really disproving my point.

If we look at most trades historically:

- Electricians in the 1920s? Infinite ways to do things. DC vs AC wars. Knob-and-tube vs conduit vs armored cable. Every electrician had their own "creative" approach to grounding. Regional variations, personal styles, competing philosophies. Almost all of those other ways are gone now. Early attempts to impose codes on electricians and electrical devices were disasters.

- Plumbers in the 1920s? Lead vs iron vs clay pipes. Every plumber had their own joint compound recipe. Creative interpretations of venting. Artistic trap designs. Now? Why does even installing a basic pipe require a license? We found out after enough cholera outbreaks, methane explosions, and backed-up city sewer systems.

- Doctors in the 1920s? Bloodletting, mercury treatments, lobotomies, and their own "creative" surgical techniques. They violently resisted the American Medical Association, licensing requirements, and standardized practices. The guy who suggested handwashing was literally driven insane by his colleagues.

We're early, not special. And just like society eventually had enough of amateur electricians, plumbers, and doctors in the 1920s, they'll have enough of us too. Give it 40 years, and they'll look at our data breaches and system designs the same way we look at exposed electrical wiring, obviously insane no matter the amount of warnings.

Post reply on HN