Live data from Hacker News

Beliefs that are true for regular software but false when applied to AI

boydkane.com

51–60 of 461 posts

Re: Beliefs that are true for regular software but false when applied to AI

#51

> Because eventually we’ll iron out all the bugs so the AIs will get more reliable over time Honestly this feels like a true statement to me. It's obviously a new technology, but so much of the "non-deterministic === unusable" HN sentiment seems to ignore the last two years where LLMs have become 10x as reliable as the initial models.

Right away my mind went to "well, are people more reliable than they used to be?" and I'm not sure they are. Of course LLMs aren't people, but an AGI might behave like a person.

Older people are generally more reliable than younger people.

Re: Beliefs that are true for regular software but false when applied to AI

#52
post #3

But this is why using the AI in the production of (almost) deterministic systems makes so much sense, including saving on execution costs. ISTR someone else round here observing how much more effective it is to ask these things to write short scripts that perform a task than doing the task themselves, and this is my experience as well. If/when AI actually gets much better it will be the boss that has the problem. Thi…

>that baffles me about the managerial globalists

It's no different from those on HN that yell loudly that unions for programmers are the worst idea ever... "it will never be me" is all they can think, then they are protesting in the streets when it is them, but only after the hypocrisy of mocking those in the street protesting today.

Re: Beliefs that are true for regular software but false when applied to AI

#53

Earlier quoted context omitted.

Neither, you’re reading it wrong. Think of it as codebases getting more reliable over time as they accumulate fixes and tests. (As opposed to, say, writing code in NodeJS versus C++)

Age of Code does not automatically equal quality of code, ever. Good code is maintained by good developers. A lot of bad code is pushed out by management, and other situations, or just bad devs. This is a can of worms you're talking your way into.

You're using different words - the top comment only mentioned the reliability of the software, which is only tangentially related to the quality, goodness, or badness of the code used to write it.

Old software is typically more reliable, not because the developers were better or the software engineering targeted a higher reliability metric, but because it's been tested in the real world for years. Even more so if you consider a known bug to be "reliable" behavior: "Sure, it crashes when you enter an apostrophe in the name field, but everyone knows that, there's a sticky note taped to the receptionist's monitor so the new girl doesn't forget."

Maybe the new software has a more comprehensive automated testing framework - maybe it simply has tests, where the old software had none - but regardless of how accurate you make your mock objects, decades of end-to-end testing in the real world is hard to replace.

As an industrial controls engineer, when I walk up to a machine that's 30 years old but isn't working anymore, I'm looking for failed mechanical components. Some switch is worn out, a cable got crushed, a bearing is failing...it's not the code's fault. It's not even the CMOS battery failing and dropping memory this time, because we've had that problem 4 times already, we recognize it and have a procedure to prevent it happening again. The code didn't change spontaneously, it's solved the business problem for decades... Conversely, when I walk up to a newly commissioned machine that's only been on the floor for a month, the problem is probably something that hasn't ever been tried before and was missed in the test procedure.

Re: Beliefs that are true for regular software but false when applied to AI

#54
post #33

Earlier quoted context omitted.

Does that really matter? He is trying to lax the general public perception around AIs shortcomings. He's giving AI a break, at the expense of regular developers. This is wrong on two fronts: First, because many people foresaw the AI shortcomings and warned about them. This "we can't fix a bug like in regular software" theatre hides the fact that we can design better benchmarks, or accountability frameworks. Again, lo…

I don't think he's doing that at all. The article is pointing out to non-technical people how AI is different than traditional software. I'm not sure how you think it's giving AI a break, as it's pointing out that it is essentially impossible to reason about. And it's not at the expense of regular developers because it's showing how regular software development is different than this. It makes two buckets, and puts A…

He is. Maybe he's just running with the pack, but that doesn't matter either.

The fact is, we kind of know how to prevent problems in AI systems:

- Good benchmarks. People said several times that LLMs display erratic behavior that could be prevented. Instead of adjusting the benchmarks (which would slow down development), they ignored the issues.

- Accountability frameworks. Who is responsible when an AI fails? How the company responsible for the model is going to make up for it? That was a demand from the very beginning. There are no such accountability systems in place. It's a clown fiesta.

- Slowing down. If you have a buggy product, you don't scale it. First, you try to understand the problem. This was the opposite of what happened, and at the time, they lied that scaling would solve the issues (when in fact many people knew for a fact that scaling wouldn't solve shit).

Yes, it's kind of different. But it's a different we already know. Stop pushing this idea that this stuff is completely new.

Re: Beliefs that are true for regular software but false when applied to AI

#55
post #45

Aim bosses at this article in The Economist.[1] If your boss doesn't read The Economist, you need to escalate to a level that does. [1] https://www.economist.com/leaders/2025/09/25/how-to-stop-ais...

https://archive.is/R0RJB

Re: Beliefs that are true for regular software but false when applied to AI

#56
post #3

But this is why using the AI in the production of (almost) deterministic systems makes so much sense, including saving on execution costs. ISTR someone else round here observing how much more effective it is to ask these things to write short scripts that perform a task than doing the task themselves, and this is my experience as well. If/when AI actually gets much better it will be the boss that has the problem. Thi…

A bunch of short scripts doesn't easily lead to a large-scale robust software platform.

I guess if managers get canned, it'll be just marketing types left?

Re: Beliefs that are true for regular software but false when applied to AI

#57

> AIs will get more reliable over time, like old software is more reliable than new software. :) Was that a humam Freudian slip, or artificial one? Yes, old software is often more reliable than new.

Perhaps better rephrased as "software that's been running for a (long) while is more reliable than software that only started running recently."

Re: Beliefs that are true for regular software but false when applied to AI

#58

Earlier quoted context omitted.

Neither, you’re reading it wrong. Think of it as codebases getting more reliable over time as they accumulate fixes and tests. (As opposed to, say, writing code in NodeJS versus C++)

Age of Code does not automatically equal quality of code, ever. Good code is maintained by good developers. A lot of bad code is pushed out by management, and other situations, or just bad devs. This is a can of worms you're talking your way into.

I think we all agree that the quality of the code itself goes down over time. I think the point that is being made is that the quality of the final product goes up over time.

E.g. you might fix a bug by adding a hacky workaround in the code; better product, worse code.

Re: Beliefs that are true for regular software but false when applied to AI

#59
> It’s entirely possible that some dangerous capability is hidden in ChatGPT, but nobody’s figured out the right prompt just yet.

This sounds a little dramatic. The capabilities of ChatGPT are known. It generates text and images. The qualities of the content of the generated text and images is not fully known.

Re: Beliefs that are true for regular software but false when applied to AI

#60

> It’s entirely possible that some dangerous capability is hidden in ChatGPT, but nobody’s figured out the right prompt just yet. This sounds a little dramatic. The capabilities of ChatGPT are known. It generates text and images. The qualities of the content of the generated text and images is not fully known.

Also, there's a reason AI Red Teaming is now an ask that is getting line item funding from C-Suites.
Post reply on HN