Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beliefs that are true for regular software but false when applied to AI

boydkane.com

Post reply on HN
Print view View original

61–70 of 461 posts

Re: Beliefs that are true for regular software but false when applied to AI

#61

Post by hn_acc1 »

pixl97 wrote: post #52
fidotron wrote: post #3

But this is why using the AI in the production of (almost) deterministic systems makes so much sense, including saving on execution costs. ISTR someone else round here observing how much more effective it is to ask these things to write short scripts that perform a task than doing the task themselves, and this is my experience as well. If/when AI actually gets much better it will be the boss that has the problem. Thi…

>that baffles me about the managerial globalists It's no different from those on HN that yell loudly that unions for programmers are the worst idea ever... "it will never be me" is all they can think, then they are protesting in the streets when it is them, but only after the hypocrisy of mocking those in the street protesting today.

Agreed. My dad was raised strongly fundamentalist, and in North America, that included (back then) strongly resisting unions. In hindsight, I've come to realize that my parent's weren't maybe even of average intelligence, and definitely of above-average gullibility.

Unionized software engineers would solve a lot of the "we always work 80 hour weeks for 2 months at the end of a release cycle" problems, the "you're too old, you're fired" issues, the "new hires seems to always make more than the 5/10+ year veterans", etc. Sure, you wouldn't have a few getting super rich, but it would also make it a lot easier for "unionized" action against companies like Meta, Google, Oracle, etc. Right now, the employers hold like 100x the power of the employees in tech. Just look at how much any kind of resistance to fascism has dwindled after FAANG had another round of layoffs..

Re: Beliefs that are true for regular software but false when applied to AI

#62

Post by Animats »

Traubenfuchs wrote: post #55
Animats wrote: post #45

Aim bosses at this article in The Economist.[1] If your boss doesn't read The Economist, you need to escalate to a level that does. [1] https://www.economist.com/leaders/2025/09/25/how-to-stop-ais...

https://archive.is/R0RJB

Management summary, from The Economist article:

"The worst effects of this flaw are reserved for those who create what is known as the “lethal trifecta”. If a company, eager to offer a powerful AI assistant to its employees, gives an LLM access to un-trusted data, the ability to read valuable secrets and the ability to communicate with the outside world at the same time, then trouble is sure to follow. And avoiding this is not just a matter for AI engineers. Ordinary users, too, need to learn how to use AI safely, because installing the wrong combination of apps can generate the trifecta accidentally."

Re: Beliefs that are true for regular software but false when applied to AI

#63

Post by nlawalker »

meonkeys wrote: post #46
nlawalker wrote: post #28

Where did "can't you just turn it off?" in the title come from? It doesn't appear anywhere in the actual title or the article, and I don't think it really aligns with its main assertions.

It shows up at https://boydkane.com under the link "Why your boss isn't worried about advanced AI". Must be some kind of sub-heading, but not part of the actual article / blog post. Presumably it's a phrase you might hear from a boss who sees AI as similar to (and as benign/known/deterministic as) most other software, per TFA

Ah, thanks for that!

>Presumably it's a phrase you might hear from a boss who sees AI as similar to (and as benign/known/deterministic as) most other software, per TFA

Yeah I get that, but I think that given the content of the article, "can't you just fix the code?" or the like would have been a better fit.

Re: Beliefs that are true for regular software but false when applied to AI

#64

Post by SkyBelow »

xutopia wrote: post #4

The most likely danger with AI is concentrated power, not that sentient AI will develop a dislike for us and use us as "batteries" like in the Matrix.

I agree.

Our best technology at current require teams of people to operate and entire legions to maintain. This leads to a sort of balance, one single person can never go too far down any path on their own unless they convince others to join/follow them. That doesn't make this a perfect guard, we've seen it go horribly wrong in the past, but, at least in theory, this provides a dampening factor. It requires a relatively large group to go far along any path, towards good or evil.

AI reduces this. How greatly it reduces this, if it reduces it to only a handful, to a single person, or even to 0 people (putting itself in charge), seems to not change the danger of this reduction.

Re: Beliefs that are true for regular software but false when applied to AI

#65

Post by kstrauser »

kazinator wrote: post #2

> AIs will get more reliable over time, like old software is more reliable than new software. :) Was that a humam Freudian slip, or artificial one? Yes, old software is often more reliable than new.

Holy survivorship bias, Batman.

If you think modern software is unreliable, let me introduce you to our friend, Rational Rose.

Re: Beliefs that are true for regular software but false when applied to AI

#66

Post by mrob »

xutopia wrote: post #4

The most likely danger with AI is concentrated power, not that sentient AI will develop a dislike for us and use us as "batteries" like in the Matrix.

Why does an AI need the ability to "dislike" to calculate that its goals are best accomplished without any living humans around to interfere? Superintelligence doesn't need emotions or consciousness to be dangerous.

Re: Beliefs that are true for regular software but false when applied to AI

#67

Post by kube-system »

giancarlostoro wrote: post #25
joomla199 wrote: post #23

Earlier quoted context omitted.

Neither, you’re reading it wrong. Think of it as codebases getting more reliable over time as they accumulate fixes and tests. (As opposed to, say, writing code in NodeJS versus C++)

Age of Code does not automatically equal quality of code, ever. Good code is maintained by good developers. A lot of bad code is pushed out by management, and other situations, or just bad devs. This is a can of worms you're talking your way into.

The author didn't mean that an older commit date on a file makes code better.

The author is talking about the maturity of a project. Likewise, as AI technologies become more mature we will have more tools to use them in a safer and more reliable way.

Re: Beliefs that are true for regular software but false when applied to AI

#68

Post by conception »

toomuchtodo wrote: post #19
ljlolel wrote: post #15

Earlier quoted context omitted.

CEOs (even most VCs) are labor too

Labor competes for compensation, CEOs compete for status (above a certain enterprise size, admittedly). Show me a CEO willingly stepping down to be replaced by generative AI. Jamie Dimon will be so bold to say AI will bring about a 3 day week (because it grabs headlines [1]) but he isn't going to give up the status of running JPMC; it's all he has besides the wealth, which does not appear to be enough. The feeling of…

Spoiler there’s no reason we couldn’t work three days a week now. And 100 might be pushing it, but having life expectancy to 90 as well within our grass today as well. We have just decided not to do that.

Re: Beliefs that are true for regular software but false when applied to AI

#69

Post by luxuryballs »

CollinEMac wrote: post #59

> It’s entirely possible that some dangerous capability is hidden in ChatGPT, but nobody’s figured out the right prompt just yet. This sounds a little dramatic. The capabilities of ChatGPT are known. It generates text and images. The qualities of the content of the generated text and images is not fully known.

Yeah, and to riff off the headline, if something dangerous is connected to and taking commands from ChatGPT then you better make sure there’s a way to turn it off.

Re: Beliefs that are true for regular software but false when applied to AI

#70

Post by kazinator »

kstrauser wrote: post #65
kazinator wrote: post #2

> AIs will get more reliable over time, like old software is more reliable than new software. :) Was that a humam Freudian slip, or artificial one? Yes, old software is often more reliable than new.

Holy survivorship bias, Batman. If you think modern software is unreliable, let me introduce you to our friend, Rational Rose.

At least that project was wise enough to use Lisp for storing its project files.
Post reply on HN