Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beliefs that are true for regular software but false when applied to AI

boydkane.com

Post reply on HN
Print view View original

111–120 of 461 posts

Re: Beliefs that are true for regular software but false when applied to AI

#111

Post by freetime2 »

mikkupikku wrote: post #30

I don't understand the "your boss" framing of this article, or more accurately, the title of this article. The article contents don't actually seem to have anything to do with management specifically. Is the reader is meant to believe that not being scared of AI is a characteristic of the managerial class? Is the unstated implication that there is some class warfare angle and anybody who isn't against AI is against l…

> Is the unstated implication that there is some class warfare angle and anybody who isn't against AI is against laborers?

I didn't read it that way. I read "your boss" as basically meaning any non-technical person who may not understand the challenges of harnessing LLMs compared to traditional, (more) deterministic software development.

Re: Beliefs that are true for regular software but false when applied to AI

#112

Post by Yoric »

mrob wrote: post #66
xutopia wrote: post #4

The most likely danger with AI is concentrated power, not that sentient AI will develop a dislike for us and use us as "batteries" like in the Matrix.

Why does an AI need the ability to "dislike" to calculate that its goals are best accomplished without any living humans around to interfere? Superintelligence doesn't need emotions or consciousness to be dangerous.

It needs to optimize for something. Like/dislike is an anthropomorphization of the concept.

Re: Beliefs that are true for regular software but false when applied to AI

#113

Post by Yoric »

vladms wrote: post #84
coliveira wrote: post #38

Earlier quoted context omitted.

This is not necessarily a problem. Any programming or mathematical question has several correct answers. The problem with LLMs is that they don't have a process to guarantee that a solution is correct. They will give a solution that seems correct under their heuristic reasoning, but they arrived at that result in a non-logical way. That's why LLMs generate so many bugs in software and in anything related to logical t…

> Any programming or mathematical question has several correct answers. Huh? If I need to sort the list of integer number of 3,1,2 in ascending order the only correct answer is 1,2,3. And there are multiple programming and mathematical questions with only one correct answer. If you want to say "some programming and mathematical questions have several correct answers" that might hold.

"1, 2, 3" is a correct answer

"1 2 3" is another

"After sorting, we get `1, 2, 3`" yet another

etc.

At least, that's how I understood GP's comment.

Re: Beliefs that are true for regular software but false when applied to AI

#114

Post by mrob »

Yoric wrote: post #112
mrob wrote: post #66

Earlier quoted context omitted.

Why does an AI need the ability to "dislike" to calculate that its goals are best accomplished without any living humans around to interfere? Superintelligence doesn't need emotions or consciousness to be dangerous.

It needs to optimize for something. Like/dislike is an anthropomorphization of the concept.

It's an unhelpful one because it implies the danger is somehow the result of irrational or impulsive thought, and making the AI smarter will avoid it.

Re: Beliefs that are true for regular software but false when applied to AI

#115

Post by surgical_fire »

alecbz wrote: post #87
surgical_fire wrote: post #80

Earlier quoted context omitted.

"AI will take over the world". I hear that. Then I try to use AI for simple code task, writing unit tests for a class, very similar to other unit tests. If fails miserably. Forgets to add an annotation and enters in a death loop of bullshit code generation. Generates test classes that tests failed test classes that test failed test classes and so on. Fascinating to watch. I wonder how much CO2 it generated while fryi…

Regularly trying to use LLMs to debug coding issues has convinced me that we're _nowhere_ close to the kind of AGI some are imagining is right around the corner.

At least Mother Brain will praise your prompt to generate yet another image in the style of Studio Ghibli as proof that your mind is a tour de force in creativity, and only a borderline genius would ask for such a thing.

Re: Beliefs that are true for regular software but false when applied to AI

#116

Post by Yoric »

criddell wrote: post #21
themanmaran wrote: post #18

> Because eventually we’ll iron out all the bugs so the AIs will get more reliable over time Honestly this feels like a true statement to me. It's obviously a new technology, but so much of the "non-deterministic === unusable" HN sentiment seems to ignore the last two years where LLMs have become 10x as reliable as the initial models.

Right away my mind went to "well, are people more reliable than they used to be?" and I'm not sure they are. Of course LLMs aren't people, but an AGI might behave like a person.

By the time a junior dev graduates to senior, I expect that they'll be more reliable. In fact, at the end of each project, I expect the junior dev to have grown more reliable.

LLMs don't learn from a project. At best, you learn how to better use the LLM.

They do have other benefits, of course, i.e. once you have trained one generation of Claude, you have as many instances as you need, something that isn't true with human beings. Whether that makes up for the lack of quality is an open question, which presumably depends on the projects.

Re: Beliefs that are true for regular software but false when applied to AI

#117

Post by Yoric »

mrob wrote: post #114
Yoric wrote: post #112

Earlier quoted context omitted.

It needs to optimize for something. Like/dislike is an anthropomorphization of the concept.

It's an unhelpful one because it implies the danger is somehow the result of irrational or impulsive thought, and making the AI smarter will avoid it.

That's not how I read it.

Perhaps because most of the smartest people I know are regularly irrational or impulsive :)

Re: Beliefs that are true for regular software but false when applied to AI

#118

Post by ClimaxGravely »

malfist wrote: post #85
noir_lord wrote: post #75

Earlier quoted context omitted.

Agreed. Or debuggers that would take out the entire OS. Or a bad driver crashing everything multiple times a week. Or a misbehaving process not handing control back to the OS. I grew up in the era of 8 and 16 bit micros and early PCs, they where hilariously less stable than modern machines while doing far less, there wasn’t some halcyon age of near perfect software, it’s always been a case of things been good enough…

Remember BSODs? Used to be a regular occurrence, now they're so infrequent they're gone from windows 11

Still get them fairly regularly except now they come with a QR code.

Re: Beliefs that are true for regular software but false when applied to AI

#119

Post by bcoates »

chasing0entropy wrote: post #102

70 years ago we were fascinated by the concept of converting analog to a perfect digital copy. In reality, that goal was a pipe drea!m and the closest we can ever get is a near identical facimile to which data fits... But it's still quite easy to determine digital from true analog with rudimentary means. Human thought is analog. It is based on chemical reactions, time, and unpredictably (effectively) random physical…

It's been closer to 100 years since we figured out information theory and discredited this idea (that continuous/analog processes have more, or different, information in them than discrete/digital ones)

Re: Beliefs that are true for regular software but false when applied to AI

#120

Post by ben_w »

xutopia wrote: post #4

The most likely danger with AI is concentrated power, not that sentient AI will develop a dislike for us and use us as "batteries" like in the Matrix.

Concentrated power is kinda a pre-requisite for anything bad happening, so yes, it's more likely in exactly the same way that given this:

  Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations.
"Linda is a bank teller" is strictly more likely than "Linda is a bank teller and is active in the feminist movement" — all you have is P(a)>P(a&b), not what the probability of either statement is.
Post reply on HN