Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sam Altman's Second Coming Sparks New Fears of the AI Apocalypse

wired.com

Post reply on HN
Print view View original

21–25 of 25 posts

Re: Sam Altman's Second Coming Sparks New Fears of the AI Apocalypse

#21

Post by Obscurity4340 »

BoiledCabbage wrote: post #20
Obscurity4340 wrote: post #19

Earlier quoted context omitted.

Maybe I am just daft but how is this not a simple privelege escalation issue? Just don't give it admin, what's the problem?

Because in software we've defined an environment where every sw method of gaining privilege/escalation is hardened. As a result, usually the most reliable way to get escalation is via social engineering. LLMs exist in an environment whose vulnerabke surface area is social engineering. How do you lock down a system against all possible social engineering? Oh yah and the "system" isn't any computer system somewhere, it…

Does my conflation here make any sense or have any applicabillity even tho the system is far more distributed or...I don't think I have a sufficiently robust mental model of any of this but I just don't know enough to dispute it.

Re: Sam Altman's Second Coming Sparks New Fears of the AI Apocalypse

#22

Post by voitvodder »

reducesuffering wrote: post #6
proc0 wrote: post #3

Right now, at worse, AI will have a similar impact as social media. It's definitely worth thinking about the potential harm it could cause to the different industries, but thinking it's an existential risk is still premature. We don't have any cognitive architecture that will reach self-awareness and become an unstoppable bad actor. This entire ordeal more or less proved that people are a little too paranoid.

All the top brass domain experts of these systems, like 95% besides Yann LeCunn, believe it's an existential risk. Nothing has "proved that people are a little too paranoid." Why people listen to HN prognosticators over Hinton, Bengio, Sutskever, Altman, Amodei, Hassabis, etc. is beyond me.

[dead]

Re: Sam Altman's Second Coming Sparks New Fears of the AI Apocalypse

#23

Post by baking »

zerohalo wrote: post #2

Despite the hyperbolic title, the article outlines pretty clearly the outcomes of the OpenAI debacle and where things are headed. "Mission-oriented" AI research is dead; it's just another tech race by the big boys now.

I would have appreciated more on the challenges facing the new OpenAI board. The quote at the end from Satya was a bit of a shocker.

Re: Sam Altman's Second Coming Sparks New Fears of the AI Apocalypse

#24

Post by ilaksh »

Obscurity4340 wrote: post #19
ilaksh wrote: post #4

Earlier quoted context omitted.

We don't have a dangerous AI yet, but because we are talking about an existential risk, yes we do want to start thinking about it now. AI does not have to be alive or truly conscious or anything to be dangerous. It just needs to be very effective at problem solving and have someone take the guardrails off and tell it to act in a self-interested way. Especially without humans in the loop. It seems likely that the syst…

Maybe I am just daft but how is this not a simple privelege escalation issue? Just don't give it admin, what's the problem?

There is a strong likelihood that these agent swarms will be significantly smarter and more effective at not only solving problems, but also operating in sync and spreading information and software than any human or group of humans. Put that in the context of the track record of avoiding privilege escalation we have with human actors, and the idea that these systems are connected to critical infrastructure and military assets.

Re: Sam Altman's Second Coming Sparks New Fears of the AI Apocalypse

#25

Post by Obscurity4340 »

BoiledCabbage wrote: post #20
Obscurity4340 wrote: post #19

Earlier quoted context omitted.

Maybe I am just daft but how is this not a simple privelege escalation issue? Just don't give it admin, what's the problem?

Because in software we've defined an environment where every sw method of gaining privilege/escalation is hardened. As a result, usually the most reliable way to get escalation is via social engineering. LLMs exist in an environment whose vulnerabke surface area is social engineering. How do you lock down a system against all possible social engineering? Oh yah and the "system" isn't any computer system somewhere, it…

But why does the LLM itself have "access to root" whatver that means in the OS equivalent of whatever it is they operate within in terms of the larger context?
Post reply on HN