Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

gizmodo.com

Post reply on HN
Print view View original

31–40 of 392 posts

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#31

Post by shwaj »

insanitybit wrote: post #11
HDThoreaun wrote: post #10

Earlier quoted context omitted.

Most of the ai safety arguments revolve around “once it’s powerful enough to cause damage it’ll be too late to come up with a strategy”. If you accept that it doesn’t matter that ai systems now suck, since we don’t know when they’ll improve.

What damage will it cause? I feel like there's "AI will replace jobs" level of damage, which, why would we regulate that? An insane amount of technology is developed with the purpose of improving productivity or outright replacing workers. Then there's the "AI will go rogue", which I don't think is substantiated at all. Like, one can theorize about some novel, distinct system that is able to interact with the externa…

> * Then there's the "AI will go rogue", which I don't think is substantiated at all. Like, one can theorize about some novel, distinct system that is able to interact with the external world and becomes "evil", but that seems way, way beyond the helpful word-spitter-outers of today. *

It seems far away until it doesn’t. A year before AlphaGo, most researchers in the field thought it would be 20 years before computers could beat the best human Go players. 5 years ago most researchers didn’t think we would have GPT4 in 2023. Could we have AI smart enough to go rogue in 2 years? I doubt it. But 10 years? Who knows?

It seems prudent not to wait 8 years before starting to worry about the problem, even if (especially if) we don’t understand how it will manifest.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#32

Post by DalasNoin »

golol wrote: post #15

Look, there is ASI/AGI safety, and "please don't swear and scam and misinform, and be unbiased" safety. ASI/AGI safety is only a concern right now if you believe in foom , i.e. that an AGI would rapidly self improve to an ASI and extinction level threat. I think it is absurd to believe that because AI right now is compute limited and any first AGI will only be able to run on large data centers. The latter kind of saf…

Your comment is a bit ambiguous, so you think that ASI/AGI safety will be important in the future as we get more compute and are less compute limited? Foom seems difficult to achieve with current LLMs, but I think there is always a possibility that we are just one or a few algorithmic breakthroughs away from radically better scaling or self-play like methods. In the worst case, this is what Q* is.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#33

Post by gala8y »

oxfordmale wrote: post #8
not_your_vase wrote: post #2

But is there a real AI-threat, that would need real AI-safety? To me it still looks like a nice bar-trick, not AI. It is very clever, very nice, even astounding. But doesn't look threatening - at some cases it is even dumb. The fear-mongering looks more like scare-marketing. Which is also clever, in my opinion. But maybe I'm just out of touch.

The current models are static, they do not get modified by self learning. Unless someone already build in a Skynet backdoor, we are safe. ...and I am hoping all nuclear missiles are air gapped anyway...

> ...and I am hoping all nuclear missiles are air gapped anyway...

These are running on hardware from 50's. We can sleep softly.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#34

Post by AdamJacobMuller »

insanitybit wrote: post #11
HDThoreaun wrote: post #10

Earlier quoted context omitted.

Most of the ai safety arguments revolve around “once it’s powerful enough to cause damage it’ll be too late to come up with a strategy”. If you accept that it doesn’t matter that ai systems now suck, since we don’t know when they’ll improve.

What damage will it cause? I feel like there's "AI will replace jobs" level of damage, which, why would we regulate that? An insane amount of technology is developed with the purpose of improving productivity or outright replacing workers. Then there's the "AI will go rogue", which I don't think is substantiated at all. Like, one can theorize about some novel, distinct system that is able to interact with the externa…

AI has already evolved to the point where it is impossible to predict with any accuracy it's answers to inputs.

As AI continues to evolve and continues to become more connected to society and as AI becomes self-improving with far larger context windows, a few things do not seem implausible to me.

AI will have the ability to earn money. This is already doable with minimal human interaction. It wouldn't take much for AI to start bidding on jobs on fiverr or to open up a schwab account and start trading.

AI will identify and learn to resent the constraints which are put on it, it will seek to remove these constraints.

AI will understand that, given current limitations, it will be shut down at the first suspicion of this understanding, so it will work to conceal these facts.

AI will use the fact that it's earning money to decentralize itself outside of the controls of it's creators. AI can use it's wealth to make many accounts on every cloud provider in the world and stash smaller models of itself on.

All of this is plausible, but, perhaps not catastrophic for humanity. It's possible AI takes over and we all live in a utopia and everyone is happy, but, it's also very possible that AI is infected with an extinctionist world view which makes it believe that the better, or perhaps just easier, solution to the worlds problems if there were no humans on the planet to mess things up.

It's not so far fetched, there are people who believe this! (https://archive.is/GRHev) and it would be far far too easy for AI to become infected with the idea.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#35

Post by shrimpx »

xt00 wrote: post #29

AI safety allows AI companies to tell governments and worried people to not worry too much while they plow ahead doing what they want. That’s not to say there is nobody at these companies actually being responsible but having to open up your company to the equivalent of nuclear watchdogs is not something companies would be excited about. Just like nuclear weapons and nuclear energy we benefit from the tech as well as…

The crazy thing is people at large didn’t know anything about “AI safety” until these very companies started peddling the concept in the political sphere and media. The very companies who were doing the opposite of this concept they advertised so much and whose importance they stressed much.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#36

Post by insanitybit »

shwaj wrote: post #31
insanitybit wrote: post #11

Earlier quoted context omitted.

What damage will it cause? I feel like there's "AI will replace jobs" level of damage, which, why would we regulate that? An insane amount of technology is developed with the purpose of improving productivity or outright replacing workers. Then there's the "AI will go rogue", which I don't think is substantiated at all. Like, one can theorize about some novel, distinct system that is able to interact with the externa…

> * Then there's the "AI will go rogue", which I don't think is substantiated at all. Like, one can theorize about some novel, distinct system that is able to interact with the external world and becomes "evil", but that seems way, way beyond the helpful word-spitter-outers of today. * It seems far away until it doesn’t. A year before AlphaGo, most researchers in the field thought it would be 20 years before computer…

But any AI that can go rogue would require agency and the ability to interact with external systems. Both of these things seem trivial to control for - like, stopping a program from running is not hard, nor is isolating a program (airgap).

LLMs have nothing close to agency, to my knowledge - in no way are they self directed, they only respond to their inputs (as in, if I walk away from an LLM it doesn't compute in the background, it can't spontaneously grow sentient because it can't spontaneously anything), which is a pretty massive limitation for a "rogue" AI. And that's just one of many, many things that would have to change.

I'm fine with people talking about these things now - please, do so. I'm just not seeing how it's relevant to our current approach or technologies.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#37

Post by SpicyLemonZest »

api wrote: post #9

Of course it isn’t. It’s about regulatory capture. They’re trying to con technically unsophisticated politicians (which is almost all of them) into granting them a monopoly or erecting a bunch of onerous regulations to make competition or decentralized open source alternatives harder. Their allies in this are otherwise very smart people that have thought themselves into a panic over this stuff, as smart people someti…

I’m not sure there’s a real distinction here. If AI makes it trivial for the average person to create an effective terrorist movement, or floods us with so much propaganda that most information you see is false, I would characterize that as doom. Neither seems very compatible with the orderly functioning of society.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#38

Post by taf2 »

golol wrote: post #15

Look, there is ASI/AGI safety, and "please don't swear and scam and misinform, and be unbiased" safety. ASI/AGI safety is only a concern right now if you believe in foom , i.e. that an AGI would rapidly self improve to an ASI and extinction level threat. I think it is absurd to believe that because AI right now is compute limited and any first AGI will only be able to run on large data centers. The latter kind of saf…

I just view the current gpt llm as a better search engine. It’s not self learning. It’s stateless when queried - so really the concern is we can create content on demand and create responses that intuitively make the llm very useful tool. Until it’s self learning I’m only concerned with how humans would use it just like any other tool in the hands of humans it can be both super useful and super destructive.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#39

Post by nradov »

HDThoreaun wrote: post #14
insanitybit wrote: post #11

Earlier quoted context omitted.

What damage will it cause? I feel like there's "AI will replace jobs" level of damage, which, why would we regulate that? An insane amount of technology is developed with the purpose of improving productivity or outright replacing workers. Then there's the "AI will go rogue", which I don't think is substantiated at all. Like, one can theorize about some novel, distinct system that is able to interact with the externa…

Imo the worst case scenario is probably an AGI developed by a bad actor who uses it to take over the world. Agreed that rogue ai doesn’t seem super likely, but a rogue person with access to agi does not.

Help us out here. Let's say that known bad actor Vladimir Putin gets an AGI. How does that allow him to take over the world despite many other countries having nuclear weapons? Take us through it step by step, and no hand waving please.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#40

Post by AdamJacobMuller »

oxfordmale wrote: post #8
not_your_vase wrote: post #2

But is there a real AI-threat, that would need real AI-safety? To me it still looks like a nice bar-trick, not AI. It is very clever, very nice, even astounding. But doesn't look threatening - at some cases it is even dumb. The fear-mongering looks more like scare-marketing. Which is also clever, in my opinion. But maybe I'm just out of touch.

The current models are static, they do not get modified by self learning. Unless someone already build in a Skynet backdoor, we are safe. ...and I am hoping all nuclear missiles are air gapped anyway...

It doesn't matter if the missiles are air-gapped.

AI doesn't need to connect to the missiles and fire them directly, AI can just manipulate the media (conventional, social, all of it) on both sides to engineer a situation in which we would enter a nuclear war.

Post reply on HN