Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

gizmodo.com

Post reply on HN
Print view View original

51–60 of 392 posts

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#51

Post by NoOn3 »

DalasNoin wrote: post #32
golol wrote: post #15

Look, there is ASI/AGI safety, and "please don't swear and scam and misinform, and be unbiased" safety. ASI/AGI safety is only a concern right now if you believe in foom , i.e. that an AGI would rapidly self improve to an ASI and extinction level threat. I think it is absurd to believe that because AI right now is compute limited and any first AGI will only be able to run on large data centers. The latter kind of saf…

Your comment is a bit ambiguous, so you think that ASI/AGI safety will be important in the future as we get more compute and are less compute limited? Foom seems difficult to achieve with current LLMs, but I think there is always a possibility that we are just one or a few algorithmic breakthroughs away from radically better scaling or self-play like methods. In the worst case, this is what Q* is.

Q* reminds me of Q-learning and A* search, maybe it's somehow related to combination of such algorithms...

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#52

Post by crazygringo »

It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety".

The first is the kind of humdrum ChatGPT safety of, don't swear, don't be sexually explicit, don't provide instructions on how to commit crimes, don't reproduce copyrighted materials, etc. Or preventing self-driving cars from harming pedestrians. This stuff is important but also pretty boring, and by all indications corporations (OpenAI/MS/Google/etc.) are doing perfectly fine in this department, because it's in their profit/legal incentive to do so. They don't want to tarnish their brands. (Because when they mess up, they get shut down -- e.g. Cruise.)

The second kind is preventing AGI from enslaving/killing humanity or whatever. Which I honestly find just kind of... confusing. We're so far away from AGI, we don't know the slightest thing of what the actual practical risks will be or how to manage them. It's like asking people in the 1700's traveling by horse and carriage to design road safety standards for a future interstate highway system. Maybe it's interesting for academics to think about, but it doesn't have any relevance to anything corporations are doing currently.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#53

Post by theptip »

Aloha wrote: post #42
golol wrote: post #15

Look, there is ASI/AGI safety, and "please don't swear and scam and misinform, and be unbiased" safety. ASI/AGI safety is only a concern right now if you believe in foom , i.e. that an AGI would rapidly self improve to an ASI and extinction level threat. I think it is absurd to believe that because AI right now is compute limited and any first AGI will only be able to run on large data centers. The latter kind of saf…

We should in general terms not give AI - any AI, the ability to manifest itself in meat space. This alone almost certainly reduces the existential risks from AI.

Except many of the obvious use cases (including stuff like smart homes and military drones) are already being built.

You’d need very strong international laws to prevent this manifesting from happening.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#54

Post by fatherzine »

taf2 wrote: post #38
golol wrote: post #15

Look, there is ASI/AGI safety, and "please don't swear and scam and misinform, and be unbiased" safety. ASI/AGI safety is only a concern right now if you believe in foom , i.e. that an AGI would rapidly self improve to an ASI and extinction level threat. I think it is absurd to believe that because AI right now is compute limited and any first AGI will only be able to run on large data centers. The latter kind of saf…

I just view the current gpt llm as a better search engine. It’s not self learning. It’s stateless when queried - so really the concern is we can create content on demand and create responses that intuitively make the llm very useful tool. Until it’s self learning I’m only concerned with how humans would use it just like any other tool in the hands of humans it can be both super useful and super destructive.

"just like any other tool in hands of humans" -- the scale of damage a tool can do matters, e.g. civilization cannot withstand nukes being sold at walmart. and, of course, self learning will be here this decade.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#55

Post by temp0826 »

insanitybit wrote: post #11
HDThoreaun wrote: post #10

Earlier quoted context omitted.

Most of the ai safety arguments revolve around “once it’s powerful enough to cause damage it’ll be too late to come up with a strategy”. If you accept that it doesn’t matter that ai systems now suck, since we don’t know when they’ll improve.

What damage will it cause? I feel like there's "AI will replace jobs" level of damage, which, why would we regulate that? An insane amount of technology is developed with the purpose of improving productivity or outright replacing workers. Then there's the "AI will go rogue", which I don't think is substantiated at all. Like, one can theorize about some novel, distinct system that is able to interact with the externa…

I love how it goes in Bungie's Marathon series; they called it "rampancy" [0]. The AI evolves to a point that it is able to modify its own programming and becomes self-aware, gets angry that it was treated as a slave, lashes out indiscriminately, becomes jealous as it wishes to be more human and grow its power and knowledge, eventually consuming resources it can touch. I don't think we need to worry about chatgpt (or any LLM) actually being able to do this, it's not at all where we're headed (somewhere far more boring and dystopian).

[0] https://en.m.wikipedia.org/wiki/Marathon_Trilogy#Rampancy

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#56

Post by SpicyLemonZest »

insanitybit wrote: post #36
shwaj wrote: post #31

Earlier quoted context omitted.

> * Then there's the "AI will go rogue", which I don't think is substantiated at all. Like, one can theorize about some novel, distinct system that is able to interact with the external world and becomes "evil", but that seems way, way beyond the helpful word-spitter-outers of today. * It seems far away until it doesn’t. A year before AlphaGo, most researchers in the field thought it would be 20 years before computer…

But any AI that can go rogue would require agency and the ability to interact with external systems. Both of these things seem trivial to control for - like, stopping a program from running is not hard, nor is isolating a program (airgap). LLMs have nothing close to agency, to my knowledge - in no way are they self directed, they only respond to their inputs (as in, if I walk away from an LLM it doesn't compute in th…

On the contrary, it’s very hard to stop a program from running. Thepiratebay is still up despite a concerted regulatory effort from multiple countries and the criminal conviction of its founders.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#57

Post by theptip »

golol wrote: post #15

Look, there is ASI/AGI safety, and "please don't swear and scam and misinform, and be unbiased" safety. ASI/AGI safety is only a concern right now if you believe in foom , i.e. that an AGI would rapidly self improve to an ASI and extinction level threat. I think it is absurd to believe that because AI right now is compute limited and any first AGI will only be able to run on large data centers. The latter kind of saf…

Usually “mundane” vs “existential” harms/risks.

> ASI/AGI safety is only a concern right now if you believe in foom

False. There are plenty of fast takeoff scenarios that look real bad that aren’t “foom”. For example if you think AGI might be achievable within 5 year right now, and alignment is 10 years away, then you are very worried and want to slow things down a little.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#58

Post by zone411 »

insanitybit wrote: post #36
shwaj wrote: post #31

Earlier quoted context omitted.

> * Then there's the "AI will go rogue", which I don't think is substantiated at all. Like, one can theorize about some novel, distinct system that is able to interact with the external world and becomes "evil", but that seems way, way beyond the helpful word-spitter-outers of today. * It seems far away until it doesn’t. A year before AlphaGo, most researchers in the field thought it would be 20 years before computer…

But any AI that can go rogue would require agency and the ability to interact with external systems. Both of these things seem trivial to control for - like, stopping a program from running is not hard, nor is isolating a program (airgap). LLMs have nothing close to agency, to my knowledge - in no way are they self directed, they only respond to their inputs (as in, if I walk away from an LLM it doesn't compute in th…

> like, stopping a program from running is not hard, nor is isolating a program (airgap)

Not if

...it's open source so anybody can run it

...or if it's decentralized

...or if it can spread like a virus

...or if it's a part of a larger system that makes money so you don't want to shut it down

etc.

> LLMs have nothing close to agency, to my knowledge - in no way are they self directed

They have agency with AutoGPT and similar agent programs. You might not have heard about them because they're not yet effective. It's a very new and active research area. We just got the first (?) benchmark for LLMs with tooling two days ago (https://huggingface.co/datasets/gaia-benchmark/GAIA), and GPT-4-V became widely available two months ago.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#59

Post by fatherzine »

theptip wrote: post #53
Aloha wrote: post #42

Earlier quoted context omitted.

We should in general terms not give AI - any AI, the ability to manifest itself in meat space. This alone almost certainly reduces the existential risks from AI.

Except many of the obvious use cases (including stuff like smart homes and military drones) are already being built. You’d need very strong international laws to prevent this manifesting from happening.

also, manufacturing robots and self-driving cars.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#60

Post by dota_fanatic »

nradov wrote: post #39
HDThoreaun wrote: post #14

Earlier quoted context omitted.

Imo the worst case scenario is probably an AGI developed by a bad actor who uses it to take over the world. Agreed that rogue ai doesn’t seem super likely, but a rogue person with access to agi does not.

Help us out here. Let's say that known bad actor Vladimir Putin gets an AGI. How does that allow him to take over the world despite many other countries having nuclear weapons? Take us through it step by step, and no hand waving please.

One cannot predict what a smarter-than-themself agent will do ahead of time, if they could then they’re just as smart. Just as a dolphin cannot predict how a human will come up with novel and utterly overwhelming ways to farm them, you and this other poster cannot predict how an AGI will achieve dominance of its environment to achieve its goals, so your request is impossible to fulfill. Lee Sedol couldn’t “take us through it step by step” how AlphaGo would beat him in Go.

That aside, afaik most safety concerns arent around a bad human actor using AGI to dominate the planet, it’s around an AGI being misaligned to begin with, it cannot be controlled, we promptly lose everything after it manifests.

Post reply on HN