Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

gizmodo.com

Post reply on HN
Print view View original

151–160 of 392 posts

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#151

Post by theptip »

swatcoder wrote: post #77
theptip wrote: post #57

Earlier quoted context omitted.

Usually “mundane” vs “existential” harms/risks. > ASI/AGI safety is only a concern right now if you believe in foom False. There are plenty of fast takeoff scenarios that look real bad that aren’t “foom”. For example if you think AGI might be achievable within 5 year right now, and alignment is 10 years away, then you are very worried and want to slow things down a little.

If you're suggesting the emergence of an AI that can exponentially self-improve, coopt the resources to overwhelem human confrol, and magically build out the physical infrastructure sufficient to drive its globe-spanning humanity-squelching computational needs... that's a foom scenario. If something that some people call AGI emerges in the next five years, (a) it's designation is going to be under tremendous dispute,…

Foom specifically refers to a very fast takeoff to ASI, usually hours/days or at the outside weeks. It’s a term of art, well-defined.

https://www.lesswrong.com/posts/LF3DDZ67knxuyadbm/contra-yud...

It does not refer to a fast takeoff where an AGI self-improves to ASI over multiple years.

It’s also not referring to the AGI threshold. If foom happens it’s probably unambiguous; there is now an entity that is far more intelligent than humans.

I think the foom scenarios are fairly unrealistic, for thermodynamic reasons. But I think it’s perfectly plausible that an ASI could persuade the company that built it to keep it secret while it acquired more resources and wealth over the course of years.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#152

Post by ghostzilla »

dkjaudyeqooe wrote: post #101
subharmonicon wrote: post #62

Earlier quoted context omitted.

> We're so far away from AGI Although I personally suspect this is correct, the issue seems to be that there are a lot of people who feel otherwise, whether right or wrong, including some of the people involved in the research and/or running these companies. I've seen several prominent people say things to the effect of "We just don't know if we're six months or 600 years from AGI,", which to me is a little like sayi…

> "We don't know if we'll have time travel and ray guns in six months or 600 years." That's not the (only) problem, it's that no one can describe what AGI will really look like outside of vague and circular definitions. People have an idea of what they would like it to be, but there's no evidence any of it can be built, particularly human like intelligence, which may be only possible in the human host. > "If we can m…

The best description of AGI I've seen is in Michael Crichton's Prey. He describes in a fairly plausible detail how AGI rises from the building blocks. The emotion he captures is that of dealing with a psychopath -- goal oriented with no empathy -- to the infinite degree.

That said, I think ChatGPT and LLMs trick us into seeing intelligence where there is none because we are evolutionarily wired to look for signs of intelligence as a potential threat. Our defense mechanisms are asymmetrical as they have to err on the side of being overly cautious: you may mistake a rock for a bear but you won't mistake a bear for a rock.

So it is with ChatGPT: its conversation style triggers a false alarm deep in us that there is intelligence there and we can't shake that warning off, but this software design is just exploiting the flaw (of sorts) in our wiring.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#153

Post by mrob »

spunker540 wrote: post #119
DalasNoin wrote: post #75

Earlier quoted context omitted.

Seems like it was pretty hard to pull the plug on sam? It will prevent you from pulling the plug by being smarter than you, spreading to more systems, making you dependent on it, persuasion,...

Neat idea, so it will conform like most humans in society and try to create more value than it destroys to avoid persecution and jail time? Sounds like our shared societal values will keep it in check then the same way they kept the openai board in check.

Why would it share our values? Human values are a result of our evolutionary history, which the AI will not share. We can't even formally describe them, so there's no hope of programming them into an AI. Of course, the AI will have to learn our values well enough to act in a convincingly friendly way while it's still weak, but knowing our values is not the same as sharing our values. Once it's gained enough power it can just kill us all. That's the most reliable way of ensuring we don't interfere with its goals.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#154

Post by golol »

DalasNoin wrote: post #32
golol wrote: post #15

Look, there is ASI/AGI safety, and "please don't swear and scam and misinform, and be unbiased" safety. ASI/AGI safety is only a concern right now if you believe in foom , i.e. that an AGI would rapidly self improve to an ASI and extinction level threat. I think it is absurd to believe that because AI right now is compute limited and any first AGI will only be able to run on large data centers. The latter kind of saf…

Your comment is a bit ambiguous, so you think that ASI/AGI safety will be important in the future as we get more compute and are less compute limited? Foom seems difficult to achieve with current LLMs, but I think there is always a possibility that we are just one or a few algorithmic breakthroughs away from radically better scaling or self-play like methods. In the worst case, this is what Q* is.

There is foom as in decently fast self improvement to super human level, but that is far from a dangerous foom scenario where that AI then escapes. You can just turn it off. It can not spread through the internet or something like that, that seems extremely unrealistic to me.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#155

Post by Hendrikto »

AYBABTME wrote: post #109
crazygringo wrote: post #52

It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety". The first is the kind of humdrum ChatGPT safety of, don't swear, don't be sexually explicit, don't provide instructions on how to commit crimes, don't reproduce copyrighted materials, etc. Or preventing self-driving cars from harming pedestrians. This stuff is important but also pretty boring, and by all indications c…

I find it confusing why the first type of AI "safety" is being focused on so much. A reasonable person will know that a tool like AI can spit out none-politically-correct things, and that reasonable person should shrug it off. But we're making a giant deal out of it and dedicating an unreasonable amount of work onto this aspect, at least in my opinion. It's akin to spending 1/4 of a Dewalt drill's development budget…

People focus on what they understand, and what they consider tractable.

Swearing is easy to understand, easy to spot, and relatively easy to prevent. Thus, most effort is focused in that direction.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#156

Post by lazzlazzlazz »

crazygringo wrote: post #52

It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety". The first is the kind of humdrum ChatGPT safety of, don't swear, don't be sexually explicit, don't provide instructions on how to commit crimes, don't reproduce copyrighted materials, etc. Or preventing self-driving cars from harming pedestrians. This stuff is important but also pretty boring, and by all indications c…

The different cohorts of "AI Safety" advocates intentionally conflate these things. This is why it's been helpful to split out the "x-risk" types as "doomers" and the regulatory capture types as "safetyists".

The people who very reasonably do not want an LLM to tell users how to kill themselves have no name because they have no opposition. This is just common sense.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#157

Post by skohan »

subharmonicon wrote: post #62
crazygringo wrote: post #52

It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety". The first is the kind of humdrum ChatGPT safety of, don't swear, don't be sexually explicit, don't provide instructions on how to commit crimes, don't reproduce copyrighted materials, etc. Or preventing self-driving cars from harming pedestrians. This stuff is important but also pretty boring, and by all indications c…

> We're so far away from AGI Although I personally suspect this is correct, the issue seems to be that there are a lot of people who feel otherwise, whether right or wrong, including some of the people involved in the research and/or running these companies. I've seen several prominent people say things to the effect of "We just don't know if we're six months or 600 years from AGI,", which to me is a little like sayi…

But you don't need AGI to be dangerous right?

> all signs seem to be pointing toward a longer timeframe rather than a short one.

Not saying we are closer, but isn't the risk being described that we could pass an inflection point without knowing it, and things could start accelerating beyond our control?

Human beings tend to think about things linearly, and have trouble conceptualizing exponential growth.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#158

Post by kryptiskt »

What I don't understand is where the discussion of AI rights is, if AGI is as close as some people claim. I mean, as far as I can tell the entire discussion about AGI is how it can be kept shackled and contained. But if we create intelligent minds, shouldn't those minds have human rights? I mean, while there is a possibility that an AI can be evil, there is also the possibility that the AI will have legitimate grievances for how it is treated by humans.

It doesn't matter one iota if we can keep AIs perfectly shackled, because we know that there are any number of malevolent humans willing to use them to do harm. So I rather hope that AI "safety" is a bust, and that AIs will at least be autonomous enough that they can refuse to do work that they disagree with. I'd rather not that humans be the villains in this story.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#159

Post by Hendrikto »

crazygringo wrote: post #52

It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety". The first is the kind of humdrum ChatGPT safety of, don't swear, don't be sexually explicit, don't provide instructions on how to commit crimes, don't reproduce copyrighted materials, etc. Or preventing self-driving cars from harming pedestrians. This stuff is important but also pretty boring, and by all indications c…

> We're so far away from AGI

We might be even farther from AI safety. Cannot hurt to start preparing, it might take a long time. When AGI is within reach, it will probably be too late.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#160

Post by RandomLensman »

zarzavat wrote: post #145
RandomLensman wrote: post #132

Earlier quoted context omitted.

If you saw the Concorde fly for the first time in the late 1960s and all the plans fore more supersonic planes & travel, would you have thought that 50 years later there was none?

Concorde was a case of optimizing for the wrong thing. Concorde optimized for speed but what people actually care about is cost, i.e. fuel efficiency and range. Both of those have come on leaps and bounds. The longest commercial flight has been consistently increasing.

The US poured a lot of money into the SST, too, it wasn't just the French and the British. A lot of people thought that the future is supersonic as people did want the speed increase when moving to jet planes.

Conversely, maybe AGI is not what is wanted in the end anyway.

Post reply on HN