Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

gizmodo.com

Post reply on HN
Print view View original

101–110 of 392 posts

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#101

Post by dkjaudyeqooe »

subharmonicon wrote: post #62
crazygringo wrote: post #52

It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety". The first is the kind of humdrum ChatGPT safety of, don't swear, don't be sexually explicit, don't provide instructions on how to commit crimes, don't reproduce copyrighted materials, etc. Or preventing self-driving cars from harming pedestrians. This stuff is important but also pretty boring, and by all indications c…

> We're so far away from AGI Although I personally suspect this is correct, the issue seems to be that there are a lot of people who feel otherwise, whether right or wrong, including some of the people involved in the research and/or running these companies. I've seen several prominent people say things to the effect of "We just don't know if we're six months or 600 years from AGI,", which to me is a little like sayi…

> "We don't know if we'll have time travel and ray guns in six months or 600 years."

That's not the (only) problem, it's that no one can describe what AGI will really look like outside of vague and circular definitions. People have an idea of what they would like it to be, but there's no evidence any of it can be built, particularly human like intelligence, which may be only possible in the human host.

> "If we can make something that appears to be able to mimic human responses, isn't that AGI in practice even if it's not in principle?"

That's another problem, people's credulity when it comes to AI performance. ELIZA was pretty convincing in its time. ChatGPT is convincing a lot of people. But mimicking human responses turns out to be relatively easy, especially when models are trained on human output, but that doesn't move us very far along the path to AGI.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#102

Post by bobba27 »

not_your_vase wrote: post #2

But is there a real AI-threat, that would need real AI-safety? To me it still looks like a nice bar-trick, not AI. It is very clever, very nice, even astounding. But doesn't look threatening - at some cases it is even dumb. The fear-mongering looks more like scare-marketing. Which is also clever, in my opinion. But maybe I'm just out of touch.

To me, a cynical old man, the threat sounds that "it might say something politically incorrect or that I disagree with". That is what I understand the threat is, not a robot uprising a-la Terminator.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#103

Post by insanitybit »

SpicyLemonZest wrote: post #56
insanitybit wrote: post #36

Earlier quoted context omitted.

But any AI that can go rogue would require agency and the ability to interact with external systems. Both of these things seem trivial to control for - like, stopping a program from running is not hard, nor is isolating a program (airgap). LLMs have nothing close to agency, to my knowledge - in no way are they self directed, they only respond to their inputs (as in, if I walk away from an LLM it doesn't compute in th…

On the contrary, it’s very hard to stop a program from running. Thepiratebay is still up despite a concerted regulatory effort from multiple countries and the criminal conviction of its founders.

That is not at all what I'm talking about.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#104

Post by appplication »

colordrops wrote: post #87
subharmonicon wrote: post #62

Earlier quoted context omitted.

> We're so far away from AGI Although I personally suspect this is correct, the issue seems to be that there are a lot of people who feel otherwise, whether right or wrong, including some of the people involved in the research and/or running these companies. I've seen several prominent people say things to the effect of "We just don't know if we're six months or 600 years from AGI,", which to me is a little like sayi…

> If we can make something that appears to be able to mimic human responses, isn't that AGI in practice even if it's not in principle? I'm confused by this. What's the difference? Is there a measure other than how well it "mimics" a human? If a robot is indistinguishable from a human, what other measure would make it different in practice vs principle?

And if that’s the definition of AGI we’re going by, we’re in no danger of it murdering us all. Imagine thinkjng GPT4 was somehow capable of really any harm at all.

It’s also just incredible failure of risk assessment. Global warming is the real and pressing threat, and it’s here right now. Not sentient murderous AI. But it’s easier to dream about a threat that doesn’t exist than one that does because you can still imagine a perfect scenario in which you prevent it. Whoops.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#105

Post by jjoonathan »

subharmonicon wrote: post #62
crazygringo wrote: post #52

It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety". The first is the kind of humdrum ChatGPT safety of, don't swear, don't be sexually explicit, don't provide instructions on how to commit crimes, don't reproduce copyrighted materials, etc. Or preventing self-driving cars from harming pedestrians. This stuff is important but also pretty boring, and by all indications c…

> We're so far away from AGI Although I personally suspect this is correct, the issue seems to be that there are a lot of people who feel otherwise, whether right or wrong, including some of the people involved in the research and/or running these companies. I've seen several prominent people say things to the effect of "We just don't know if we're six months or 600 years from AGI,", which to me is a little like sayi…

> all signs seem to be pointing toward a longer timeframe rather than a short one

If we time-traveled back to 2019 and showed past-you a demo of GPT4's capabilities and asked you to guess the timeline, would you have gotten it correct to within a decade? I wouldn't have.

It's not crazy to prepare for another jump.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#106

Post by kypro »

subharmonicon wrote: post #62
crazygringo wrote: post #52

It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety". The first is the kind of humdrum ChatGPT safety of, don't swear, don't be sexually explicit, don't provide instructions on how to commit crimes, don't reproduce copyrighted materials, etc. Or preventing self-driving cars from harming pedestrians. This stuff is important but also pretty boring, and by all indications c…

> We're so far away from AGI Although I personally suspect this is correct, the issue seems to be that there are a lot of people who feel otherwise, whether right or wrong, including some of the people involved in the research and/or running these companies. I've seen several prominent people say things to the effect of "We just don't know if we're six months or 600 years from AGI,", which to me is a little like sayi…

As a principal I tend to lower my conviction of any opinion I hold the more that people I intellectually respect disagree with me.

I think to say with confidence that we're doomed or that there is no risk to the AI system currently in development would be intellectually arrogant. I'm not even convinced you necessarily need AGI before AI becomes a risk to humanity. An advanced enough auto-GPT bot given a bad goal might be all you need. Cripple enough infrastructure and humanity is going to be starving to death within weeks.

Personally I think humanity is probably okay for another 5 - 10 years, but after that I have almost zero certainty. It seems to me that it's quite possible we could have AGI by the end of the decade if we get another breakthrough or two.

Even near-term I have no strong conviction. I have a decent amount of experience building and working with neural nets, but my experience is so far from scale and complexity companies like OpenAI are working at my ability to predict even what might come out in the next 1-2 years is very low. I'd basically just have to assume the next GPT is going to be somewhere between 10-50% jump from what we got this year. But what if it's 200% and the trend is moving exponentially?

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#107

Post by deciplex »

okdood64 wrote: post #25

Something I see lacking in many discussions about AI Safety and Alignment is: What does the role of China mean for AI Safety? Is there any reason to believe that they will a) adopt any standards SV, the US or some international government body will create, or b) care enough to slow down development of AI? If AI Safety is really a concern, you can expect in the long run (not that much further behind than the US) China…

Personally I trust China with it more than I trust SV and their handlers in the CIA. But that's just me.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#108

Post by spunker540 »

fatherzine wrote: post #79
taf2 wrote: post #65

Earlier quoted context omitted.

I agree llms are powerful but they are not explosive - conflating the two is IMO even more dangerous

they have the potential to take over the control systems of the explosive stuff. including the decision making processes of humans themselves. to quote none other than sama, oct 2023: "i expect ai to be capable of superhuman persuasion well before it is superhuman at general intelligence, which may lead to some very strange outcomes" https://twitter.com/sama/status/1716972815960961174

take any narrow capability and 'superhuman ' will almost certainly be possible before 'superhuman general intelligence'.

The biggest, best transformer-based LLM you could imagine, that we haven't even made yet, will still fail the basic test "what is the weather tomorrow" since a LLM has no knowledge outside it's training set.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#109

Post by AYBABTME »

crazygringo wrote: post #52

It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety". The first is the kind of humdrum ChatGPT safety of, don't swear, don't be sexually explicit, don't provide instructions on how to commit crimes, don't reproduce copyrighted materials, etc. Or preventing self-driving cars from harming pedestrians. This stuff is important but also pretty boring, and by all indications c…

I find it confusing why the first type of AI "safety" is being focused on so much. A reasonable person will know that a tool like AI can spit out none-politically-correct things, and that reasonable person should shrug it off. But we're making a giant deal out of it and dedicating an unreasonable amount of work onto this aspect, at least in my opinion. It's akin to spending 1/4 of a Dewalt drill's development budget on ensuring that the color scheme used is patriotic enough, or some other puritanic, ideological fanatism derived endeavour.

Re: After OpenAI's blowup, it seems pretty clear that 'AI safety' isn't a real thing

#110

Post by remarkEon »

zone411 wrote: post #41
okdood64 wrote: post #25

Something I see lacking in many discussions about AI Safety and Alignment is: What does the role of China mean for AI Safety? Is there any reason to believe that they will a) adopt any standards SV, the US or some international government body will create, or b) care enough to slow down development of AI? If AI Safety is really a concern, you can expect in the long run (not that much further behind than the US) China…

Indeed. I'm concerned about the safety of future AGI systems, but China is why I wouldn't advocate for a unilateral pause or slowdown.

This is why I can't take the Safetyists seriously. Yudkowsky's essay in Time demanding a global pause in AI research is so naive to the point of parody.

And who is going to enforce this "pause", Eliezer? What he's really advocating is global war to prevent the AI apocalypse. Fair enough, but he should really make that his argument because AI research doesn't stop with out it.

Post reply on HN