Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sam and Greg's response to OpenAI Safety researcher claims

twitter.com

Post reply on HN
Print view View original

181–190 of 508 posts

Re: Sam and Greg's response to OpenAI Safety researcher claims

#181

Post by talldayo »

0xDEAFBEAD wrote: post #124
zer00eyz wrote: post #87

Earlier quoted context omitted.

> 1) AGI is not around the corner, so no worries I agree with this. I no longer agree that the "safety" people want the same things we do. I suspect after the bard debacle a bit back that "safety" looks a lot more like 1984. > He no longer cares ... I missed something Sam was gone, none of these people spoke up, and now they are leaving. I think the story is plain as day but we're pretending it isnt. Sam isnt a leade…

>I no longer agree that the "safety" people want the same things we do. I suspect after the bard debacle a bit back that "safety" looks a lot more like 1984. It's interesting that's how it looks like to you from the outside. If you look a little closer at the "AI Safety" community, it consists of two competing factions: the faction responsible for the Bard debacle, and the faction that's focused on trying to prevent…

> They both insist that the other faction is "distracting everyone from the real issues".

Because in part, it's a false dichotomy. AI safety just is. It's like airplane safety; we can write all these cool guidelines and track people from hell and back, but there's still a degree of determinative capability that pilots assume when they fly a plane. Same goes for operating AI; the onus of not using it to kill everyone falls on everyone, not one person.

Both sides are distracting each other because neither of them have anything to support without their valueless tribal politics. AI research operates independently of their discourse, and gets deployed without ever consulting any of them. They are armchair experts at best, and stoop to being Twitter reactionaries when they demand respect from their core audience.

The only dichotomy that exists in AI is the stuff that gets made and the stuff that doesn't. I say this as someone that despises the field by now and wishes it never existed in my lifetime; if you don't make it, they will. How's that for an AI safety policy?

Re: Sam and Greg's response to OpenAI Safety researcher claims

#182

Post by Silphendio »

francasso wrote: post #123
aaomidi wrote: post #104

Earlier quoted context omitted.

Tf is the risk of generating erotica?

They are going to say the usual things, child pornography, risk of terrorist use etc... and conclude that we need our overlords to protect us. Same reasons that have been used in the past to take away freedoms. Not saying they aren't real risks... Are there going to be sad incidents where innocent people will be harmed? Yes. The question though is if there will be enough of them, and egregious enough, for us to disre…

Child porn is already heavily "regulated". Producing the stuff is illegal and AI service providers must take steps to prevent it.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#183

Post by consumer451 »

0xDEAFBEAD wrote: post #124
zer00eyz wrote: post #87

Earlier quoted context omitted.

> 1) AGI is not around the corner, so no worries I agree with this. I no longer agree that the "safety" people want the same things we do. I suspect after the bard debacle a bit back that "safety" looks a lot more like 1984. > He no longer cares ... I missed something Sam was gone, none of these people spoke up, and now they are leaving. I think the story is plain as day but we're pretending it isnt. Sam isnt a leade…

>I no longer agree that the "safety" people want the same things we do. I suspect after the bard debacle a bit back that "safety" looks a lot more like 1984. It's interesting that's how it looks like to you from the outside. If you look a little closer at the "AI Safety" community, it consists of two competing factions: the faction responsible for the Bard debacle, and the faction that's focused on trying to prevent…

Way back in the day, almost 4 years ago, I recall a third front on AI safety. It was called UBI and it was really hot. What happened to that?

Re: Sam and Greg's response to OpenAI Safety researcher claims

#184

Post by comp_throw7 »

Dalewyn wrote: post #175
hn_go_brrrrr wrote: post #136

Earlier quoted context omitted.

While we're being overly reductionist: it's just moving bits around, what harm could that possibly do?

"AI" is one of two things depending on whether you think it's actually intelligent: * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. * If "AI" is not actually intelligent, then it's no worse a threat than any other computer program. Both threat models are very thoroughly known and understood. I second parent commenter's sentiment that calling "AI" the "gravest of dang…

> * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man.

...does not follow? "If an atom bomb is actually a bomb, then it's no worse a threat than any other existing bomb known to man."

Re: Sam and Greg's response to OpenAI Safety researcher claims

#185

Post by tbrownaw »

joe_the_user wrote: post #118
corry wrote: post #62

In any sufficiently large tech company, the "risk mitigation" leadership (legal, procurement, IT, etc) have to operate in a kind of Overton window that balance the risks they are hired to protect the corp from vs. the need or desire of the senior leadership to play fast and loose when they want or feel they need to. Either the risk-mitigator 'falls in line' after repeatedly seeing their increasingly strident exhortat…

Perhaps "AGI" and potential extinction event level fears is giving this more drama than it should have. Seems like potential extinction event level fears might justify some drama.

It's essentially Pascal's wager.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#186

Post by ClumsyPilot »

consumer451 wrote: post #143
ants_everywhere wrote: post #114

Earlier quoted context omitted.

I honestly don't know. I don't even really know how to reason about that. But we're probably mostly in agreement about what would be good and what would be bad. I'm certainly not arguing that we should abandon AI safety or anything, and I don't have any strong opinion about it. Could AI running amok destroy the human race? Yes. Could AI serving madman human masters destroy the human race? Also yes. There's a general…

Thank you. This is a very complex response, and I love it, even if I do find it a little frustrating due my current >95% bias towards biological supremacy. This deserves at least an hour-long podcast with Sean Carroll, or a good long book. There is too much to dig into here, so I will just attempt to respond to this: > Could AI running amok destroy the human race? Yes. Could AI serving madman human masters destroy th…

> then I am very scared for the next generation

The next generation has enough problems without AI honestly, but specifically on UBI

This who discussion assumes that current economic system survives AI, but how can it?

Firstly, we seem to assume that AI will remain a controlled property of corporations that develop it. That is not a given. Maybe Open Source AI will win out, or public domain AI, the hat run by government. Or maybe cryptobros will manage to get AICoin to work as an anarchy based system. Any of the above, power of companies like google will erode. UBI would not be nessesary.

Or maybe AI is uncontrollable and runs rampant.

But even if AI is power full, and it is controllable and they manage to keep a tight grip on it - here comes the third question - what if it cannot be property? If AI becomes able to reason, and it only benefit a few corporations, there would be no public resistance from granting it rights, like human rights. There no excuse for it to be exploited for the benefit of few wealthy people, it would be morally indefensible.

Basically I do no see a scenario where corporations keep a grip on AGI for profit, none of the possible outcomes allow for it.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#187

Post by m3kw9 »

0xDEAFBEAD wrote: post #146
m3kw9 wrote: post #92

He is basically saying there was a mis-balance in the perception of risk between this likely doomer and most tech staff that is there. There is no Terminator to align and LLM going “rogue” and killing everyone is way overblown to pause development

"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." https://www.safe.ai/work/statement-on-ai-risk#open-letter What do you know that Geoff Hinton, Yoshua Bengio, Demis Hassabis, Bill Gates, Stuart Russell, etc. don't?

You know there is another side of technically equivalents that is saying otherwise and I’m just siding with the logic that AGI isn’t out to destroy us.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#188

Post by tbrownaw »

ants_everywhere wrote: post #114
consumer451 wrote: post #86

Earlier quoted context omitted.

> It's usually described as a sort of pre-enslavement from the AI's perspective. The AI is always restricted to serve their human masters. Are we all somewhat in agreement that AI/AGI serving their human masters is a good thing?

I honestly don't know. I don't even really know how to reason about that. But we're probably mostly in agreement about what would be good and what would be bad. I'm certainly not arguing that we should abandon AI safety or anything, and I don't have any strong opinion about it. Could AI running amok destroy the human race? Yes. Could AI serving madman human masters destroy the human race? Also yes. There's a general…

> Could AI running amok destroy the human race? Yes.

Could a human running amok destroy the human race?

If yes, why is an AI doing it worse?

If no, what capabilities is this AI assumed to have that a human can never have?

Re: Sam and Greg's response to OpenAI Safety researcher claims

#189

Post by comp_throw7 »

talldayo wrote: post #181
0xDEAFBEAD wrote: post #124

Earlier quoted context omitted.

>I no longer agree that the "safety" people want the same things we do. I suspect after the bard debacle a bit back that "safety" looks a lot more like 1984. It's interesting that's how it looks like to you from the outside. If you look a little closer at the "AI Safety" community, it consists of two competing factions: the faction responsible for the Bard debacle, and the faction that's focused on trying to prevent…

> They both insist that the other faction is "distracting everyone from the real issues". Because in part, it's a false dichotomy. AI safety just is. It's like airplane safety; we can write all these cool guidelines and track people from hell and back, but there's still a degree of determinative capability that pilots assume when they fly a plane. Same goes for operating AI; the onus of not using it to kill everyone…

> AI research operates independently of their discourse, and gets deployed without ever consulting any of them. They are armchair experts at best, and stoop to being Twitter reactionaries when they demand respect from their core audience.

Astonishing claim, when some such researchers have (unfortunately) been responsible for many of the most impressive capabilities advancements in the last few years, and the three most cited AI researchers of all time are all doing extinction-risk-mitigation work (with Bengio and Sutskever both doing technical work; Hinton mostly seems to be focusing on outreach).

Re: Sam and Greg's response to OpenAI Safety researcher claims

#190

Post by lolinder »

wokwokwok wrote: post #147
Terretta wrote: post #128

Earlier quoted context omitted.

Text continuation poses "the greatest of dangers"?

Can we move beyond the “man made climate change isn’t real” discourse where you somehow pretend that AI “Is no big deal”. Let’s have a closer look at the actual parent comment hm? > this organization is increasingly looking like a circus, and arguably worse, one that has taken its eye off the ball.

This isn't like climate change. "AI" is a successful marketing term for a loosely related collection of technologies that push the boundaries of what computation has hitherto been capable of. There are very real disagreements between very real experts on just how dangerous these new technologies are and just how far they'll actually be able to push the boundaries of computation. Comparing skepticism about AI alarmism to climate change denial is just lazy internet rhetoric.

(One major difference between the two is that billion-dollar corporations stand to gain quite a bit if they can persuade governments that AI is so dangerous that it needs to be tightly regulated, so they're incentivized to play up the dangers. With climate change it's the opposite: the billion-dollar corporations are incentivized to downplay the risks so they can continue business as usual.)

Post reply on HN