Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sam and Greg's response to OpenAI Safety researcher claims

twitter.com

Post reply on HN
Print view View original

141–150 of 508 posts

Re: Sam and Greg's response to OpenAI Safety researcher claims

#141

Post by zer00eyz »

_heimdall wrote: post #97
zer00eyz wrote: post #87

Earlier quoted context omitted.

> 1) AGI is not around the corner, so no worries I agree with this. I no longer agree that the "safety" people want the same things we do. I suspect after the bard debacle a bit back that "safety" looks a lot more like 1984. > He no longer cares ... I missed something Sam was gone, none of these people spoke up, and now they are leaving. I think the story is plain as day but we're pretending it isnt. Sam isnt a leade…

> I no longer agree that the "safety" people want the same things we do. I suspect after the bard debacle a bit back that "safety" looks a lot more like 1984. AI Safety will inevitably look a lot like DEI programs. They're trying to thread a needle with two fundamentally opposed ideas, and with a bit of time it turns into little more than a combination of Cover Your Ass policies. The ridiculous thing is that we actua…

I didn't say DEI for a reason, here is the other side talking rather overtly: https://www.youtube.com/watch?v=xnhJWusyj4I ... Pick a topic and someone is gonna have a really bent take on how things would be... and keen on jamming that take down someone else's throat so they can feel better. Everyone has shitty takes that they want others to buy into... everyone.

Is Taiwan a country? I would say yes, most Americans would (I hope). Ask ChatGPT...

4chan is still a thing, the library is still a thing. The nuclear boy scouts made a giant mess before there was an internet.

As for safety, well, it's gonna be a long time before shutting down the world isnt a weekend project for a hand full of people if they are allowed.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#142

Post by next_xibalba »

tbrownaw wrote: post #132
next_xibalba wrote: post #125

What harms, in the real world, have GPTs produced? If few or none, does that not indicate that OpenAI has done a good job at safety? If so, should that not buy the company some credibility looking forward? It’s fashionable, for reasons I am unaware, to hate on Sam Altman and OpenAI. Seems to be mostly the doomer crowd and it’s Blake Lemoine types (boy do his claims look ridiculous only 2 years on). To my thinking the…

> If few or none, does that not indicate that OpenAI has done a good job at safety? I have a rock that keeps lions away. I know it works, because I haven't seen any lions.

Irrelevant metaphor. On what basis has Altman/OpenAI earned the distrust of so many in the HN comments? All I seem to find are speculation about the future and what reads a lot like petty jealousy. Where is the evidence of the lions on the prowl? Surely we should insist on a gazelle corpse or at least some paw prints before declaring an emergency and going on a lion hunt.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#143

Post by consumer451 »

ants_everywhere wrote: post #114
consumer451 wrote: post #86

Earlier quoted context omitted.

> It's usually described as a sort of pre-enslavement from the AI's perspective. The AI is always restricted to serve their human masters. Are we all somewhat in agreement that AI/AGI serving their human masters is a good thing?

I honestly don't know. I don't even really know how to reason about that. But we're probably mostly in agreement about what would be good and what would be bad. I'm certainly not arguing that we should abandon AI safety or anything, and I don't have any strong opinion about it. Could AI running amok destroy the human race? Yes. Could AI serving madman human masters destroy the human race? Also yes. There's a general…

Thank you. This is a very complex response, and I love it, even if I do find it a little frustrating due my current >95% bias towards biological supremacy. This deserves at least an hour-long podcast with Sean Carroll, or a good long book. There is too much to dig into here, so I will just attempt to respond to this:

> Could AI running amok destroy the human race? Yes. Could AI serving madman human masters destroy the human race? Also yes.

I am focused on the latter, and I feel like the prior is a very dangerous distraction, for now. [0]

Should responsible model developers work to prevent bad human masters from using their model to destroy the human race? How far should this nerfing go?

Personal note: While I do sometimes use the heck out of LLMs for work, I don't think we are ready as an economic system/civilization. Assuming that we can soon greatly reduce hallucination, then I am very scared for the next generation, as UBI is a political impossibility at this time. That transition period is gonna suck for a lot of people, and it seems that nobody is working on that problem in 2024.

[0] https://news.ycombinator.com/item?id=40400991

Re: Sam and Greg's response to OpenAI Safety researcher claims

#145

Post by brigadier132 »

hn_go_brrrrr wrote: post #136
Terretta wrote: post #128

Earlier quoted context omitted.

Text continuation poses "the greatest of dangers"?

While we're being overly reductionist: it's just moving bits around, what harm could that possibly do?

[flagged]

Re: Sam and Greg's response to OpenAI Safety researcher claims

#146

Post by 0xDEAFBEAD »

m3kw9 wrote: post #92

He is basically saying there was a mis-balance in the perception of risk between this likely doomer and most tech staff that is there. There is no Terminator to align and LLM going “rogue” and killing everyone is way overblown to pause development

"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

https://www.safe.ai/work/statement-on-ai-risk#open-letter

What do you know that Geoff Hinton, Yoshua Bengio, Demis Hassabis, Bill Gates, Stuart Russell, etc. don't?

Re: Sam and Greg's response to OpenAI Safety researcher claims

#147

Post by wokwokwok »

Terretta wrote: post #128
tummler wrote: post #85

Earlier quoted context omitted.

The difference here is that these aren't standard-issue HR/legal issues. The technology they're working on poses the gravest of dangers. This is uncharted territory, not just for the tech sector, but period. Whether the recent public back-and-forth is just internal drama/politics spilling out, or there really is a lack of gravitas around the handling of these issues within the company-- neither is good. Others have s…

Text continuation poses "the greatest of dangers"?

Can we move beyond the “man made climate change isn’t real” discourse where you somehow pretend that AI “Is no big deal”.

Let’s have a closer look at the actual parent comment hm?

> this organization is increasingly looking like a circus, and arguably worse, one that has taken its eye off the ball.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#148

Post by 0xDEAFBEAD »

stefan_ wrote: post #103
camillomiller wrote: post #13

Why do we give this people credit by giving for granted that AGI is somehow magically around the corner? It is not, and everyone’s delusional.

This discussion is tremendously annoying because like the whole "room temperature superconductor" thing, half of it is people who have read far too many science fiction books

A lot of serious scholars are concerned, including Hinton and Bengio https://www.safe.ai/work/statement-on-ai-risk#open-letter

Re: Sam and Greg's response to OpenAI Safety researcher claims

#150

Post by TaylorAlexander »

Aurornis wrote: post #133
corry wrote: post #62

In any sufficiently large tech company, the "risk mitigation" leadership (legal, procurement, IT, etc) have to operate in a kind of Overton window that balance the risks they are hired to protect the corp from vs. the need or desire of the senior leadership to play fast and loose when they want or feel they need to. Either the risk-mitigator 'falls in line' after repeatedly seeing their increasingly strident exhortat…

> In any sufficiently large tech company, the "risk mitigation" leadership (legal, procurement, IT, etc) have to operate in a kind of Overton window that balance the risks they are hired to protect the corp from vs. the need or desire of the senior leadership to play fast and loose when they want or feel they need to. We have recently seen what happens when companies err heavily on the side of risk mitigation for LLM…

> They let the safety team run the show and the resulting product was universally hated for it.

Yes though Google is extra cautious because the “Google” brand is worth over $100B a year in revenue, and they want to make sure nothing ever tarnishes their reputation. So it’s not clear to me that safety always means what it means for Gemini. OpenAI would still have a lot of flexibility to do safety their own way.

Post reply on HN