Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jan Leike's OpenAI departure statement

jnnnthnn.com

Post reply on HN
Print view View original

71–80 of 83 posts

Re: Jan Leike's OpenAI departure statement

#71

Post by reducesuffering »

123yawaworht456 wrote: post #64
reducesuffering wrote: post #60

Earlier quoted context omitted.

Again, the superalignment team that Jan Leike and Ilya was working on, along with Yudkowsky's opinions, are unrelated to any DEI and "racy"-ness. You can read the Superalignment announcement and what it focuses on. The entire thing is about AGI x-risk, with a small paragraph about how there's other people's work about whatever bias and PC-ness. These are different concerns by different people. You and many others are…

my point is that all evidence I see so far strongly suggests that Yudish scifi skynet bullshit all the big AI companies pay lip service to serves no purpose but scaring senile boomers and clueless pedestrians into regulating their competitors, while in reality, the 'safety' they actually work on and dedicate resources to is focused on minimizing the likelihood of some corporate chatbot accidentally recalling a gamer…

> I just refuse to believe that people whose opinions and decisions actually matter are influenced by such fears

Well, at least I'm glad you admit it's due to your stubbornness and unwillingness to change beliefs when confronted with evidence.

Sam Altman ("Development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity"), Ilya Sutskever, Geoffrey Hinton, Yoshua Bengio, Jan Leike, Paul Christiano (creator of RLHF), Dario Amodei (Anthropic), Demis Hassabis (Google DeepMind) all believe AGI poses an existential risk to humanity.

Re: Jan Leike's OpenAI departure statement

#72

Post by 123yawaworht456 »

reducesuffering wrote: post #71
123yawaworht456 wrote: post #64

Earlier quoted context omitted.

my point is that all evidence I see so far strongly suggests that Yudish scifi skynet bullshit all the big AI companies pay lip service to serves no purpose but scaring senile boomers and clueless pedestrians into regulating their competitors, while in reality, the 'safety' they actually work on and dedicate resources to is focused on minimizing the likelihood of some corporate chatbot accidentally recalling a gamer…

> I just refuse to believe that people whose opinions and decisions actually matter are influenced by such fears Well, at least I'm glad you admit it's due to your stubbornness and unwillingness to change beliefs when confronted with evidence. Sam Altman ("Development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity"), Ilya Sutskever, Geoffrey Hinton, Yoshua Be…

is it not ironic to list Sam Altman ("Development of superhuman machine intelligence is probably the greatest threat to the continued existence of humanity") among those paragons of virtue, half a year after the internal struggle between safetyists and pragmatists at openai has been revealed? is it not naive to assume that the other virtuous men you list are not the minority, given that the vast majority of openai had sided with Sam, revealing themselves as pragmatists who valued their lavish salaries over mitigating the supposed x-risks? is it not especially ironic to present Sam as a paragon of virtue here, in the context of a safety cultist leaving openai because he realized what you do not - it's all bullshit, mirror and smoke to misdirect the press and convince the politicians to smother the smaller competitors (not backed with billions of VC and unable to lobby for concessions) with regoolations.

>But over the past few years, safety culture and processes have taken a backseat to shiny products.

you know what else happened over the past few years? openai started to make money. so while sama was making soundbites for headlines about the existential threat of AI, internally, all the useful idiots were already told to shut the fuck up.

Re: Jan Leike's OpenAI departure statement

#73

Post by staticman2 »

Atotalnoob wrote: post #22
ldayley wrote: post #3

Thank you Jan for your work and your courage to act and speak candidly about these important topics. Open question to HN: to your knowledge/experience which AGI-building companies or projects have a culture most closely aligned with keeping safety, security, privacy, etc. as high a priority as “winning the race” on this new frontier land-grab? I’d love to find and support those teams over the teams that spend more ti…

Anthropic would probably be the closest to this. AFAIK they are still for profit, but they split from OpenAI because they disagreed with the lack of safety culture from OpenAI. This is also seen in their llms which are less capable due to their safety limitations

Claude Opus doesn't appear to have more safety limitations than ChatGPT.

The older Claude 2.1 on the other hand was so ridiculously incapable of functioning due to a safety first design I'm guessing it inspired the goodie 2 parody AI. https://www.goody2.ai/

Re: Jan Leike's OpenAI departure statement

#74

Post by KennyBlanken »

amenhotep wrote: post #27

"Learn to feel the AGI" shades of > Anticipating the arrival of this all-powerful technology, Sutskever began to behave like a spiritual leader, three employees who worked with him told us. His constant, enthusiastic refrain was “feel the AGI,” a reference to the idea that the company was on the cusp of its ultimate goal. At OpenAI’s 2022 holiday party, held at the California Academy of Sciences, Sutskever led employ…

This feels like the 2022 version of Steve Ballmer's infamous, deeply cringe on-stage freakout at a Microsoft event: https://www.youtube.com/watch?v=_WW2JWIv6G8

Re: Jan Leike's OpenAI departure statement

#75

Post by jononor »

Aperocky wrote: post #19
llamaimperative wrote: post #6

This won't make a dent in the logical armor of AI optimists: [ ] If you are not intimately familiar with the development of AI, your warnings on safety can be disregarded due to your basic ignorance about the development of AI [x] If you are intimately familiar with the development of AI, your warnings on safety can be disregarded due to potential conflicts of interest and koolaid drinking Unbridled optimism lives an…

That's an over complication, how about my naive belief that LLMs (and increasing the size of) don't lead to AGI. Not saying AGI is impossible, just think the large models and the underlying statistic model beneath are not the path.

Many teams are trying to combine their ideas with LLM. Because despite their weaknesses, it seems LLMs (and related concepts such as RLFH, transformers, self-supervised learning, internet-scale datasets), have made some remarkable gains. Those team are coming from the whole spectrum of ML and AI research. And they wish to use their ideas to overcome some of the weaknesses of current day LLMs. Do you also think that none of these children can lead to AGI? Why not?

Re: Jan Leike's OpenAI departure statement

#76

Post by jononor »

PeterStuer wrote: post #45
daniel_iversen wrote: post #17

> “Over the past few months […] we were struggling for compute OpenAI literally said they were setting aside 20% of compute to ensure alignment [1] but if you read the fine print, what they said was that they are “dedicating 20% of the compute we’ve secured ‘to date’ to this effort” (emphasis mine). So if their overall compute has increased by 10x then that 20% is suddenly 2%, right? Is OpenAI going to be responsible…

Alignment with whom?

Anyone or anything at all is probably the first steps, which we still don't know how to do - even conceptually. But the first real target would be alignment with the creator of the AI, and the goals an values they set.

That is: Alignment with humanity/society as a whole is even further away. And might even be considered out-of-scope for AI research: ensuring that the AI creator (persons/organization) is aligned with society might be considered to be in the political domain.

Re: Jan Leike's OpenAI departure statement

#77

Post by jononor »

flakiness wrote: post #25

> "Learn to feel the AGI." This is flabbergasted statement to me but is probably the necessary attitude to push the AI/ML frontier, I guess. I feel old.

This is some internal reference/joke. We on the internet don't really know what it actually means.

Re: Jan Leike's OpenAI departure statement

#78

Post by jononor »

reducesuffering wrote: post #56
nen-nomad wrote: post #35

It is excellent that he is speaking out. I don't believe we can create something more intelligent than us. Smarter, yes, but not more intelligent. We will always have a deeper and more nuanced understanding of reality than our artificial counterparts. But it does not mean that AI can not be a danger.

> We will always have a deeper and more nuanced understanding of reality than our artificial counterparts. Wishful thinking already disproven by collectives (Google and OpenAI) that are already better at understanding and acting upon reality than you or any single human intelligence. These are systems, and they can soon hollow out their biological components.

Do you not consider such collectives part of we (the humans)? I can understand arguing that organizations have some superinteligence. But they are still made of / run on humans - not AI/computers?

Re: Jan Leike's OpenAI departure statement

#80

Post by reducesuffering »

jononor wrote: post #78
reducesuffering wrote: post #56

Earlier quoted context omitted.

> We will always have a deeper and more nuanced understanding of reality than our artificial counterparts. Wishful thinking already disproven by collectives (Google and OpenAI) that are already better at understanding and acting upon reality than you or any single human intelligence. These are systems, and they can soon hollow out their biological components.

Do you not consider such collectives part of we (the humans)? I can understand arguing that organizations have some superinteligence. But they are still made of / run on humans - not AI/computers?

They’re currently made up of humans, but more and more will be made of computers. In their human forms, they’re still a subversion of human values, doing things as a whole, because of incentives, that the individual humans wouldn’t do. But in the future, entire corporations will get whittled down to machine processes. An AGI decision maker at the top and each individual task replaced with robot or software agent.
Post reply on HN