Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jan Leike's OpenAI departure statement

jnnnthnn.com

Post reply on HN
Print view View original

51–60 of 83 posts

Re: Jan Leike's OpenAI departure statement

#51

Post by brigadier132 »

llamaimperative wrote: post #29
123yawaworht456 wrote: post #18

"sailing against the wind" is a very apt description of hardlining Yuddite philosophy when your company's models got maybe 20% better in the past two years (original GPT4 is still the best model I've dealt with to this day), while local models got 1000% better. we should all thank G-d these people weren't around during the advent of personal computing and the internet - we'd have word filters in our fucking text proc…

we should all thank G-d similar people have been generally in charge of bioengineering and nuclear development labs else we'd have... err nothing. We'd have pretty much nothing at all.

No, we only have roughly 70,000 nuclear bombs and no widespread nuclear energy. Not quiet the worst outcome but pretty close.

Re: Jan Leike's OpenAI departure statement

#52

Post by Imnimo »

I was and am very skeptical of the "superalignment" plan (which was "build an AI that does alignment research for us, and then ask it what to do"). But it's a bad look for OpenAI if they pledged 20% of their compute for this effort and then just didn't actually give them the compute.

Re: Jan Leike's OpenAI departure statement

#53

Post by Footkerchief »

skepticATX wrote: post #44
llamaimperative wrote: post #6

This won't make a dent in the logical armor of AI optimists: [ ] If you are not intimately familiar with the development of AI, your warnings on safety can be disregarded due to your basic ignorance about the development of AI [x] If you are intimately familiar with the development of AI, your warnings on safety can be disregarded due to potential conflicts of interest and koolaid drinking Unbridled optimism lives an…

All I’d like to see from AI safety folks is an empirical argument demonstrating that we’re remotely close to AGI, and that AGI is dangerous. Sorry, but sci-fi novels are not going to cut it here. If anything, the last year and a half have just supported the notion that we’re not close to AGI.

The flipside: it's equally hard for people who assume AI is safe to establish empirical criteria for safety and behavior. Neither side of the argument has a strong empirical basis, because we know of no precedent for an event like the rise of non-biological intelligence.

If AGI happens, even in retrospect, there may not be a clear line between "here is non-AGI" and "here is AGI". As far as we know, there wasn't a dividing line like this during the evolution of human intelligence.

Re: Jan Leike's OpenAI departure statement

#54

Post by Tenoke »

tptacek wrote: post #11
llamaimperative wrote: post #7

Earlier quoted context omitted.

What does "if you think AGI superintelligence is a thing" mean? If you believe it exists currently, could exist, will exist, or something else?

Any of the above. Maybe, if you really want to put some unneeded extra rigor on the analysis, add "within the lifetime of anyone born in 2024". I don't want to, like, have the argument here about it. Nobody will persuade anybody of anything. But it is not a truth universally acknowledged that AGI superintelligence is actually a thing. There are other reasons to have qualms about OpenAI! They could be misleading the m…

There's a lot of things posted on HN that I dont need to and end up not paying attention to. I respect you, but these comments of how you dont need to pay attention to this topic are at best unproductive.

Re: Jan Leike's OpenAI departure statement

#55

Post by reducesuffering »

123yawaworht456 wrote: post #18

"sailing against the wind" is a very apt description of hardlining Yuddite philosophy when your company's models got maybe 20% better in the past two years (original GPT4 is still the best model I've dealt with to this day), while local models got 1000% better. we should all thank G-d these people weren't around during the advent of personal computing and the internet - we'd have word filters in our fucking text proc…

> incomprehensibly stupid assumption

is that you think what Jan Leike was working on or "Yuddite" philosophy is in anyway supportive of DEI. These things aren't related, and you're not anywhere close to the real problem by screeching about DEI.

Re: Jan Leike's OpenAI departure statement

#56

Post by reducesuffering »

nen-nomad wrote: post #35

It is excellent that he is speaking out. I don't believe we can create something more intelligent than us. Smarter, yes, but not more intelligent. We will always have a deeper and more nuanced understanding of reality than our artificial counterparts. But it does not mean that AI can not be a danger.

> We will always have a deeper and more nuanced understanding of reality than our artificial counterparts.

Wishful thinking already disproven by collectives (Google and OpenAI) that are already better at understanding and acting upon reality than you or any single human intelligence. These are systems, and they can soon hollow out their biological components.

Re: Jan Leike's OpenAI departure statement

#57

Post by Aperocky »

ben_w wrote: post #49
Aperocky wrote: post #34

Earlier quoted context omitted.

tbh it smells like they've brought in too much to their own hype. Nothing OpenAI have done more recently suggest to me that they are closer to the AGI. They did create a bunch of other models with different input format that are probably based off pretty much the same thing. And it looks like those other models will be much more expensive to run and burn up their runway sooner.

> AGI Trouble is, everyone has a different interpretation of those words. Do you mean they're no closer to "highly autonomous systems that outperform humans at most economically valuable work", or do you mean no closer to "a general-purpose machine-learning based system, even if it's only IQ 85", or do you mean no closer to "a conscious and self-aware artificial mind", or do you mean "a super-intelligent system that…

Until they are turning a profit, I would assume they are on a runway.

Re: Jan Leike's OpenAI departure statement

#58

Post by Aperocky »

artninja1988 wrote: post #50
Aperocky wrote: post #34

Earlier quoted context omitted.

tbh it smells like they've brought in too much to their own hype. Nothing OpenAI have done more recently suggest to me that they are closer to the AGI. They did create a bunch of other models with different input format that are probably based off pretty much the same thing. And it looks like those other models will be much more expensive to run and burn up their runway sooner.

Agreed. Honestly gpt4o is a massive disappointment. They made it multimodal, but it seems as dumb as ever reasoning wise. I think it's become pretty clear that we've reached diminishing returns with LLMs and AGI isn't coming. Some of OpenAI have realized this and have moved tool full on productization. That said, I do believe some of the people there have genuinely drunk the kool aid.

And it would make sense that the compliance and safety side are the most hyped because only that would justify their continued involvement in the project.

Re: Jan Leike's OpenAI departure statement

#59

Post by 123yawaworht456 »

reducesuffering wrote: post #55
123yawaworht456 wrote: post #18

"sailing against the wind" is a very apt description of hardlining Yuddite philosophy when your company's models got maybe 20% better in the past two years (original GPT4 is still the best model I've dealt with to this day), while local models got 1000% better. we should all thank G-d these people weren't around during the advent of personal computing and the internet - we'd have word filters in our fucking text proc…

> incomprehensibly stupid assumption is that you think what Jan Leike was working on or "Yuddite" philosophy is in anyway supportive of DEI. These things aren't related, and you're not anywhere close to the real problem by screeching about DEI.

https://cdn.openai.com/papers/DALL_E_3_System_Card.pdf

very well. straight from the horse's mouth:

>When designing the red teaming process for DALL·E 3, we considered a wide range of risks3 such as:

>1. Biological, chemical, and weapon related risks

>2. Mis/disinformation risks

>3. Racy and unsolicited racy imagery

>4. Societal risks related to bias and representation

(4) is DEI bullshit verbatim, (3) is DEI bullshit de facto - we all know which side of the kulturkampf screeches about "racy" things (like images of conventionally attractive women in bikinis) in the current year.

I don't know which exact role did that exact individual play at trust/safety/ethics/fart-fart-blah-fart department over at openai, but it is painfully, very painfully obvious what are openai/microsoft/google/meta/anthropic/stability/etc afraid their models might do. in every fucking press release, they all bend over backwards to appease the kvetchers, who are ever ready, eager and willing to post scalding hot takes all over X (formerly known as twitter).

Re: Jan Leike's OpenAI departure statement

#60

Post by reducesuffering »

123yawaworht456 wrote: post #59
reducesuffering wrote: post #55

Earlier quoted context omitted.

> incomprehensibly stupid assumption is that you think what Jan Leike was working on or "Yuddite" philosophy is in anyway supportive of DEI. These things aren't related, and you're not anywhere close to the real problem by screeching about DEI.

https://cdn.openai.com/papers/DALL_E_3_System_Card.pdf very well. straight from the horse's mouth: >When designing the red teaming process for DALL·E 3, we considered a wide range of risks3 such as: >1. Biological, chemical, and weapon related risks >2. Mis/disinformation risks >3. Racy and unsolicited racy imagery >4. Societal risks related to bias and representation (4) is DEI bullshit verbatim, (3) is DEI bullshit…

Again, the superalignment team that Jan Leike and Ilya was working on, along with Yudkowsky's opinions, are unrelated to any DEI and "racy"-ness.

You can read the Superalignment announcement and what it focuses on. The entire thing is about AGI x-risk, with a small paragraph about how there's other people's work about whatever bias and PC-ness.

These are different concerns by different people. You and many others are pattern matching AGI x-risk to the AI bias people to your detriment and it's poisoning the discourse. Listen to Emmett Shear (former OpenAI/Twitch CEOs) explain this in depth: https://www.youtube.com/watch?v=jZ2xw_1_KHY&t=800s

Post reply on HN