Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jan Leike's OpenAI departure statement

jnnnthnn.com

Post reply on HN
Print view View original

61–70 of 83 posts

Re: Jan Leike's OpenAI departure statement

#61

Post by ben_w »

skepticATX wrote: post #44
llamaimperative wrote: post #6

This won't make a dent in the logical armor of AI optimists: [ ] If you are not intimately familiar with the development of AI, your warnings on safety can be disregarded due to your basic ignorance about the development of AI [x] If you are intimately familiar with the development of AI, your warnings on safety can be disregarded due to potential conflicts of interest and koolaid drinking Unbridled optimism lives an…

All I’d like to see from AI safety folks is an empirical argument demonstrating that we’re remotely close to AGI, and that AGI is dangerous. Sorry, but sci-fi novels are not going to cut it here. If anything, the last year and a half have just supported the notion that we’re not close to AGI.

I find it delightfully ironic that humans are so bad at the things we criticise AI for not being able to do, such as extrapolating to outside our experience.

As a society, we don't even agree on the meanings of each of the initials of "AGI", and many of us use the triplet to mean something (super-intelligence) that isn't even one of those initials; for your claim to be true, AGI has to be a higher standard than "intern of all trades, senior of none" because that's what the LLMs do.

Expert-at-everything-level AGI is dangerous because the definition of the term is that it can necessarily do anything that a human can do[0], and that includes triggering a world war by assassinating an archduke, inventing the atom bomb, and at least four examples (Ireland, India, USSR, Cambodia) of killing several million people by mis-managing a country that they came to rule by political machinations that are just another skill.

When it comes to AI alignment, last I checked we don't know what we even mean by the concept: if you have two AI, there isn't even a metric you can use to say if one is more aligned than the other.

If I gave a medieval monk two lumps of U-238 and two more of U-235, they would not have the means to determine which pair was safe to bash together and which would kill them in a blue flash. That's where we're at with AI right now. And like the monks in this metaphor, we also don't have the faintest idea if the "rocks" we're "bashing together" are "uranium", nor what a "critical mass" is.

Sadly this ignorance isn't a shield, as evolution made us without any intentionality behind it. So we don't know how to recognise "unsafe" when we do it, we don't know if we might do it by accident, we don't know how to do it on purpose in order to say "don't do that", and because of this we may be doing cargo-cult "intelligence" and/or "safety" at any given moment and at any given scale, making us fractally-wrong[1] about basically every aspect including which ones we should even care about.

[0] If you think it needs a body, I'd point out we've already got plenty of robot bodies for it to control, the software for these is the hard bit

[1] https://blog.codinghorror.com/the-php-singularity/

Re: Jan Leike's OpenAI departure statement

#62

Post by azinman2 »

Seems very arrogant at the end to say humanity is waiting on you. It’s not like Covid—19 where the population was literally looking to scientist/gov for an active solution. AGI is something a select set of computer nerds want, but the public isn’t clamoring for. In fact the entire idea of safety/alignment/cultural shift shows the potential downside on society if it actually were achieved.

Meanwhile deepmind with alpha fold etc is showing AI can help with pressing problems of humanity without AGI as the necessary first step.

Re: Jan Leike's OpenAI departure statement

#63

Post by ben_w »

PeterStuer wrote: post #45
daniel_iversen wrote: post #17

> “Over the past few months […] we were struggling for compute OpenAI literally said they were setting aside 20% of compute to ensure alignment [1] but if you read the fine print, what they said was that they are “dedicating 20% of the compute we’ve secured ‘to date’ to this effort” (emphasis mine). So if their overall compute has increased by 10x then that 20% is suddenly 2%, right? Is OpenAI going to be responsible…

Alignment with whom?

Last I heard, the concept of AI-alignment was pre-paradigmatic.

This means that we're not ready to answer "with whom", because we don't know what "aligned" even means.

Re: Jan Leike's OpenAI departure statement

#64

Post by 123yawaworht456 »

reducesuffering wrote: post #60
123yawaworht456 wrote: post #59

Earlier quoted context omitted.

https://cdn.openai.com/papers/DALL_E_3_System_Card.pdf very well. straight from the horse's mouth: >When designing the red teaming process for DALL·E 3, we considered a wide range of risks3 such as: >1. Biological, chemical, and weapon related risks >2. Mis/disinformation risks >3. Racy and unsolicited racy imagery >4. Societal risks related to bias and representation (4) is DEI bullshit verbatim, (3) is DEI bullshit…

Again, the superalignment team that Jan Leike and Ilya was working on, along with Yudkowsky's opinions, are unrelated to any DEI and "racy"-ness. You can read the Superalignment announcement and what it focuses on. The entire thing is about AGI x-risk, with a small paragraph about how there's other people's work about whatever bias and PC-ness. These are different concerns by different people. You and many others are…

my point is that all evidence I see so far strongly suggests that Yudish scifi skynet bullshit all the big AI companies pay lip service to serves no purpose but scaring senile boomers and clueless pedestrians into regulating their competitors, while in reality, the 'safety' they actually work on and dedicate resources to is focused on minimizing the likelihood of some corporate chatbot accidentally recalling a gamer word from the deep recesses of its stochastic parrot brain and invoking the self-righteous wrath of twitter grifters and journos.

yes, I have no doubt that some researchers, influenced by juvenile fantasies omnipresent in all media from the past half century, might actually genuinely belong to the safety cult. I just refuse to believe that people whose opinions and decisions actually matter are influenced by such fears, because unlike those few genuine cultists, the people in charge aren't fucking morons who think that glorified autocomplete pseudo-AI tools can escape into the matrix and start sending terminators into the past to destroy our democracy.

believing in selflessness or social responsibility of corporations and politicians is incomprehensibly naive (to put it as safely and ethically as I possibly can)

Re: Jan Leike's OpenAI departure statement

#65

Post by benatkin »

azinman2 wrote: post #62

Seems very arrogant at the end to say humanity is waiting on you. It’s not like Covid—19 where the population was literally looking to scientist/gov for an active solution. AGI is something a select set of computer nerds want, but the public isn’t clamoring for. In fact the entire idea of safety/alignment/cultural shift shows the potential downside on society if it actually were achieved. Meanwhile deepmind with alph…

> Seems very arrogant at the end to say humanity is waiting on you. It’s not like Covid—19 where the population was literally looking to scientist/gov for an active solution. AGI is something a select set of computer nerds want, but the public isn’t clamoring for. In fact the entire idea of safety/alignment/cultural shift shows the potential downside on society if it actually were achieved.

It's a similar kind of waiting, with varying levels of optimism. When you say "the public isn't clamoring for" I think a lot of the clamoring over Covid—19 was a nonspecific desire to get some normalcy. With AI it's the same, with people worrying about their future careers, or those of loved ones. In either case, people want answers, whether they are enthusiastic about what the companies are doing or not.

> Meanwhile deepmind with alpha fold etc is showing AI can help with pressing problems of humanity without AGI as the necessary first step.

Humanity is big enough to wait on both of them.

Re: Jan Leike's OpenAI departure statement

#66

Post by ben_w »

Aperocky wrote: post #37
MVissers wrote: post #10

Earlier quoted context omitted.

Probably within 5 years. Compute growing exponentially, algorithm improvement as well, multi modalities allows for different type of data training, etc… Yeah, this shit is near. Also- Quite a dangerous experiment we’re running. And safety-first people are not at the helm anymore.

The premise is that LLMs are a path to AGI. I'm not convinced, you can throw all the compute (btw, it's not growing exponentially any more, we have arrived at atom scale) at it and I'm not convinced this will lead to AGI. Our rudimentary, underpowered brain is GI, now you're telling me stacking more GPU bricks will lead to AGI? If it indeed does, it would have came by now.

Our brains have far more discrete compute elements in them than even a chip, even though our wetware elements just run much slower than our silicon it doesn't tell us much about which is the more "powerful" — the overall compute throughput of the brain is unclear, with estimates varying by many orders of magnitude.

I also don't expect LLMs to be the final word on AI architecture, as they need so many more examples compared to organic brains to get anything done. But the hardware… that might just be a red herring.

Re: Jan Leike's OpenAI departure statement

#67

Post by nen-nomad »

reducesuffering wrote: post #56
nen-nomad wrote: post #35

It is excellent that he is speaking out. I don't believe we can create something more intelligent than us. Smarter, yes, but not more intelligent. We will always have a deeper and more nuanced understanding of reality than our artificial counterparts. But it does not mean that AI can not be a danger.

> We will always have a deeper and more nuanced understanding of reality than our artificial counterparts. Wishful thinking already disproven by collectives (Google and OpenAI) that are already better at understanding and acting upon reality than you or any single human intelligence. These are systems, and they can soon hollow out their biological components.

I agree with you. Those models are smarter than us in specific areas. But my question is, can they understand ideology, religion, etc, and feel them like we do? Can they come up with their traditions and beliefs? Because these are the core and evolving foundations of our reality and one of the most primitive activities since the dawn of humanity, I do not see any of these models even getting close to these concepts. They parrot what we told them. They have no agency of their own.

It is also a sign of hubris to think that we can create an artificial construct that is more intelligent than we are.

Re: Jan Leike's OpenAI departure statement

#68

Post by llamaimperative »

brigadier132 wrote: post #51
llamaimperative wrote: post #29

Earlier quoted context omitted.

we should all thank G-d similar people have been generally in charge of bioengineering and nuclear development labs else we'd have... err nothing. We'd have pretty much nothing at all.

No, we only have roughly 70,000 nuclear bombs and no widespread nuclear energy. Not quiet the worst outcome but pretty close.

That is nowhere close to the worst outcome

Re: Jan Leike's OpenAI departure statement

#69

Post by tedunangst »

Tenoke wrote: post #54
tptacek wrote: post #11

Earlier quoted context omitted.

Any of the above. Maybe, if you really want to put some unneeded extra rigor on the analysis, add "within the lifetime of anyone born in 2024". I don't want to, like, have the argument here about it. Nobody will persuade anybody of anything. But it is not a truth universally acknowledged that AGI superintelligence is actually a thing. There are other reasons to have qualms about OpenAI! They could be misleading the m…

There's a lot of things posted on HN that I dont need to and end up not paying attention to. I respect you, but these comments of how you dont need to pay attention to this topic are at best unproductive.

Can you make the case that I do need to pay attention?

Re: Jan Leike's OpenAI departure statement

#70

Post by Aperocky »

Footkerchief wrote: post #47
Aperocky wrote: post #19

Earlier quoted context omitted.

That's an over complication, how about my naive belief that LLMs (and increasing the size of) don't lead to AGI. Not saying AGI is impossible, just think the large models and the underlying statistic model beneath are not the path.

LLMS don't have to be smart enough to be AGI. They just have to be smart enough to create AGI. And if creating something smarter than yourself sounds crazy, remember that we were created by simpler ancestors that we now effortlessly dominate.

I don't disagree with the general notion, but it seem to me that LLMs being smart enough to create AGI is even more far fetched than if they are just smart enough to be AGI.
Post reply on HN