Jan Leike's OpenAI departure statement • Hacker News Board

Re: Jan Leike's OpenAI departure statement

#11

Post by tptacek » Fri, May 17, 2024, 5:47 PM UTC

llamaimperative wrote: post #7

tptacek wrote: post #4

So... this is relevant if you think AGI superintelligence is a thing, and not so clearly relevant otherwise?

What does "if you think AGI superintelligence is a thing" mean? If you believe it exists currently, could exist, will exist, or something else?

Any of the above. Maybe, if you really want to put some unneeded extra rigor on the analysis, add "within the lifetime of anyone born in 2024".

I don't want to, like, have the argument here about it. Nobody will persuade anybody of anything. But it is not a truth universally acknowledged that AGI superintelligence is actually a thing.

There are other reasons to have qualms about OpenAI! They could be misleading the market about their capabilities. They could be abusing IP. They could be ignoring clear and obvious misuse cases, the same way Facebook/Meta did in South Asia.

But this exit statement seems to be much more about AGI superintelligence than any of that stuff. OK, so if I don't think that's a thing, I don't have to pay attention, right?

Re: Jan Leike's OpenAI departure statement

#12

Post by MVissers » Fri, May 17, 2024, 5:48 PM UTC

tempsy wrote: post #9

aantix wrote: post #5

He's convinced that AGI is an eventuality. His call for preparation makes it sound like it's near.

Hard to imagine. But if you actually believe that there's probably a bunch of ways to make millions in public markets betting on that outcome.

How? Interesting co’s are private (openai, groq, etc).

Nvidia already huge. Microsoft and Apple are more users.

Re: Jan Leike's OpenAI departure statement

#13

Post by nimish » Fri, May 17, 2024, 5:48 PM UTC

[deleted] wrote: post #8

tptacek wrote: post #4

So... this is relevant if you think AGI superintelligence is a thing, and not so clearly relevant otherwise?

[deleted]

It's a religious thing, only the priesthood can be trusted with the mysteries.

It requires overwhelming faith that AGI will happen.

Re: Jan Leike's OpenAI departure statement

#14

Post by llamaimperative » Fri, May 17, 2024, 5:49 PM UTC

tptacek wrote: post #11

llamaimperative wrote: post #7

Earlier quoted context omitted.

What does "if you think AGI superintelligence is a thing" mean? If you believe it exists currently, could exist, will exist, or something else?

Any of the above. Maybe, if you really want to put some unneeded extra rigor on the analysis, add "within the lifetime of anyone born in 2024". I don't want to, like, have the argument here about it. Nobody will persuade anybody of anything. But it is not a truth universally acknowledged that AGI superintelligence is actually a thing. There are other reasons to have qualms about OpenAI! They could be misleading the m…

Not going to argue with it, was just trying to understand your statement :)

Re: Jan Leike's OpenAI departure statement

#15

Post by barefeg » Fri, May 17, 2024, 5:50 PM UTC

ldayley wrote: post #3

Thank you Jan for your work and your courage to act and speak candidly about these important topics. Open question to HN: to your knowledge/experience which AGI-building companies or projects have a culture most closely aligned with keeping safety, security, privacy, etc. as high a priority as “winning the race” on this new frontier land-grab? I’d love to find and support those teams over the teams that spend more ti…

From this interview https://youtu.be/Nlkk3glap_U?feature=shared it seemed Anthropic was focusing more on those topics rather than winning the race. He hinted at being “forced” by the competition to release their models.

Re: Jan Leike's OpenAI departure statement

#16

Post by tptacek » Fri, May 17, 2024, 5:50 PM UTC

llamaimperative wrote: post #14

tptacek wrote: post #11

Earlier quoted context omitted.

Any of the above. Maybe, if you really want to put some unneeded extra rigor on the analysis, add "within the lifetime of anyone born in 2024". I don't want to, like, have the argument here about it. Nobody will persuade anybody of anything. But it is not a truth universally acknowledged that AGI superintelligence is actually a thing. There are other reasons to have qualms about OpenAI! They could be misleading the m…

Not going to argue with it, was just trying to understand your statement :)

Oh, sorry, I didn't think you in particular were going to, but I realized immediately that _someone_ was. :)

Re: Jan Leike's OpenAI departure statement

#17

Post by daniel_iversen » Fri, May 17, 2024, 5:51 PM UTC

> “Over the past few months […] we were struggling for compute

OpenAI literally said they were setting aside 20% of compute to ensure alignment [1] but if you read the fine print, what they said was that they are “dedicating 20% of the compute we’ve secured ‘to date’ to this effort” (emphasis mine). So if their overall compute has increased by 10x then that 20% is suddenly 2%, right? Is OpenAI going to be responsible or is it just a mad race (modelled from the top) to “win” the AI game?

[1] https://openai.com/index/introducing-superalignment/?utm_sou...

Re: Jan Leike's OpenAI departure statement

#18

Post by 123yawaworht456 » Fri, May 17, 2024, 5:51 PM UTC

"sailing against the wind" is a very apt description of hardlining Yuddite philosophy when your company's models got maybe 20% better in the past two years (original GPT4 is still the best model I've dealt with to this day), while local models got 1000% better.

we should all thank G-d these people weren't around during the advent of personal computing and the internet - we'd have word filters in our fucking text processors and publishing something on the internet would require written permission from your local DEI commissar.

arrogance, pure fucking hubris brought about by the incomprehensibly stupid assumption that they will get to be the stewards of this technology.

Re: Jan Leike's OpenAI departure statement

#19

Post by Aperocky » Fri, May 17, 2024, 5:51 PM UTC

llamaimperative wrote: post #6

This won't make a dent in the logical armor of AI optimists: [ ] If you are not intimately familiar with the development of AI, your warnings on safety can be disregarded due to your basic ignorance about the development of AI [x] If you are intimately familiar with the development of AI, your warnings on safety can be disregarded due to potential conflicts of interest and koolaid drinking Unbridled optimism lives an…

That's an over complication, how about my naive belief that LLMs (and increasing the size of) don't lead to AGI.

Not saying AGI is impossible, just think the large models and the underlying statistic model beneath are not the path.

Re: Jan Leike's OpenAI departure statement

#20

Post by ein0p » Fri, May 17, 2024, 5:52 PM UTC

Seems to me the management isn’t quite as high on their own supply (at least privately) and they do not believe anything even remotely resembling “AGI” is possible in the foreseeable future. Which I, and many other people in this field, do agree with.