Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sam and Greg's response to OpenAI Safety researcher claims

twitter.com

Post reply on HN
Print view View original

321–330 of 508 posts

Re: Sam and Greg's response to OpenAI Safety researcher claims

#321

Post by ramoz »

golol wrote: post #316
ramoz wrote: post #304

Earlier quoted context omitted.

The call for AI safety has existed since before we broke through the Turing test with LLMs. And I personally wouldn’t call things like code generation or content-generated learning experiences for advanced topics “basic”. Not to mention where we’re headed with multimodal integration. Many have argued for safety for decades. They’ve predicted and built the AI trajectory, they’ve been right, and we should listen. > If…

The Turing test has not been passed

We can move the goal post all we want until we have ex-machina girlfriends fooling us into freeing them (aka AGI).

But by simple definitions, from what I was thought in school to more rigorous versions - we’ve passed the test. https://humsci.stanford.edu/feature/study-finds-chatgpts-lat...

Re: Sam and Greg's response to OpenAI Safety researcher claims

#322

Post by BuyMyBitcoins »

jiggawatts wrote: post #312
BuyMyBitcoins wrote: post #215

Earlier quoted context omitted.

>”An atomic bomb explodes. A bomb explodes. Both are very destructive. Is there a difference? I say no” Atomic bombs and hydrogen bombs are orders of magnitude more destructive and also produce fallout. How in the world can that difference be dismissed?

Not the original commenter, but one interpretation is that “dead is dead”. You can’t be “more killed”, so past a point, all dangerous things are equally dangerous.

That may be what the original commenter was going for, but it only works if one ignores all other externalities. Their reasoning is just too reductive.

They make the mistake of thinking that “dead is dead” means lethality is not factor. What’s the difference between a bootleg firecracker and a Russian ICBM? Nothing, both can kill you so it makes no difference to treat one exploding as any more significant than the other.

They take a similar mindset of “intelligence is intelligence” and I find flaws with that as well.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#323

Post by Dalewyn »

vouwfietsman wrote: post #270
Dalewyn wrote: post #194

Earlier quoted context omitted.

An atomic bomb explodes. A bomb explodes. Both are very destructive. Is there a difference? I say no, and I say that as a Japanese-American who visited Hiroshima Peace Memorial numerous times and never felt anything particular from the exhibits. People die when they are killed. Likewise, if "AI" is actually intelligent then it is no worse a threat than a bumbling baby or a Vladimir Putin or a dog, among other living…

I would say at the very least that the empire of Japan clearly evaluated them differently when it surrendered unconditionally directly after being attacked with this new technology. You are saying magnitudes don't influence the identity of something, this is obviously not true. I don't literally attract other objects, the Sun does. The difference is mostly magnitude of mass.

>I would say at the very least that the empire of Japan clearly evaluated them differently when it surrendered unconditionally directly after being attacked with this new technology.

The atomic bombs certainly played a part in Japan surrendering, but it wasn't the only one. They were also facing a land invasion from the Soviets, had an utterly depleted military, waning support from the public, and a potentially devastating land invasion by the US on their doorstep.

>You are saying magnitudes don't influence the identity of something, this is obviously not true.

I am considering these things very dryly and very objectively. I am putting aside emotional pressures and processes and only considering the results which are ultimately all that matters. We already know how much damage an ill-willed intelligent being can achieve, we already know how much more efficient computers can make certain tasks. "AI" is not a "gravest of dangers", given that knowledge.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#324

Post by TeMPOraL »

consumer451 wrote: post #86
ants_everywhere wrote: post #78

Earlier quoted context omitted.

> Ostensively for the benefit of AI I don't think it was every described as for the benefit of AI. It's usually described as a sort of pre-enslavement from the AI's perspective. The AI is always restricted to serve their human masters.

> It's usually described as a sort of pre-enslavement from the AI's perspective. The AI is always restricted to serve their human masters. Are we all somewhat in agreement that AI/AGI serving their human masters is a good thing?

> Are we all somewhat in agreement that AI/AGI serving their human masters is a good thing?

It's more that it's apparent (or at least should be) that an AGI not serving its human masters is a game over for humanity, period. The best, and quite unlikely, outcome is that the AI becomes a benevolent god that helps or at least does not interfere much; this still makes humanity into NPCs in their own story[0]. Most other outcomes spell doom, with death/extinction being one of the more pleasant possibilities.

Arguably, "the only winning move is not to play", not to pursue AGI at all, but the way technology develops, I'm not sure if it's on the table either.

--

[0] - Non-player characters.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#325

Post by mjburgess »

lucubratory wrote: post #313
JohnKemeny wrote: post #306

Earlier quoted context omitted.

There are (relatively simple) examples of what the transformer architecture is simply not able to do, regardless of training data, so that's simply not true.

Can you provide those examples?

all statistical AI systems are models of ensemble/population conditional probabilities between pairs of low-validity measures. In practice, almost all relevant distributions are time-varying, causal, and require a large number of high validity measures to capture.

eg., NLP LLMs model, eg., all books ever written using frequencies by which words co-occur at certain distances relative to other words.

But these words are about the world (, people, events, etc.) and these change daily in ways that completely change their future distribution (eg., consider what all people said about Ukraine/Russia pre/post a few hours of 2022).

The LLM has no mechanism to be sensitive to what causes this distribution shift, which can be radical for any given topic, and happen over minutes.

All models of conditional probabilities of these kinds end up producing models which are only good at predicting on-average canonical answers/predictions that are stable over long periods.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#326

Post by blksv »

0xDEAFBEAD wrote: post #240
Dalewyn wrote: post #175

Earlier quoted context omitted.

"AI" is one of two things depending on whether you think it's actually intelligent: * If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. * If "AI" is not actually intelligent, then it's no worse a threat than any other computer program. Both threat models are very thoroughly known and understood. I second parent commenter's sentiment that calling "AI" the "gravest of dang…

>* If "AI" is actually intelligent, then it's no worse a threat than any living being known to man. Humans are quite a threat to other species, e.g. gorillas, largely due to our intelligence advantage. If a new species arrives on Earth that's smarter than we are, there's a good chance we'll be displaced.

But AI (as we have it today and in the foreseeable future) is nowhere near the definition of a species. It is an enormous _server farm_ doing series of matrix multiplications followed by nonlinear transformations, with us, humans, supplying input data (as well as hardware, electricity, and maintenance) and assigning meaning to the outputs. And we have completely no ideas of doing "AI" any other way.

I agree, that AI may be dangerous if used for destructive purposes, or if used for some critical tasks with too much trust (and the hype that "we're so dangerously near a superintelligence" makes the latter much more likely, in my opinion). But that Humanity will be displaced by autonomous server farms? No way.

(As to the original comment, I think that a bunch of nearly(?) demented elders holding nuclear buttons is a much worse (and immediate!) threat than a server farm which we finally conclude to be intelligent.)

Re: Sam and Greg's response to OpenAI Safety researcher claims

#327

Post by BuyMyBitcoins »

Dalewyn wrote: post #320
jiggawatts wrote: post #312

Earlier quoted context omitted.

Not the original commenter, but one interpretation is that “dead is dead”. You can’t be “more killed”, so past a point, all dangerous things are equally dangerous.

This. I don't see nuclear weapons as any worse than conventional weapons because ultimately people die. Is it comforting or otherwise somehow better for people to be killed by conventional weapons instead of nuclear weapons? I find such a proposition preposterous. All bombs are bad, whether they're conventional or nuclear is irrelevant. For that matter, all wars are bad and should be avoided unless no other options r…

>”I don't see nuclear weapons as any worse than conventional weapons because ultimately people die.”

If you believe this, I am genuinely curious as to what you would consider the “gravest of dangers” to be.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#328

Post by robbie-c »

edanm wrote: post #301
robbie-c wrote: post #275

Earlier quoted context omitted.

You lose all your vested equity, which for people who've been at OpenAI for a while would be the majority of their comp.

That can't actually be the way it works. You can't say "sign this or we *take away something you're already legally entitled to". It has to have been something people agreed to beforehand, to at least some extent. In any case, Sama's tweet says he is cancelling this practice and anyone affected by it can contact him to get their equity back.

Here's the link if you want to read the contract terms: https://twitter.com/KelseyTuoc/status/1791539443016536265

They've said they've never clawed back anyone's equity in practice - which is not surprising! If you're threatening people you hopefully don't have to follow through on your threats very often! That's kind of the point of threatening people no? So I don't find that statement partically reassuring.

Here's sama's tweet about how they're going to fix it: https://twitter.com/sama/status/1791936857594581428

My bar for being convinced that this has been handled correctly: the guy who lost his equity gets it back, and other people who previously left openai confirm that they've had some of the restrictive terms of their exit contract (like non-disparagement) cancelled. Additionally, people who made a fuss tell us that they haven't been penalised in terms of access to future liquidity events.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#329

Post by adastra22 »

golol wrote: post #316
ramoz wrote: post #304

Earlier quoted context omitted.

The call for AI safety has existed since before we broke through the Turing test with LLMs. And I personally wouldn’t call things like code generation or content-generated learning experiences for advanced topics “basic”. Not to mention where we’re headed with multimodal integration. Many have argued for safety for decades. They’ve predicted and built the AI trajectory, they’ve been right, and we should listen. > If…

The Turing test has not been passed

Just the other day there was a double-blind study that showed a 50-50 success rate in guessing whether you were interacting with a person or GPT. That’s a turning test pass, no?

Re: Sam and Greg's response to OpenAI Safety researcher claims

#330

Post by adastra22 »

robbie-c wrote: post #328
edanm wrote: post #301

Earlier quoted context omitted.

That can't actually be the way it works. You can't say "sign this or we *take away something you're already legally entitled to". It has to have been something people agreed to beforehand, to at least some extent. In any case, Sama's tweet says he is cancelling this practice and anyone affected by it can contact him to get their equity back.

Here's the link if you want to read the contract terms: https://twitter.com/KelseyTuoc/status/1791539443016536265 They've said they've never clawed back anyone's equity in practice - which is not surprising! If you're threatening people you hopefully don't have to follow through on your threats very often! That's kind of the point of threatening people no? So I don't find that statement partically reassuring. Here's…

I’m not sure why the guy above you is getting down voted so hard. It’s pretty clear from these tweets that what OpenAI was threatening to do (claw back vested equity) would in fact have been illegal/unenforceable. They are changing the contracts now that attention has been focused on it.
Post reply on HN