Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sam and Greg's response to OpenAI Safety researcher claims

twitter.com

Post reply on HN
Print view View original

501–508 of 508 posts

Re: Sam and Greg's response to OpenAI Safety researcher claims

#501

Post by comp_throw7 »

Dalewyn wrote: post #492
komali2 wrote: post #461

Earlier quoted context omitted.

Do you believe any country has enough conventional ordinance to end human life on the entire planet? You must know more about this than me, I've never heard of something like this.

Any of the major military superpowers have more than enough conventional boomsticks to end all human life. It might take slightly longer than if nukes were used, but that's hardly comforting.

(For confused readers: no, they don't. Not even the nuclear stockpiles would be enough to cause human extinction, even if they were turned deliberately to that end.)

Re: Sam and Greg's response to OpenAI Safety researcher claims

#502

Post by mitthrowaway2 »

sensanaty wrote: post #430

The talk of AI exterminating us via paperclips is an annoying one to me that detracts from the real dangers that are currently unfolding: - scammers are having the time of their lives, and their capabilities are only going to get better and easier - people are actively getting dumber as they trust the hallucinated outputs blindly - regular communication is starting to break down because you can't trust that people ar…

They're both real dangers. Neither one is a distraction from the other.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#503

Post by 0xDEAFBEAD »

blksv wrote: post #465
0xDEAFBEAD wrote: post #392

Earlier quoted context omitted.

>But AI (as we have it today and in the foreseeable future) is nowhere near the definition of a species. It is an enormous _server farm_ doing series of matrix multiplications followed by nonlinear transformations, with us, humans, supplying input data (as well as hardware, electricity, and maintenance) and assigning meaning to the outputs. And we have completely no ideas of doing "AI" any other way. One gorilla says…

> "Those human brains are just synapses firing. They depend on nature to survive. Not a problem" When it comes to AI we have just a lone detached brain, not in control over anything, so that it cannot even "fire" by itself: someone has to provide its inputs. > Suppose we create a server emulation of a highly intelligent, manipulative serial killer How do you feel about this? Quite indifferent: the only field I can se…

>When it comes to AI we have just a lone detached brain, not in control over anything, so that it cannot even "fire" by itself: someone has to provide its inputs.

By assumption, this AI has already been created, so presumably someone is willing to do that -- and given its superhuman manipulation abilities, their willingness will probably not change.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#504

Post by brokenmachine »

comp_throw7 wrote: post #501
Dalewyn wrote: post #492

Earlier quoted context omitted.

Any of the major military superpowers have more than enough conventional boomsticks to end all human life. It might take slightly longer than if nukes were used, but that's hardly comforting.

(For confused readers: no, they don't. Not even the nuclear stockpiles would be enough to cause human extinction, even if they were turned deliberately to that end.)

I absolutely believe that if there was a project where every nuclear weapon was systematically used for the purpose of making humans extinct, it would be successful.

https://en.wikipedia.org/wiki/Cobalt_bomb

>After one half-life of 5.27 years, only half of the cobalt-60 will have decayed, and the dose rate in the affected area would be 5 Sv/hour. At this dose rate, a person exposed to the radiation would receive a lethal dose in 1 hour.

>After 10 half-lives (about 53 years), the dose rate would have decayed to around 10 mSv/hour. At this point, a healthy person could spend up to 4 days exposed to the fallout with no immediate effects.

People can't stay in bunkers for 20 years at a time, just to emerge into a radiated hellscape.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#505

Post by int_19h »

nl wrote: post #351
conception wrote: post #208

Earlier quoted context omitted.

https://www.theguardian.com/technology/article/2024/may/17/a... "...I also went to a panel in Palantir’s booth titled Civilian Harm Mitigation. It was led by two “privacy and civil liberties engineers” – a young man and woman who spoke exclusively in monotone. They also used countless euphemisms for bombing and death. The woman described how Palantir’s Gaia map tool lets users “nominate targets of interest” for “the…

> map tool lets users “nominate targets of interest” for “the target nomination process”. She meant it helps people choose which places get bombed. So... people are choosing who to kill? I'm reasonably sure there are historical precedents for this.

The danger here is that the model hallucinates while "simplifying" and omits important information that would prevent a strike.

Or, if you're more conspiracy-minded (which is not necessarily unwarranted with the likes of Palantir), the model can be deliberately induced into "simplifying" away collateral damage through careful prompting, providing a veneer of plausible deniability.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#506

Post by blksv »

0xDEAFBEAD wrote: post #503
blksv wrote: post #465

Earlier quoted context omitted.

> "Those human brains are just synapses firing. They depend on nature to survive. Not a problem" When it comes to AI we have just a lone detached brain, not in control over anything, so that it cannot even "fire" by itself: someone has to provide its inputs. > Suppose we create a server emulation of a highly intelligent, manipulative serial killer How do you feel about this? Quite indifferent: the only field I can se…

>When it comes to AI we have just a lone detached brain, not in control over anything, so that it cannot even "fire" by itself: someone has to provide its inputs. By assumption, this AI has already been created, so presumably someone is willing to do that -- and given its superhuman manipulation abilities, their willingness will probably not change.

Then, AI's abilities are effectively limited by those of this malicious operator who has to understand and perform what the AI suggests and feed back the results.

Re: Sam and Greg's response to OpenAI Safety researcher claims

#507

Post by 0xDEAFBEAD »

blksv wrote: post #506
0xDEAFBEAD wrote: post #503

Earlier quoted context omitted.

>When it comes to AI we have just a lone detached brain, not in control over anything, so that it cannot even "fire" by itself: someone has to provide its inputs. By assumption, this AI has already been created, so presumably someone is willing to do that -- and given its superhuman manipulation abilities, their willingness will probably not change.

Then, AI's abilities are effectively limited by those of this malicious operator who has to understand and perform what the AI suggests and feed back the results.

Not true for a couple reasons:

* If the malicious operator has a superhuman advisor, that will increase their ability

* If the emulation gets connected to the internet, it can work way faster. Many jobs can be done remotely

The broader point is: abandon wishful thinking and actually consider the possibility of a worst-case scenario

Re: Sam and Greg's response to OpenAI Safety researcher claims

#508

Post by blksv »

0xDEAFBEAD wrote: post #507
blksv wrote: post #506

Earlier quoted context omitted.

Then, AI's abilities are effectively limited by those of this malicious operator who has to understand and perform what the AI suggests and feed back the results.

Not true for a couple reasons: * If the malicious operator has a superhuman advisor, that will increase their ability * If the emulation gets connected to the internet, it can work way faster. Many jobs can be done remotely The broader point is: abandon wishful thinking and actually consider the possibility of a worst-case scenario

> If the malicious operator has a superhuman advisor, that will increase their ability

Only when it comes to information processing. The inputs may (and will) be incomplete, incorrect, ambiguously formulated... The outputs may (and will) be misunderstood. And misunderstood instructions may (and will) be poorly performed.

> If the emulation gets connected to the internet, it can work way faster. Many jobs can be done remotely

What one man has connected, some other always can disconnect... And not anything is on the internet.

> abandon wishful thinking and actually consider the possibility of a worst-case scenario

The problem with all those scenarios is equating superintelligence with omniscience and omnipotence, which is plain wrong. Physics matters.

Post reply on HN