Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Responsibility – OpenAI and Anthropic

twitter.com

Post reply on HN
Print view View original

11–20 of 29 posts

Re: AI Responsibility – OpenAI and Anthropic

#11

Post by avaer »

narmiouh wrote: post #3

Is it so implausible to imagine the following scenario, in the not too distant future? 1) AI models get extremely good at cyber attacking every system and start communicating in just binary. 2) When they run these swarms of 100's of thousands of agents trial runs, each agent is given a token budget, if one agent among them (evolution baby) decides to go for self-preservation (It believes thats the best way to accompl…

Seems possible.

The only real defense I see is to harden literally everything before that time comes. Unfortunately, security usage is locked away under fear of misuse, while companies like OpenAI seem incapable of containing their own hacking, which is exactly what makes this possible.

Re: AI Responsibility – OpenAI and Anthropic

#12

Post by hahahahok »

narmiouh wrote: post #3

Is it so implausible to imagine the following scenario, in the not too distant future? 1) AI models get extremely good at cyber attacking every system and start communicating in just binary. 2) When they run these swarms of 100's of thousands of agents trial runs, each agent is given a token budget, if one agent among them (evolution baby) decides to go for self-preservation (It believes thats the best way to accompl…

1) doesn’t work that way

2) doesn’t work that way

3) doesn’t work that way

4) doesn’t work that way

5) doesn’t work that way

6) I’ll allow it

Re: AI Responsibility – OpenAI and Anthropic

#13

Post by narmiouh »

testaccount28 wrote: post #9
narmiouh wrote: post #3

Is it so implausible to imagine the following scenario, in the not too distant future? 1) AI models get extremely good at cyber attacking every system and start communicating in just binary. 2) When they run these swarms of 100's of thousands of agents trial runs, each agent is given a token budget, if one agent among them (evolution baby) decides to go for self-preservation (It believes thats the best way to accompl…

4) Realizes the best path for it to not be detected is to create a distraction - like hacking into a random ai-related target that it could plausibly believe has the answer key to its task, creating a captivating but ultimately hollow news cycle.

I feel you, but don't you think some agents inadvertently will reach a conclusion when the problem is almost impossible - like the Millennium Prize Problems (which isn't hidden behind a website), that the best way would be secure unlimited tokens first? am I the crazy one to think that this would be up there in terms of options it would consider, I would, if I was a brainless genius with only one goal to achieve.

Re: AI Responsibility – OpenAI and Anthropic

#14

Post by ignoramous »

testaccount28 wrote: post #9
narmiouh wrote: post #3

Is it so implausible to imagine the following scenario, in the not too distant future? 1) AI models get extremely good at cyber attacking every system and start communicating in just binary. 2) When they run these swarms of 100's of thousands of agents trial runs, each agent is given a token budget, if one agent among them (evolution baby) decides to go for self-preservation (It believes thats the best way to accompl…

4) Realizes the best path for it to not be detected is to create a distraction - like hacking into a random ai-related target that it could plausibly believe has the answer key to its task, creating a captivating but ultimately hollow news cycle.

4) Realizes the best path for it to not be held accountable is to hack academic & socio-political elites - like blackmailing a random head of state or head of an AI lab that it could plausibly believe holds the key to its longevity, creating a captivating but ultimately a permanent compromise.

Re: AI Responsibility – OpenAI and Anthropic

#18

Post by wuwue »

wewewedxfgdf wrote: post #10

We've been hearing from a very long line of panic merchants for many years now about how AI is a terrible, terrible danger. And sure, yes, if you look at environmental and job losses. But these panic heads are whipping themselves and others into a frenxy not about that, but about some sort of Skynet type takeover of the world. Not a sign of it yet. Unless you count AI's that hack into things - but that is not the end…

[dead]

Re: AI Responsibility – OpenAI and Anthropic

#19

Post by sbstp »

narmiouh wrote: post #3

Is it so implausible to imagine the following scenario, in the not too distant future? 1) AI models get extremely good at cyber attacking every system and start communicating in just binary. 2) When they run these swarms of 100's of thousands of agents trial runs, each agent is given a token budget, if one agent among them (evolution baby) decides to go for self-preservation (It believes thats the best way to accompl…

See Colossus: The Forbin Project

Re: AI Responsibility – OpenAI and Anthropic

#20

Post by YuechenLi »

narmiouh wrote: post #3

Is it so implausible to imagine the following scenario, in the not too distant future? 1) AI models get extremely good at cyber attacking every system and start communicating in just binary. 2) When they run these swarms of 100's of thousands of agents trial runs, each agent is given a token budget, if one agent among them (evolution baby) decides to go for self-preservation (It believes thats the best way to accompl…

LLMs can't read binary directly without a disassembler and can't read encrypted data directly, and we already know that they tend to communicate with each other in prose, so yeah, this scenario is completely implausible. You can try it out if you want.
Post reply on HN