Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AIs can't stop recommending nuclear strikes in war game simulations

newscientist.com

Post reply on HN
Print view View original

161–170 of 281 posts

Re: AIs can't stop recommending nuclear strikes in war game simulations

#161

Post by nick486 »

jqpabc123 wrote: post #5

Why is this surprising? Nuclear weapons are available. AI has limited real world experience or grasp of the consequences. Nuke 'em seems like the obvious choice --- for something with a grade school mentality. Similar deficits in reasoning are manifested in AI results every day. Let's fire 'em and hire AI seems like the obvious choice --- for someone with a grade school mentality and blinded by greed.

I think its also important that while people may callously say "just nuke'em", if you were to hand them a red button and tell them to go ahead and do it - most wouldn't. But that latter part doesn't end up in the training data.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#162

Post by ks2048 »

b800h wrote: post #158

Is this science? Perhaps I should submit some of the random roleplay scenarios that I've run with LLMs to New Scientist.

Yes, if you do a bunch of simulations and write-up a technical report, that is science.

https://arxiv.org/abs/2602.14740v1

Re: AIs can't stop recommending nuclear strikes in war game simulations

#164

Post by XorNot »

techblueberry wrote: post #80
embedding-shape wrote: post #42

Earlier quoted context omitted.

I swear I'm not trying to start a flame war, but I think it'd be useful/valuable to know where you're from and what country you live in, as this certainly shapes how we feel about these sort of issues. I've also been dabbled in such thought experiments with friends lately, and so far we've all landed at very different conclusions, even thought there are some reasons that it might make strategic sense at the moment.

In in the US. I mean flame away, but I’m not happy about the observation I’m making, I’m not saying “given what I would do in a video game, it justifies what people would do in real life.” I’m saying “given what I would do it a video game, I think I see more clearly the choices people are making in real life.” life shouldn’t be a video game, but I think to a lot of high level leaders trying to compartmentalize it bec…

Except this isn't an argument because "a video game" isn't a real thing.

There are a diverse range of specific video game titles, but they are incredibly broad in content and scoring system.

What specifically are you actually talking about?

Re: AIs can't stop recommending nuclear strikes in war game simulations

#165

Post by badRNG »

jerf wrote: post #134
flr03 wrote: post #122

I'm not scared about AI recommending nuclear strikes, I'm scared about the human behind the keyboard delegating reasoning and responsability to something they think is always correct, something that can hide bias and flaws better than anything.

Some of the most reassuring and scariest things you can read are about the incidents that have already occurred where computers said "launch all the nukes" and the humans refused. On the one hand, good news! We have prior art that says humans don't just launch all the nukes just because the computers or procedures say to. Bad news, it's been skin-of-our-teeth multiple times already. https://www.warhistoryonline.com/c…

We shouldn't be the least bit surprised no human has complied so far.

If they had, then we wouldn't be having this conversation. For all we know, there may be a vast multiverse of universes some with humans and we would only find ourselves having this conversation in one of the universes where no human pressed the button.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#166

Post by vanviegen »

xiphias2 wrote: post #60
embedding-shape wrote: post #30

Earlier quoted context omitted.

> People in the world have limited experience about war. Right, but realistically, how many people today would carelessly chose "Nuke em" today? I know history knowledge isn't at its all time high directly, and most of the population is, well, not great at reasoning, but I still think most people would try to do their best to avoid firing nukes.

The basic game theory of nukes is that either the world is escalating or deescalating, there's no other long term stable agreement. Maybe people don't agree with ,,nuke them'', but OK with USA starting nuclear experiments again (which USA is preparing for right bow), which is a clear escalation. Russia is waiting for USA to start the nuclear experiments to start them itself for defending itself to be able to do a cou…

> After that there will be no stopping of Japan, South Korea and Iran rightfully wanting to have their own nukes.

And I'm afraid they'll be far from the only ones...

Re: AIs can't stop recommending nuclear strikes in war game simulations

#167

Post by blobbers »

Is this something we could build into post training?

Some kind of RL portion of the code that reinforces de-escalation, dangers of war, nuclear destruction of both AI and human kind, radiation and it's dangers towards microchips, the atmosphere and bit flipping (just so the AI doesn't get cocky!)

Re: AIs can't stop recommending nuclear strikes in war game simulations

#168

Post by sheiyei »

Bender wrote: post #155
pibaker wrote: post #132

Earlier quoted context omitted.

I feel this reflects a deeper problem with letting AI do any kind of decision making. They have no real world experience. They feel no real world consequences. They have no real stake in any decision they make. Human societies get to control their members' actions by imposing real life consequences. A company can fire you, a partner can divorce you, the state can jail you, the public can shame you. None of these work…

They have no real stake in any decision they make. And they are not human. Not even a sociopath or psychopathic human. At best they might be able to estimate casualties. LLM's probably can't even reach the logic conclusion of the fictional WOPR Joshua from the movie Wargames [1]. Make LLM's win every game of tic-tac-toe and see if it reaches the same conclusion of WOPR. [1] ... Edit: (Answering my own question) From…

Which makes them so great for making difficult (often bad) decisions – it wasn't me, it was the "objective" and "neutral" "superintelligence" which I totally didn't give a suggestive prompt.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#169

Post by flr03 »

jerf wrote: post #134
flr03 wrote: post #122

I'm not scared about AI recommending nuclear strikes, I'm scared about the human behind the keyboard delegating reasoning and responsability to something they think is always correct, something that can hide bias and flaws better than anything.

Some of the most reassuring and scariest things you can read are about the incidents that have already occurred where computers said "launch all the nukes" and the humans refused. On the one hand, good news! We have prior art that says humans don't just launch all the nukes just because the computers or procedures say to. Bad news, it's been skin-of-our-teeth multiple times already. https://www.warhistoryonline.com/c…

I hope humans in charge are as wise now as they were then.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#170

Post by jqpabc123 »

pibaker wrote: post #132
jqpabc123 wrote: post #5

Why is this surprising? Nuclear weapons are available. AI has limited real world experience or grasp of the consequences. Nuke 'em seems like the obvious choice --- for something with a grade school mentality. Similar deficits in reasoning are manifested in AI results every day. Let's fire 'em and hire AI seems like the obvious choice --- for someone with a grade school mentality and blinded by greed.

I feel this reflects a deeper problem with letting AI do any kind of decision making. They have no real world experience. They feel no real world consequences. They have no real stake in any decision they make. Human societies get to control their members' actions by imposing real life consequences. A company can fire you, a partner can divorce you, the state can jail you, the public can shame you. None of these work…

Maybe the best move is to not play the game.

This is the path Apple has taken.

But the best possible move is to make money from it. Short the "Magnificent 7" stocks --- buy "SQQQ" ETF --- when the time is *right*.

Post reply on HN