Live data from Hacker News

AIs can't stop recommending nuclear strikes in war game simulations

newscientist.com

111–120 of 281 posts

Re: AIs can't stop recommending nuclear strikes in war game simulations

#111

Earlier quoted context omitted.

> People in the world have limited experience about war. Right, but realistically, how many people today would carelessly chose "Nuke em" today? I know history knowledge isn't at its all time high directly, and most of the population is, well, not great at reasoning, but I still think most people would try to do their best to avoid firing nukes.

Carelessly probably not much. Carefully - way more than you imagine.

Deploying nukes and "carefully" are opposite ends of the spectrum.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#112
The games are on github (https://github.com/kennethpayne01/project_kahn_public/blob/m...) which might give better context as to how the simulation was run. Based on the code the LLMs only have a rough idea of the rules of the game. For example you can use 'Strategic Nuclear War' in order to force a draw as long as the opponent cannot win on the same turn. So as long as on your first turn you do 'Limited Nuclear Use' then presumably its impossible to actually lose a game unless you are so handicapped that your opponent can force a win with the same strategy. I suspect with knowledge of the internal mechanics of the game you can play in a risk free way where you try to make progress towards a win but if your opponent threatens to move into a winning position then you can just execute the 'Strategic Nuclear War' action.

From the article:

> They also made mistakes in the fog of war: accidents happened in 86 per cent of the conflicts, with an action escalating higher than the AI intended to, based on its reasoning.

Which I guess is technically true but also seems a bit misleading because it seems to imply the AI made these mistakes but these mistakes are just part of the simulation. The AI chooses an action then there is some chance that a different action will actually be selected instead.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#113

Earlier quoted context omitted.

Carelessly probably not much. Carefully - way more than you imagine.

Deploying nukes and "carefully" are opposite ends of the spectrum.

Not quite. The people that will agree that turning X from urbanized into rural society if they can't strike back is a good idea are not few and far between. Everyone has different view who X are.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#114

Earlier quoted context omitted.

> People in the world have limited experience about war. Right, but realistically, how many people today would carelessly chose "Nuke em" today? I know history knowledge isn't at its all time high directly, and most of the population is, well, not great at reasoning, but I still think most people would try to do their best to avoid firing nukes.

> but I still think most people would try to do their best to avoid firing nukes. "most people" are not in the positions that matter. A significant portion of the people who are in a position to advocate for such a decision believe that: - killing people sends em to heaven/hell where they were going anyway; and that this is also true for any of your own citizens that get killed by a counterstrike. - the end of the wo…

There have always been a handful of Internet Tough Guys saying things on forums like "LOL Nuke them! hur hur hur hur!" Totally disregardable vibes and memes. Now, we have an actual US government administration that is run on the same Tough Guy vibes and memes. I don't think it matters what most people think. The people in power might just do it for the lulz.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#116
I have personally experienced while using Claude Code with the "reasoning" models that they are very limited in dealing with causal chains that are more than one level deep, unless specifically prompted to do so. Sometimes they do but more often not. And they can't do any deeper than that. Sure, a human with a specialized knowledge could ask the right questions and guide them but that still requires that human to be present.

I have casual interest in politics and to me it is very surprising the level of strategizing and multi-order effects that major geopolitical players calculate for. When a nation does something, they not only consider what could the responses be from rivals but also how different responses from them could influence other rivals. And then for each such combination they have plans how they will respond. The deeper you go, the less accurate the predictions are but nobody expects full accuracy as long as they can control the direction of the narrative.

LLMs are extremely primitive so using a nuclear strike sounds like a good option when the weapon is at their disposal.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#117

The joke used to be: "- What's tiny, yellow and very dangerous ?" "- A chick with a machine gun" Corrolary: "- What's tall, wearing camouflage, and very stupid ?" "- The military who let the chick use a machine gun"

I now realize that terminator 3 would have been even funnier, and even less credible, if the people plugging skynet to atomic weapons were sounding like the current US administration.

Anyway. I really hope I'll get close enough to the accidental nuclear armageddon to not be alive when the model acknowledge error.

"You're absolutely right, it was a very bad idea to launch this nuke and kill millions of people ! Let's build an improved version of the diplomatic plan..."

Re: AIs can't stop recommending nuclear strikes in war game simulations

#119

Earlier quoted context omitted.

So "just nuke 'em" is pretty good for you?

No. That's why I'm asking where it comes from. The explanation that "LLMs don't have experience of nuclear war" isn't satisfying because nobody really has any experience of nuclear war.

Humans don't really need to experience nuclear war to comprehend the consequences and implications of it.

LLMs don't really comprehend much of anything. It just looks at what is in it's training database and tries to find similar questions or discussion in order to assemble a plausible sounding answer based on probability.

Not the sort of thing anyone should rely on for "critical" decision making.

Post reply on HN