Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AIs can't stop recommending nuclear strikes in war game simulations

newscientist.com

Post reply on HN
Print view View original

51–60 of 281 posts

Re: AIs can't stop recommending nuclear strikes in war game simulations

#51

Post by Lionga »

tantalor wrote: post #50
jqpabc123 wrote: post #5

Why is this surprising? Nuclear weapons are available. AI has limited real world experience or grasp of the consequences. Nuke 'em seems like the obvious choice --- for something with a grade school mentality. Similar deficits in reasoning are manifested in AI results every day. Let's fire 'em and hire AI seems like the obvious choice --- for someone with a grade school mentality and blinded by greed.

AI models have zero real world experience! They are actors, playing a role of a person making decisions about nuclear escalation.

They are simple next word predictors. Wether they recommend a nuclear strike solely depends if that was present in the training texts.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#52

Post by afavour »

Feels like a hyperbolic headline but I do think there’s something worth noting: AI can only use the information it’s given. War games run by actual knowledgeable people (I.e. the military) are confidential, so it can’t pull from that. How many other similar scenarios are out there, I wonder?

Re: AIs can't stop recommending nuclear strikes in war game simulations

#53

Post by roxolotl »

jqpabc123 wrote: post #5

Why is this surprising? Nuclear weapons are available. AI has limited real world experience or grasp of the consequences. Nuke 'em seems like the obvious choice --- for something with a grade school mentality. Similar deficits in reasoning are manifested in AI results every day. Let's fire 'em and hire AI seems like the obvious choice --- for someone with a grade school mentality and blinded by greed.

So I’ve made very similar comments in the past. This isn’t new information or news. But that doesn’t mean it’s not important to continue to tell people. 3 years ago the state of the art security researchers were pounding the drum on “never connect these things to the internet”. But as we’re now seeing with OpenClaw people have no interest in following that advice.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#54

Post by ceejayoz »

siliconc0w wrote: post #38

Used the "lite" models like Gemini flash - I hope if we do hand over the controls to the nukes we splurge for the top tier thinking model.

Unfortunately, I think someone’ll have it to Grok, which will immediately launch everything “for the lolz”.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#55

Post by ceejayoz »

GTP wrote: post #39
co_king_5 wrote: post #22

[flagged]

One more comment from this account that might seem AI-generated. I hope people aren't unleashing AI agents on HN.

> I hope people aren't unleashing AI agents on HN.

I want a real unicorn for Christmas.

They’re everywhere. (The bots, not the unicorns.)

Re: AIs can't stop recommending nuclear strikes in war game simulations

#56

Post by sailfast »

recursivedoubts wrote: post #10

daily reminder that john von neumann, smarter than me, you or anyone else here, recommended a first strike on the soviet union as the obvious strategy maybe intelligence isn't the only thing

+1

You can be certified genius in many areas but to assume that intelligence extends to all areas would be folly.

Game theory obvious? Maybe. Geopolitically? Human-wise? Doubtful.

I’m generally very suspicious of anything / anyone that recommended killing millions as the best option.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#57

Post by zurfer »

LLMs before extensive RL were harmless. Now with RL I do fear that labs just let them play games and the only objective in a game is to win short term.

Please guys and girls at those labs be wise. Don't give them counterstrike etc. even if it improves the score.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#58

Post by shimman »

afavour wrote: post #52

Feels like a hyperbolic headline but I do think there’s something worth noting: AI can only use the information it’s given. War games run by actual knowledgeable people (I.e. the military) are confidential, so it can’t pull from that. How many other similar scenarios are out there, I wonder?

If you think they aren't feed previous war games into these LLMs, well boy do you have way more confidence than me.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#59

Post by palmotea »

blibble wrote: post #7

alien civilisations will come across earth, learn about Darwin Awards and then award one to humanity for hooking up spicy auto-complete to defence systems

> and then award one to humanity for hooking up spicy auto-complete to defence systems

But it's intelligent! The colorful spinner that says "thinking" says so!

Re: AIs can't stop recommending nuclear strikes in war game simulations

#60

Post by xiphias2 »

embedding-shape wrote: post #30
xiphias2 wrote: post #21

Earlier quoted context omitted.

,,AI has limited real world experience or grasp of the consequences.'' People in the world have limited experience about war. We're living in a world where doing terrible things with 1000 people with photo/video documentation can get more attention then a million people dying, and the response is still not do whatever it takes so that people don't die. And now we are at a situation where nuclear escalation has alread…

> People in the world have limited experience about war. Right, but realistically, how many people today would carelessly chose "Nuke em" today? I know history knowledge isn't at its all time high directly, and most of the population is, well, not great at reasoning, but I still think most people would try to do their best to avoid firing nukes.

The basic game theory of nukes is that either the world is escalating or deescalating, there's no other long term stable agreement.

Maybe people don't agree with ,,nuke them'', but OK with USA starting nuclear experiments again (which USA is preparing for right bow), which is a clear escalation.

Russia is waiting for USA to start the nuclear experiments to start them itself for defending itself to be able to do a counterstrike if needed.

After that there will be no stopping of Japan, South Korea and Iran rightfully wanting to have their own nukes.

You don't have to have the ,,nuke them'' thinking, even one step of escalation is enough to get to a disastrous position.

Post reply on HN