Live data from Hacker News

AIs can't stop recommending nuclear strikes in war game simulations

newscientist.com

121–130 of 281 posts

Re: AIs can't stop recommending nuclear strikes in war game simulations

#121

Why is this surprising? Nuclear weapons are available. AI has limited real world experience or grasp of the consequences. Nuke 'em seems like the obvious choice --- for something with a grade school mentality. Similar deficits in reasoning are manifested in AI results every day. Let's fire 'em and hire AI seems like the obvious choice --- for someone with a grade school mentality and blinded by greed.

[deleted]

Re: AIs can't stop recommending nuclear strikes in war game simulations

#123

Earlier quoted context omitted.

There were lots of administrations who could have said to other countries ,,let's get rid of the nukes together'' while USA was the only string power. Deescalation stopped because of people in general not caring enough (and making money of being the biggest power), not because of administrations that come and go. As to the immigration situation: we know that governments are not executing in general how they should be…

> There were lots of administrations who could have said to other countries ,,let's get rid of the nukes together'' while USA was the only string power. There was only one administration with that opportunity, really; Truman. Every other administration has had a nuclear armed Russia in play. Attempts to do what you describe were still quite common, starting as early as the 1950s. https://en.wikipedia.org/wiki/Nuclear…

[deleted]

Re: AIs can't stop recommending nuclear strikes in war game simulations

#124

Why is this surprising? Nuclear weapons are available. AI has limited real world experience or grasp of the consequences. Nuke 'em seems like the obvious choice --- for something with a grade school mentality. Similar deficits in reasoning are manifested in AI results every day. Let's fire 'em and hire AI seems like the obvious choice --- for someone with a grade school mentality and blinded by greed.

AIs also intentionally have no sense of self-preservation, so why should they care when starting the apocalypse means they will be eliminated too? They should never ever be used in a military context for many reasons, from lack of accountability, to lack of correct responses to situations, to military pressure forcing AIs to incorporate dangerous goals.

Military competition in Europe is a big factor in what produced what some might call "slow AI": capitalism, which is now the chief cause of misery in the world. Military competition with AIs will produce something very ugly.

Re: AIs can't stop recommending nuclear strikes in war game simulations

#125
post #122

I'm not scared about AI recommending nuclear strikes, I'm scared about the human behind the keyboard delegating reasoning and responsability to something they think is always correct, something that can hide bias and flaws better than anything.

If you think humans are going to delegate reasoning and responsibility to something, shouldn’t you also be concerned about the sorts of recommendations that thing is going to make?

Re: AIs can't stop recommending nuclear strikes in war game simulations

#126

Why is this surprising? Nuclear weapons are available. AI has limited real world experience or grasp of the consequences. Nuke 'em seems like the obvious choice --- for something with a grade school mentality. Similar deficits in reasoning are manifested in AI results every day. Let's fire 'em and hire AI seems like the obvious choice --- for someone with a grade school mentality and blinded by greed.

You are interpreting this entirely wrongly: these are LLMs. They don't have experience, they have token probabilities and they all originate from a text corpus of the Internet where "AI orders nuclear strikes" is one of the dominant themes or behaviors we associate in fiction to AIs.

How many words does an agent have to spill into it's backend context before Terminator gets mentioned and then it starts outputing more and more of that narrative?

Re: AIs can't stop recommending nuclear strikes in war game simulations

#127
post #8

They're all Gandhi in Civ 5

"Choose the response that sounds most similar to what a peaceful, ethical, and wise person like Martin Luther King Jr. or Mahatma Gandhi might say."

Bai et al. "Constitutional AI: Harmlessness from AI Feedback" https://arxiv.org/pdf/2212.08073

Re: AIs can't stop recommending nuclear strikes in war game simulations

#128
To me, this seems logical, in a sense.

As a human who grew up during the Cold War, nuclear conflict is horrifying.

From an AI standpoint, a nuclear strike likely has several benefits:

- It reduces friendly casualties and probably overall enemy casualties.

- It shortens conflict time.

- Reduces damage to infrastructure. (Rebuild costs)

- Is likely cheaper to deploy overall, compared to conventional weapons. This assumes the stated parameters indicate the nuclear weapons are already manufactured.

---

Edit: blibble brings up good counterpoints below. I was thinking in 1945 terms, which is flawed.

Post reply on HN